Job Description: Manager, Data Engineering

Overview

The Manager of Data Engineering will lead a dynamic team responsible for designing, building, and maintaining scalable data platforms and solutions to support the organization’s data science, analytics, and emerging AI initiatives. This role oversees both data engineering and data science functions within the team, providing data models and infrastructure to a separate business-led Data Analysis group that builds and maintains Power BI dashboards. The manager will drive expertise in Microsoft’s Azure data warehousing technology stack and champion robust data governance, integrity, and stewardship frameworks. The role involves managing a team of 8-12 full-time employees (FTEs), including onshore and offshore resources, and advancing the maturity of data engineering and data science capabilities to align with strategic business objectives.

Key Responsibilities

Strategic Leadership and Expertise:

Provide thought leadership and serve as a subject matter expert in designing modern data warehousing, data lake, and data lakehouse architectures, leveraging Microsoft Azure technologies (Azure Synapse Analytics, Databricks, Azure Data Lakehouse, Medallion Architecture, hot/cold storage, and Parquet files).

Define and execute the data engineering and data science strategy, roadmap, and initiatives to support analytics, business intelligence, and AI-driven use cases.

Oversee the development of advanced data models and pipelines to enable data science workflows, including machine learning, predictive analytics, and AI model development.

Team Management and Mentorship:

Manage a team of 8-12 FTEs, including onshore and offshore resources, covering both data engineering and data science roles.

Mentor team members, promoting certifications (e.g., Azure Data Engineer Associate, Databricks Certified Professional, or data science certifications like Microsoft Certified: Azure AI Engineer Associate), guiding skill development, and preparing them for career growth.

Foster collaboration and effective workload balancing across global teams to ensure high performance and delivery.

Data Platform and Data Science Development:

Design, develop, and support scalable data platforms, data models, and integrations using Azure Synapse, Databricks, and Azure Data Lakehouse architectures.

Optimize data storage solutions, including hot/cold storage tiers and Parquet file formats, for performance, cost-efficiency, and scalability.

Lead data science initiatives within the team, including building and maintaining data pipelines for machine learning, feature engineering, and model deployment.

Provide robust data models and shared datasets to the business-led Data Analysis group for Power BI dashboard development.

Data Governance and Integrity:

Establish and enforce comprehensive data governance frameworks, including data stewardship, data integrity, data quality, and metadata management.

Develop and maintain a centralized data dictionary to ensure transparency and accessibility of data assets across the organization.

Manage governance and SDLC processes for data platforms and Power BI shared datasets, including release management, version control, DEV/QA refresh, and coding standards.

Ensure compliance with data privacy regulations and organizational policies.

Power BI and Analytics Support:

Oversee the Power BI service, managing the development and support of shared datasets while ensuring the business-led Data Analysis group adheres to governance standards for dashboard creation.

Collaborate with the Data Analysis group to ensure data models meet their needs for business intelligence and reporting.

Operational Excellence:

Drive cost optimization through efficient cloud resource utilization and data storage strategies.

Implement automation and LEAN processes to streamline data engineering and data science workflows.

Report progress on data engineering and data science initiatives to stakeholders, ensuring alignment with business goals.

Cross-Functional Collaboration:

Work closely with the business-led Data Analysis group and Application Development department to deliver data solutions that support organizational objectives.

Direct IT department functional and strategic planning, including budgeting, business requirements, project planning, and resource allocation within the data engineering and data science group.

Innovation and Maturity:

Advance the maturity of data engineering and data science practices, incorporating modern methodologies such as event-driven architectures, real-time data processing (e.g., Azure Event Hubs), and CI/CD for data pipelines.

Prepare the team for AI initiatives by building scalable data infrastructure and fostering data science expertise.

Stay current with industry trends in data engineering, data science, cloud technologies, and AI to drive innovation.

Other Duties:

Perform additional responsibilities as assigned to support organizational objectives.

Qualifications

Education and Experience:

Bachelor’s degree in Computer Science, Data Engineering, Data Science, or a related field; Master’s degree preferred.

8+ years of experience in data engineering and/or data science, with at least 3 years in a leadership or managerial role.

Extensive experience with Microsoft Azure data technologies, including Azure Synapse Analytics, Databricks, Azure Data Lakehouse, Medallion Architecture, hot/cold storage, and Parquet files.

Technical Skills:

Expertise in designing and implementing data warehouses, data lakes, and data lakehouses using Azure technologies.

Proficiency in ETL/ELT pipelines, data modeling, and data integration techniques.

Strong knowledge of data science practices, including machine learning, feature engineering, and model deployment.

Experience with Power BI, including managing shared datasets and supporting business-led dashboard development.

Deep understanding of data governance, data stewardship, data quality, and metadata management.

Familiarity with modern data engineering practices, such as real-time streaming, event-driven architectures, and CI/CD for data pipelines.

Leadership and Soft Skills:

Proven ability to lead and mentor diverse, global teams with both data engineering and data science responsibilities.

Strong communication skills to collaborate with technical teams, business units, and non-technical stakeholders.

Strategic mindset to align data engineering and data science initiatives with business objectives.

Experience driving efficiency through automation, cost optimization, and process improvements.

Certifications (Preferred):

Microsoft Certified: Azure Data Engineer Associate.

Databricks Certified Professional Data Engineer.

Microsoft Certified: Azure AI Engineer Associate or similar data science certifications.

Certifications in data governance (e.g., DAMA Certified Data Management Professional).

Why Join Us?

Join a forward-thinking organization committed to leveraging data, analytics, and AI to drive business success. As the Manager of Data Engineering, you will lead a talented team of data engineers and data scientists, shape the future of our data ecosystem, and deliver impactful solutions that empower data-driven decision-making and innovation.