Job Description:

The Azure Big Data Engineer is a pivotal, hands-on role tasked with the creation and maintenance of data pipelines, data storage solutions, data ingestion processes, and data processing systems. This position is essential for facilitating data-driven decision making throughout the Experian Employer Services (EES) organization. The ideal candidate will be responsible for designing, implementing, and managing comprehensive big data solutions on the Azure platform. This will enable the organization to efficiently collect, store, process, and analyze large datasets, thereby supporting informed decision making and strategic initiatives.

Responsibilities:

Design, implement, and maintain scalable and reliable data pipelines using Azure Data Factory, Databricks, and Synapse Analytics. Microsoft Fabric Develop, configure, and optimize data lakes and warehouses on Azure using services like Azure Data Lake Storage (ADLS), Azure Lakehouse, Warehouse and monitor data pipelines for performance, scalability, and reliability Collaborate with data scientists, architects, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs. Ensure that data is secure and meets all regulatory compliance standards, including role-based access control (RBAC) and data encryption in Azure environments. Develop and configure monitoring and ing mechanisms to proactively enhance performance and optimize data systems. Troubleshoot and resolve data-related issues in a timely manner. Stay up to date with the latest trends and best practices in data engineering and cloud computing.  Produce clear, concise technical documentation for all developed solutions

Requirements:  

Proven experience with Azure services (Microsoft Fabric, Azure Synapse, Azure SQL Database, Azure Key Vault, etc.). Strong proficiency in Azure data services, including Azure Data Factory, Azure Databricks, Azure Synapse Analytics and Azure Data Lake Storage. Experience in data modeling, data architecture, and implementing ETL/ELT solutions Proficiency in SQL and familiarity with other programming languages such as Python or Scala. Knowledge of data modeling, data warehousing, and big data technologies. Experience with data governance and security best practices. Excellent problem-solving skills and attention to detail.

Preferred: 

Experience with Microsoft Fabric and Azure Synapse. 
Understanding ML/AI concepts.
Azure certifications