We are seeking a highly skilled Senior Data Engineer with extensive experience in designing, building, and optimizing high-volume data pipelines. The ideal candidate will have strong expertise in Python, Databricks on Azure Cloud services, DevOps/data factory, and CI/CD tools, along with a solid understanding of AI/ML techniques and big data processing frameworks like Apache Spark and PySpark. Responsibilities: + Adhere to coding and Numerator technology standards + Build suitable automation test suites within Azure data factory + Maintain and update automation test suites as required + Carry out manual testing, load testing, exploratory testing as required + Work closely with Business Analysts and Senior Developers to consistently achieve sprint goals + Assist in estimation of sprint-by-sprint stories and tasks + Pro-actively take a responsible approach to product delivery Requirements: + 5-7 years of experience in data engineering roles + Good Python skills + Experience working with Microsoft Azure Cloud + Experience in Agile methodologies (Scrum/Kanban) + Experience with Apache Spark, PySpark, Databricks (Must haves) + Experience working with data pipeline, preferably Azure data factory. Preferred Qualifications: + Bachelor's or master's degree in computer science, Information Technology, Data Science, or a related field. + Experience working in a support focused role + Certification in relevant Data Engineering discipline or related fields.