Job Summary:

We are seeking a Senior Data Engineer with expertise in AWS, Databricks, and BI visualization tools to design, develop, and optimize cloud-based data solutions. The ideal candidate will have a strong background in ETL/ELT processes, data warehousing, and real-time/batch data processing, working closely with cross-functional teams to ensure efficient data management and analytics capabilities.

Key Responsibilities:

Data Engineering & Pipeline Development: Perform source-to-target data analysis and mapping for data transformation. Design, build, and optimize data pipelines for real-time and batch processing. Work with AWS data lake solutions and BI tools like Power BI for analytical reporting. Ensure data quality, integrity, and governance across platforms. Collaboration & Agile Delivery: Work closely with admins, developers, data engineers, and analysts to ensure successful data functionality delivery. Engage with project managers and scrum masters to analyze requirements and drive sprint planning. Support product backlog management and ensure alignment with business objectives. Identify and mitigate potential risks related to data engineering processes. Cloud & DevOps Practices: Follow Agile, DevOps, DataOps, and DevSecOps methodologies for delivering scalable solutions. Participate in EDP (Enterprise Data Platform) architecture roadmap activities and enhancements. Ensure teams adhere to prescribed development processes and best practices. Provide support for data engineering development activities when required.

Required Skills & Experience:

6+ years of experience in data engineering and cloud-based data solutions. Proven expertise in AWS Data Services (S3, Glue, Redshift, Lambda, etc.). Strong hands-on experience with PySpark on Databricks for large-scale data processing. Experience in building BI solutions using Power BI or similar visualization tools. Proficiency in ETL/ELT processes, data warehousing, and data architecture best practices. Familiarity with Agile methodologies, DevOps, and DataOps principles. Strong analytical, documentation, and problem-solving skills. Ability to work in cross-functional teams across multiple time zones using communication tools like MS Teams and email.