Experience Required: 5 to 7 Years
Location: Bangalore
Primary Responsibilities:
Design, develop, and maintain scalable ETL pipelines for processing and transforming large datasets.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective data solutions.
Optimize and tune data processing workflows to improve performance and efficiency.
Implement data quality checks to ensure integrity and consistency across data sources.
Manage and maintain relational databases and data warehouses.
Leverage cloud-based data platforms (e.g., Snowflake, Databricks) for data storage and processing.
Monitor and troubleshoot data pipelines to ensure reliability and minimize downtime.
Create and maintain documentation for data engineering processes and best practices.
Required Skills & Qualifications:5 to 7 years of experience as a Data Engineer or in a similar role.
Proficiency in Apache Spark for large-scale data processing.
Strong programming skills in Python.
Advanced knowledge of SQL for data querying and manipulation.
Experience working with relational databases and building scalable ETL pipelines.
Familiarity with cloud data platforms such as Snowflake or Databricks.
Strong problem-solving skills and high attention to detail.
Excellent communication and collaboration abilities.
Desired Skills:Experience with big data technologies such as Kafka.
Understanding of data modeling and data warehousing concepts.
Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
Experience with version control systems like Git.
Knowledge of data governance and compliance requirements.