Job Title: Data Engineer

Experience Range: 6–12 Years
Hiring Location: Bangalore

Must-Have Skills:

Expert knowledge of SQL and relational databases (e.g., PostgreSQL, Redshift, TIDB, MySQL, Oracle, Teradata)

Extensive experience with big data technologies (e.g., Hadoop, Spark, Hive, Flink)

Proficiency in at least one programming language: Python, Java, or Scala

Experience with data modeling, data warehousing, and ETL pipeline development

Strong knowledge of data pipeline and workflow management tools (e.g., Airflow, Luigi, NiFi)

Experience working with cloud platforms (AWS, Azure, or GCP); AWS preferred

Hands-on experience with building streaming pipelines using Flink (preferred), Kafka, or Kinesis

Solid understanding of data governance and data security

Experience with version control (Git) and CI/CD practices

Good-to-Have Skills:

Experience with Docker and Kubernetes

Basic knowledge of machine learning workflows and MLOps

Familiarity with NoSQL databases (MongoDB, Cassandra, etc.)

Experience with data visualization tools (Tableau, Power BI)

Real-time data processing experience

Understanding of compliance frameworks like GDPR, CCPA

Experience with Infrastructure-as-Code tools (Terraform, CloudFormation)

Key Responsibilities:

Design, develop, and maintain scalable data pipelines and ETL processes

Optimize data flow and collection for cross-functional teams

Build and maintain infrastructure for optimal data extraction, transformation, and loading (ETL)

Ensure data quality, reliability, and integrity across systems

Collaborate with data scientists and analysts to integrate models and algorithms

Automate manual processes and optimize data delivery pipelines

Create and maintain clear and comprehensive technical documentation

Evaluate and integrate emerging data tools and technologies