Job Title: Data Engineer (Python/R with Big Data Technologies)

Experience: 5 to 10 Years
Location: Bangalore
Work Mode: Hybrid (3 days from office per week)

Job Summary:

We are seeking a skilled and experienced Data Engineer with a strong background in Python or R, along with expertise in SQL/Hive and/or Java, Kafka, and Storm. The ideal candidate will be involved in building scalable data pipelines, transforming data, and enabling real-time analytics in a hybrid cloud environment.

Key Responsibilities:

Develop, optimize, and maintain robust data pipelines for batch and real-time processing.

Write clean, efficient, and modular code using Python or R.

Work with SQL/Hive to query and manipulate large-scale datasets.

Leverage Kafka and Storm for real-time data streaming and event processing.

Collaborate with data scientists, analysts, and engineering teams to support analytics initiatives.

Implement data quality checks and monitoring processes to ensure data reliability.

Participate in code reviews, design discussions, and Agile ceremonies.

Mandatory Skills:

Strong programming experience in Python or R.

Proficiency in SQL or Hive for data querying and transformation.

Experience with one or more of the following: Java, Kafka, Apache Storm.

Solid understanding of data architecture and ETL/ELT concepts.

Experience working with large datasets in distributed computing environments.

Preferred Qualifications:

Experience in building real-time data pipelines using Kafka or Storm.

Exposure to cloud platforms such as AWS, Azure, or GCP.

Familiarity with CI/CD practices and version control (e.g., Git).

Background in data warehousing and data modeling.

Education:

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.