Data Engineer
Kforce
Kforce has a client in Greenwood Village, CO that is seeking a skilled Data Engineer to support a high-impact data engineering initiative focused on AWS infrastructure, Spark-based transformations, and orchestration tools. This individual contributor role requires hands-on experience building data pipelines, processing large-scale JSON messages, and deploying solutions in cloud environments. The ideal candidate has a solid foundation in data engineering within enterprise environments, strong Spark/PySpark expertise, and experience with AWS-native services and modern orchestration tools.
Responsibilities:
* Build and optimize scalable data pipelines to process and transform structured and semi-structured data using PySpark and SparkSQL
* Work with JSON objects to parse messages and send MDK messages via Kafka to S3
* Create and manage data endpoints with defined schemas, orchestrated through RabbitMQ and Kafka
* Execute data transformations from JSON to RDDs using Spark on AWS EMR/EC2
* Support orchestration through AWS Step Functions with a future transition to Airflow
* Use SQL to extract, transform, and load data into reporting and dashboarding tools
* Collaborate with DevOps and CI/CD teams to maintain GitLab pipelines and automate infrastructure deployment using Terraform
* Maintain version-controlled ETL code in GitLab and participate in testing, deployment, and monitoring workflows
* Participate in cross-functional engineering discussions, aligning with best practices and timelines
Por favor confirme su dirección de correo electrónico: Send Email