We are seeking a highly skilled and experienced Senior Spark Developer to join our team. This role demands a deep understanding of Apache Spark and its ecosystem, along with a strong foundation in Java/J2EE application development. You will be responsible for designing, developing, and optimizing Spark-based solutions to process large volumes of data, collaborating with various teams to deliver high-performance and scalable data applications. This is a challenging and rewarding opportunity to contribute to a cutting-edge data engineering team. **Responsibilities:** + Design, develop, and implement complex Spark applications for processing large datasets, leveraging your expertise in distributed computing and data manipulation techniques. + Partner with management teams to integrate Spark-based solutions into existing data infrastructure, ensuring seamless data flow and optimal performance. + Resolve critical data processing challenges by evaluating complex business processes, system processes, and industry standards, applying your deep understanding of Spark's capabilities. + Provide technical leadership and mentorship to other engineers, guiding them on Spark best practices and promoting efficient development strategies. + Develop and maintain coding standards, testing procedures, and debugging techniques specifically for Spark applications. + Contribute to the overall data architecture, ensuring that Spark solutions align with business goals and integrate effectively with other data technologies. + Appropriately assess risk when making technical decisions, considering factors such as data security, performance, and scalability. + Drive compliance with applicable laws, rules, and regulations, adhering to company policies and demonstrating sound ethical judgment. **Qualifications:** + 6+ years of application development experience, with a significant focus on Java/J2EE and Apache Spark. + Strong command of Object-Oriented Programming (OOP) principles and design patterns in Java. + Excellent understanding of data structures and algorithms, and their application in Spark development. + Proven experience developing complex Java/J2EE applications, encompassing all areas of the technology stack. + Thorough knowledge and hands-on experience with Hadoop, MapReduce, YARN, Sqoop, Pig, Hue, Unix, Impala, and Cassandra on Mesos. + Extensive experience designing, developing, and deploying Spark applications in a production environment, handling large volumes of data. + Deep understanding of Spark architecture, including Spark SQL, Spark Streaming, RDDs, and DataFrames. + Experience with ETL and BI tools is a plus. + Experience working in an agile environment using Scrum methodologies. + Expertise in designing and optimizing software solutions for performance and stability, particularly in the context of Spark. + Expertise in troubleshooting and problem-solving complex technical issues. + Experience with test-driven development (TDD). + Cloudera certification (CCDH) is an added advantage. **Preferred Qualifications:** + Experience with cloud-based Spark platforms (e.g., Databricks, AWS EMR, Azure HDInsight). + Experience with other big data technologies (e.g., Kafka, Hive, HBase). + Experience with data visualization tools and techniques. **Education:** + Bachelor's degree in Computer Science, Engineering, or a related field. This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required. ------------------------------------------------------ **Job Family Group:** Technology ------------------------------------------------------ **Job Family:** Applications Development ------------------------------------------------------ **Time Type:** Full time ------------------------------------------------------ **Most Relevant Skills** Please see the requirements listed above. ------------------------------------------------------ **Other Relevant Skills** For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------ _Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._ _If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi (https://www.citigroup.com/citi/accessibility/application-accessibility.htm)_ _._ _View Citi’s_ _EEO Policy Statement (https://www.citigroup.com/global/eeo-aa-policy)_ _and the_ _Know Your Rights (https://www.eeoc.gov/sites/default/files/2023-06/22-088\_EEOC\_KnowYourRights6.12ScreenRdr.pdf)_ _poster._ Citi is an equal opportunity and affirmative action employer. Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.