MAJOR RESPONSIBILITIES:
Work closely with team of AI engineers to design, build, and serve LLMs to solve complex business challenges using Azure (CPU & GPU environments)
Researching & implementing state of the art LLM techniques including pre-training, fine-tuning, preference alignment, and deployment while also focusing on prompt engineering and generative AI more broadly
Heavily focusing on developing novel data sets that enable LLMs to perform new tasks as well as tooling/platforming to collect these samples at scale. You will need strong python data fundamentals coupled with a software mindset for making data processing and collection pipelines repeatable, scalable, and high quality.
Ensure high quality code that meets business objectives, quality standards and development guidelines.
Building reusable pipelines, processes, and tools to streamline LLM and generative AI workflows
Manage project stakeholder expectations and issue communications on progress.
React to shifting priorities without compromising deadlines and momentum.
QUALIFICATIONS:
Must have:
3 + years’ experience in AI/ML.
Experience in LLM Engineering – pretraining, post-training /alignment
Deep expertise in writing and reviewing production code in Python.
Understanding the development lifecycle for LLMs— developing data sets for pre-training, instruction tuning, and preference alignment alongside the modelling techniques for each stage and LLM deployment is as MAJOR plus.
Multi-disciplinary approach to problem solving, including excellent interpersonal and communication skills (written and verbal). This includes crisply talking about technical solutions while being able to collaborate with business architects effectively.
Strong knowledge of LLM frameworks and libraries (such as transformers, trl, deepspeed, PyTorch), and exposure to various ML techniques and their practical implementation in production at large scale.
Experience on distributed, high throughput and low latency architectures
Strong fundamentals in NLP techniques for text representation, semantic extraction techniques, data structures and modeling.
Experience building software on top of major container technology (Kubernetes, Docker etc.)
Nice to have:
Experience defining system architectures and exploring technical feasibility tradeoffs is a huge plus
Familiarity with end-to-end application development using full stack is a plus.
Experience in P&C insurance is a plus.