Applied Machine Learning Engineer (Model Layer) | Senior LLM & AI Model Routing Engineer

Our client is building a private AI chat platform that gives users simple access to powerful open-source and leading AI models without requiring technical setup. The focus is on delivering fast, high-quality, privacy-conscious AI experiences for real users in production.

This role sits at the core of that product. As the Applied Machine Learning Engineer (Model Layer) / Senior LLM & AI Model Routing Engineer, you will own the systems that train, evaluate, align, serve, and optimize the model layer behind the chat experience. This is a hands-on engineering role for someone who has moved beyond prototypes and has real experience improving LLM behavior in production.

We are looking for someone who can combine strong model engineering skills with practical judgment around alignment, safety, evaluation, and performance.

What You'll Own

You will help design and improve the model layer that powers the product, with a strong focus on:

• Fine-tuning and adapting LLMs for real-world use cases
• Improving model behavior, alignment, and safety without over-restricting helpful responses
• Building high-quality training and evaluation datasets
• Designing and implementing guardrails, safety classifiers, and policy controls
• Evaluating model performance across quality, latency, reliability, safety, and cost
• Supporting model serving, inference optimization, and production reliability
• Contributing to model routing decisions across different models and providers

Key Responsibilities

LLM Fine-Tuning and Optimization

• Fine-tune and adapt large language models using approaches such as SFT, LoRA, QLoRA, and preference optimization
• Improve model quality, instruction-following, and behavioral consistency for production use
• Experiment with tuning strategies that improve helpfulness while staying within defined safety requirements
• Work with open-source and external model providers where needed

AI Alignment and Safety

• Improve model alignment and reduce unnecessary refusals while maintaining clear guardrails
• Design, test, and refine safety mechanisms that balance usability with policy compliance
• Build and implement guardrails, safety classifiers, and policy control systems
• Help define safe model behavior for the product's target use cases

Training and Evaluation Data

• Build, curate, clean, and maintain training and evaluation datasets
• Develop practical methods to measure model behavior, quality, refusal patterns, and safety performance
• Create evaluation workflows that support continuous model improvement
• Use production feedback and testing data to identify failure modes and improvement opportunities

Model Evaluation, Serving, and Performance

• Evaluate models across quality, latency, cost, safety, and reliability
• Support or improve model serving and inference optimization for production environments
• Help improve throughput, response times, and runtime efficiency
• Contribute to monitoring, observability, and performance analysis for deployed models

Red Teaming and Robustness

• Conduct or support red-teaming and adversarial testing
• Identify vulnerabilities, weak spots, and unsafe or low-quality model behavior
• Use adversarial findings to strengthen model robustness and safety systems

Cross-Functional Collaboration

• Work closely with founders and engineering stakeholders to translate product goals into model behavior
• Communicate trade-offs clearly across performance, safety, quality, and cost
• Document systems, evaluation logic, and key model decisions in a clear, practical way

Required Skills and Experience

We are looking for a candidate with strong hands-on experience in the following areas:

• Proven experience working directly with LLMs in production or advanced applied environments
• Strong hands-on experience with fine-tuning, including:
• Supervised Fine-Tuning (SFT)
• LoRA / QLoRA
• Preference optimization
• Experience in AI alignment and safety, especially reducing unnecessary model refusals while preserving defined guardrails
• Experience building and managing training and evaluation datasets
• Strong capability in measuring and analyzing model behavior
• Experience designing and implementing:
• Guardrails
• Safety classifiers
• Policy controls
• Strong programming skills in:
• Python
• PyTorch
• Hugging Face
• Experience with:
• Model evaluation
• Model serving
• Inference optimization
• Strong analytical thinking and practical problem-solving skills
• Ability to work independently and take ownership in a lean, fast-moving environment
• Strong written and verbal English communication skills

Nice to Have

• Experience with red-teaming and adversarial testing
• Experience working with open-weight LLMs in production
• Familiarity with model routing or multi-model systems
• Experience improving model behavior through evaluation-driven iteration
• Experience in startup or early-stage product environments

Ideal Candidate Profile

The ideal candidate is a hands-on builder who understands how modern LLM systems behave in the real world and knows how to improve them in measurable ways. They are comfortable working across tuning, safety, evaluation, and deployment rather than focusing only on research.

They should be:

• Practical and execution-focused
• Strong in experimentation and measurement
• Comfortable balancing safety, quality, speed, and cost
• Proactive in finding issues and improving systems
• Confident working with ambiguity and ownership

Why This Role Matters

This role will directly shape how the product behaves in production. The right person will influence model quality, alignment, safety, cost efficiency, and overall user experience at the system level.

If you want, I can also turn this into a shorter, more candidate-friendly job ad version that's better suited for posting on careers pages or LinkedIn.