Search by job, company or skills

Quality Engineering (AI & LLM Systems) / Test Automation / Evaluation & Validation Engineering

4-6 Years
  • Posted 18 hours ago
  • Be among the first 10 applicants

Job Description

About the Role - Evaluation & Validation Engineer

As an Evaluation & Validation Engineer, you'll evaluate, validate, and improve enterprise AI solutions to ensure they are accurate, reliable, secure, and ready for production. Working within NNIT's AI Center of Excellence, you'll establish evaluation frameworks, quality standards, and validation processes that build confidence in AI systems deployed across global organizations.

You'll collaborate with AI engineers, solution architects, consultants, product owners, and business stakeholders across Europe, Asia, and other regions to evaluate AI solutions throughout their lifecycle—from solution design and development to deployment, monitoring, and continuous improvement.

Success in this role requires analytical thinking, structured problem solving, and a strong understanding of AI evaluation methodologies, software quality engineering, and enterprise delivery.

Roles and Responsibilities:

1. Design AI Evaluation Frameworks

Develop evaluation methodologies that measure AI quality, reliability, and business readiness.

  • Design evaluation frameworks for enterprise AI applications.
  • Define quality metrics, benchmarks, and acceptance criteria.
  • Evaluate LLMs, AI agents, and Retrieval-Augmented Generation (RAG) solutions.
  • Develop datasets and test cases for automated AI evaluations.
  • Support Responsible AI and AI governance initiatives.

2. Build Validation & Quality Engineering

Build scalable validation processes that improve AI quality and production readiness.

  • Develop automated evaluation and regression testing frameworks.
  • Detect hallucinations, model drift, and quality degradation.
  • Improve AI reliability through continuous benchmarking and testing.
  • Build reusable validation assets, test suites, and engineering standards.
  • Recommend improvements to AI quality and engineering practices.

3. Partner with Stakeholders

Collaborate with technical and business teams to ensure AI solutions meet business, operational, and regulatory expectations.

  • Work closely with AI engineers, architects, consultants, and product owners.
  • Translate business expectations into measurable evaluation criteria.
  • Participate in solution reviews and production readiness assessments.
  • Communicate evaluation results, quality findings, and technical risks.
  • Support continuous improvement across the AI Center of Excellence.

What Success Looks Like

Success in this role means you:

  • Improve the quality, accuracy, and reliability of enterprise AI solutions.
  • Reduce production issues through proactive evaluation and validation.
  • Establish reusable evaluation frameworks and engineering standards.
  • Increase stakeholder confidence in AI solutions.
  • Support the continued growth and technical maturity of NNIT's AI Center of Excellence.

Qualifications:

Technical Skills:

  • At least 4 years of Quality Engineering, Test Automation, Validation, AI Engineering, or Software Engineering experience.
  • Experience designing automated testing or evaluation frameworks.
  • Experience validating enterprise software or AI applications.
  • Experience working within Agile delivery environments.
  • Experience working within consulting, enterprise IT, or global delivery organizations is highly desirable.
  • OpenAI Evals
  • Promptfoo
  • DeepEval
  • LangSmith Evaluation
  • RAGAS
  • TruLens
  • Prompt Evaluation
  • AI Quality Metrics
  • Benchmark Testing
  • Azure OpenAI
  • Azure AI Foundry
  • LangChain
  • LangGraph
  • AI Agents
  • Retrieval-Augmented Generation (RAG)
  • Python
  • Git
  • CI/CD
  • REST APIs
  • Test Automation Frameworks

Industry Experience

Many of our customers operate in highly regulated industries where quality, traceability, compliance, and governance are essential for successful AI adoption.

Experience supporting one or more of the following industries is advantageous:

  • Life Sciences
  • Pharmaceuticals
  • Biotechnology
  • Healthcare
  • Medical Devices
  • Banking & Financial Services
  • Insurance
  • Manufacturing

Exposure to Responsible AI, Computer System Validation (CSV), GxP, FDA, EMA, HIPAA, GDPR, ISO 27001, SOC 2, or similar governance and compliance frameworks is beneficial but not required.

Why Join NNIT's AI Center of Excellence

Joining the AI Center of Excellence gives you the opportunity to build the data foundation that powers enterprise AI and digital transformation for global organizations while expanding your expertise across cloud data platforms, Artificial Intelligence, enterprise consulting, and global delivery.

  • Build Solutions That Matter
  • Work Across Borders
  • Grow Your Expertise
  • Shape the Future

WHO WE ARE

NNIT is a fast-growing IT company with ambitious goals and 1,600 employees. At NNIT, you can make your mark on some of the most challenging projects in the industry, on our journey forward as a company, and on your own career. You'll work in agile teams with other passionate people in a culture with room for independent thinking and individual responsibility.

We supply IT services to large enterprises that demand high levels of quality and security, and we take on the responsibility for our clients IT as if it were our own. We live our values every day, so you should be able to see yourself in them. We are: Conscience driven. Value adding. Open and honest.

Our everyday work is characterized by an open and honest atmosphere, and we put emphasis on creating a challenging yet fun working environment. We pay significant attention to job satisfaction, and personal and professional development.

On top of the statutory benefits stipulated by the local law, NNIT DLSP employees enjoy the following benefits upon joining:

  • Competitive Compensation
  • 13th Month Pay
  • Performance Bonus (discretionary)
  • Rice Subsidy
  • Communication Allowance
  • Clothing and Laundry Allowance
  • Medical Allowance (Employee and Dependent)
  • HMO (Employee and Dependent/s)
  • Life Insurance
  • Vacation Leaves and Sick Leaves
  • Emergency Loan
  • Year-End Gift
  • Wedding Gift
  • First Baby Gift
  • Breakfast once a week
  • Fruits, Snacks / Well-stocked pantry
  • Flexible Working Hours
  • Hybrid set up

Related positions:

  • Evals Engineer
  • AI Validation Engineer
  • Quality Assurance Engineer
  • Test Automation Engineer
  • Validation Engineer
  • Software Quality Engineer
  • CSV Engineer / Computer System Validation Engineer

Notes:

1. By applying to this job ad, you acknowledge and agree for your information to be collected by NNIT. Rest assured, any information you share will only be used for recruitment/hiring purposes.

2. This position is available only to those with current authorization to live and work in the Philippines.

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 151666583