Search by job, company or skills

SMBC Group

AVP/VP, AI/ML Model Validation Engineer, Data Management Office

SMBC Group
Singapore
4-6 Years
Save
  • Posted a month ago
  • Be among the first 10 applicants
Early Applicant

Job Description

Responsibilities

  • Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models.
  • Perform functional, regression and scenario‑based testing of model behaviours and workflows.
  • Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection.
  • Assess end‑to‑end model workflows including data inputs, feature transformations, task completion, tool‑use accuracy and multi‑step reasoning.
  • Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks.
  • Validate API and tool‑integration behaviour in production‑like environments, identifying dependency or orchestration issues.
  • Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly.
  • Collaborate with data scientists across departments to understand modelling intent, feature logic and expected behaviours.
  • Perform data‑management tasks to support AI/ML model testing, including maintaining metadata, documenting key datasets and ensuring clarity of data inputs.
  • Contribute to AI/ML proof‑of‑concept (POC) initiatives to strengthen evaluation methodologies and support innovation.
  • Support data‑management/analytics initiatives such as the Analytics Workbench and contribute to AI/ML/data analytics enablement.

Requirements

  • Minimum 4 years of relevant experience in model testing, QA/QC, AI/ML evaluation, CI/CD, MLOps, data engineering, or related technical roles.
  • Proficiency in Python (especially PySpark, MLlib, pytest), R and SQL; knowledge of Scala, Rust, Java, JS or C++ is a plus.
  • Experience designing and executing test strategies for ML/AI models, including automated pipelines and regression frameworks.
  • Ability to evaluate statistical, ML and LLM models using performance, bias, robustness and drift metrics.
  • Strong ability to assess feature engineering logic, dataset integrity, workflow reliability and tool‑integration behaviours.
  • Experience troubleshooting using logs, traces and debugging tools to identify root‑cause issues.
  • Strong documentation and communication skills to articulate findings, risks and remediation requirements.
  • Ability to collaborate effectively with data science, engineering, IT and governance functions.
  • Understanding of Responsible AI concepts and quality expectations for production‑ready AI/ML systems.

More Info

Job Type:
Permanent Job
Industry:
Other
Function:
Ai Ml Evaluation
Employment Type:
Full time

About Company

SMBC Group

Job ID: 147267415

Jobs by Skill - IT
Jobs by Skill - Non IT
Jobs By Industry
International Jobs
Jobs in Top Cities
Popular Jobs
Last Updated: 24-05-2026 08:35:01 PM
Homejobs in SingaporeAVP/VP, AI/ML Model Validation Engineer, Data Management Office

Similar Jobs

AVP/VP, Quantitative Analyst, AI Alpha Group

Gic
Singapore

Skills:

risk modeling , Python, Generative AI concepts, Statistical data analysis libraries, Portfolio construction

AVP/VP, DevOps Engineer

sumitomo mitsui banking corporation singapore branch
S
Singapore

Skills:

Elk, Prometheus, Bash, Grafana, Docker, Terraform, Splunk, Python, Kubernetes, Azure DevOps, HashiCorp Vault, Go, Azure Key Vault, GitLab CI, GitHub Actions, Bicep

VP, Data Management & Analytics, Data Management Office (DMO)

Sumitomo Mitsui Banking Corporation
Singapore

Skills:

Bi Tools, Data Management, Data Governance, Data Analytics, Dashboard Development, Python/R