AI Product Quality Analyst

Key Responsibilities

• Review annotated datasets to ensure accuracy, consistency, and completeness
• Implement and monitor data quality assurance processes for annotation workflows
• Measure and report quality using statistical metrics such as F1 score and inter-annotator agreement
• Work with annotation teams to clarify guidelines and improve labeling consistency
• Use annotation and QA platforms to audit and validate datasets
• Collaborate with ML and data teams to improve dataset readiness for model training and evaluation

Must-Have Qualifications

• 2+ years of experience in data quality management, data operations, or similar roles
• Experience working in AI/ML or data annotation environments
• Hands-on experience with data annotation processes and quality assurance methodologies
• Familiarity with statistical quality metrics such as F1 score or inter-annotator agreement
• Proficiency with annotation or QA tools such as Labelbox, Dataloop, or LabelStudio
• Understanding of AI/ML pipeline fundamentals, including data preparation, model training, and evaluation

Good-to-Have

• Experience working with large-scale datasets used for machine learning
• Exposure to improving annotation guidelines or QA frameworks
• Experience collaborating with ML engineers or data scientists