AI Safety Annotation Reviewer
About this role
About Turing Turing is an AI-powered tech services company with a mission to accelerate AGI advancement and deployment by bridging the gap between global talent from 100+ countries and the world's best foundational LLM companies, by helping them improve performance through model evaluation, fine-tuning, feedback, factuality and handling diverse data. Role Overview A US based AI company that is at the forefront of AI research is looking for a Team Lead to oversee a team of business analysts. This opportunity is tailored for professionals who have experience in leading teams and has interest in supporting the development of solutions at the cutting-edge of technology. Job Responsibilities - Audit trainer-submitted evaluations for policy misapplication, incomplete reasoning, scoring drift, schema errors, and annotation inconsistencies. - Apply a detailed AI safety policy framework consistently across high-stakes review tasks. - Identify individual and systematic errors in human-generated annotations and written assessments. - Validate annotation completeness, scoring accuracy, policy alignment, and adherence to required schemas. - Provide direct, constructive feedback to trainers and recommend corrective actions when quality issues are identified. - Document recurring error patterns and contribute to quality improvement, calibration, and reviewer training initiatives. - Maintain accuracy and consistency while working under time-sensitive conditions and handling sensitive content. Job Requirements - Sharp attention to detail and strong analytical and logical reasoning skills. - Demonstrated experience identifying systematic errors in human-generated annotations, evaluations, or written assessments. - Ability to quickly understand and consistently apply detailed policy guidelines. - Native-level English fluency and strong written communication skills. - Comfort providing clear, direct, and constructive feedback. - Strong understanding of US cultural context and communication norms. - Ability to work independently, manage priorities, and maintain quality under time pressure. - Availability to commit to 40 hours per week; immediate availability is preferred. Preferred Qualifications - Experience in AI annotation review, Trust & Safety quality assurance, test engineering, content moderation, or editorial fact-checking. - Previous experience with schema validation, policy compliance reviews, annotation completeness checks, or quality auditing. - Experience working with sensitive or high-stakes content. Importante Note: This project includes sensitive content involving topics such as self-harm, violence, and abuse. Appropriate waivers, support resources, and wellbeing measures will apply. Benefits - Opportunity to work on cutting-edge AI projects. - Competitive compensation. - Flexible working hours and remote work environment. Application Process - Shortlisted candidates will be reached out.
Application listings
Choose a source listing and check its country requirements.