Machine Learning Evaluation Specialist
About this role
About the Role What if your years of hard-won research expertise could directly shape the future of AI? We're looking for machine learning domain experts to design expert-level evaluation challenges that push state-of-the-art AI systems to their limits. Your work won't sit in a drawer — it will directly influence how the next generation of AI models are measured, benchmarked, and improved. - Organization: Alignerr - Type: Hourly Contract - Location: Remote - Commitment: 10–40 hours/week --- What You'll Do - Design original, expert-level ML problems rooted in your specialized domain of expertise - Craft evaluation tasks that require advanced domain knowledge well beyond standard ML pipelines - Draw from your own research experience to create problems that challenge the most capable AI systems available - Define clear problem statements, evaluation criteria, and gold-standard solutions - Assess AI-generated ML solutions for correctness, creativity, and methodological rigor - Document problem difficulty, required domain knowledge, and expected failure modes - Collaborate asynchronously with a global team of researchers and engineers --- Who You Are - Graduate-level expertise (MS or PhD preferred) in a scientific or technical domain that intersects with machine learning - Strong working knowledge of ML methods — model selection, feature engineering, evaluation metrics, and pipeline design - Deep familiarity with active, open research problems in your field - Ability to identify precisely where general ML knowledge falls short and specialized domain insight becomes critical - Experience publishing or conducting original research is highly valued - Excellent written communication — able to articulate complex, nuanced problems clearly and precisely - Self-motivated and comfortable working independently on intellectually demanding tasks --- Example Domains (Not Exhaustive) - Computational biology, genomics, or bioinformatics - Climate science and environmental modeling - Medical imaging and healthcare ML - Materials science and computational chemistry - Astrophysics and signal processing - Natural language processing for low-resource or specialized corpora - Robotics, control theory, or reinforcement learning in complex environments - Financial modeling and quantitative analysis --- Why Join Us - Work at the frontier — contribute directly to AI evaluation and safety research at the cutting edge of the field - Your expertise finally gets its due — the niche, deep knowledge you've spent years building is exactly what this role demands - Collaborate with top minds — work asynchronously alongside researchers and engineers from leading AI labs around the world - Full autonomy and flexibility — set your own hours and work entirely on your own schedule, from anywhere - High-impact, meaningful work — your contributions shape how the world's most advanced AI systems are tested and improved - Ongoing opportunity — strong performers are considered for contract extensions and deeper research involvement
Application listings
Choose a source listing and check its country requirements.