Machine Learning Evaluation Specialist
About this role
About the Role What if your years of hard-earned research expertise could directly shape the future of AI? We're looking for domain experts with deep machine learning knowledge to design evaluation challenges that push state-of-the-art AI systems to their limits. This isn't reviewing chatbot responses or rating simple outputs. You'll be crafting complex, nuanced problems that only a true domain expert could construct — and that only the most capable AI systems could hope to solve. - Organization: Alignerr - Type: Hourly Contract - Location: Remote - Commitment: 10–40 hours/week --- What You'll Do - Design original, expert-level machine learning problems grounded in your area of specialization - Build evaluation tasks that go far beyond standard ML pipelines — requiring genuine, deep domain knowledge - Draw from your own research experience to craft challenges that rigorously stress-test highly capable LLMs - Define precise problem statements, evaluation criteria, and gold-standard solutions - Assess AI-generated solutions for correctness, creativity, and methodological soundness - Document problem difficulty levels, required knowledge, and expected failure modes - Collaborate asynchronously with a global team of researchers and engineers --- Who You Are - You hold a graduate-level degree (MS or PhD preferred) in a scientific or technical field that intersects with machine learning - You have strong working knowledge of ML methods — model selection, feature engineering, evaluation metrics, and pipeline design - You're deeply familiar with active, open research problems in your field - You can identify precisely where general ML knowledge breaks down and specialized domain insight becomes essential - You have experience conducting or publishing original research (highly valued) - You communicate complex ideas clearly and precisely in writing - You're self-motivated and thrive working independently on intellectually demanding tasks --- Example Domains (Not Exhaustive) - Computational biology, genomics, or bioinformatics - Climate science and environmental modeling - Medical imaging and healthcare ML - Materials science and computational chemistry - Astrophysics and signal processing - Natural language processing for low-resource or specialized corpora - Robotics, control theory, or reinforcement learning in complex environments - Financial modeling and quantitative analysis --- Why Join Us - Top-tier compensation — among the highest rates available for contract research work - Work at the frontier — your contributions directly influence how next-generation AI models are measured and improved - Use your expertise meaningfully — this role is built for specialists, not generalists - Full autonomy and flexibility — set your own schedule and work from anywhere - Global collaboration — connect asynchronously with leading researchers and engineers worldwide - Ongoing opportunity — strong contributors are considered for contract extensions and deeper research involvement - Build your profile — establish yourself as a recognized contributor to cutting-edge AI development and safety research
Application listings
Choose a source listing and check its country requirements.