About this role
Role Overview
Help define and evaluate high-quality real-world data science work for an AI research project. Instead of producing the underlying analyses or models, you will create task-specific standards for excellence and assess completed work samples with rigorous written reasoning.
Key Responsibilities
- Develop precise evaluation criteria for data science deliverables, including analyses, models, dashboards, experiment readouts, and written recommendations.
- Assess AI-generated and human work samples against established criteria, providing detailed written justification for each score.
- Apply consistent, evidence-based judgment so evaluations are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and quickly refine your work.
Qualifications
- 5+ years of professional data science experience in industry.
- Experience in business operations, product, or growth data science at top-tier technology companies.
- Strong expertise in experiment design and A/B testing, metric definition, SQL and Python analysis, and communicating findings to executive stakeholders.
- Exceptional written communication skills.
- A detail-oriented, consistent approach and comfort having your judgment reviewed and calibrated with peers.
- Experience with AI training, evaluation, or human-data projects is a strong plus.
Work Terms
- Remote, hourly engagement.
Compensation
- $120 to $170 per hour.
Application Process
- Submit a resume or relevant technical background.
- Qualified applicants may be asked to complete a brief technical assessment or provide additional information.