About this role
Role Overview
Help improve advanced AI models by creating rigorous, real-world evaluations grounded in your domain expertise. This remote contract opportunity offers flexible, pay-per-task work for specialists in history, law, finance, journalism, and technology who have prior AI training experience.
Key Responsibilities
- Design nuanced, domain-specific tasks that test AI model limits and reveal reasoning flaws.
- Create questions and scenarios that require deep subject-matter expertise beyond general knowledge.
- Evaluate AI-generated responses and verify accuracy against authoritative primary sources in your specialty.
- Document clear rationales for model shortcomings, including missed reasoning, source errors, and overlooked nuances.
- Apply task guidelines and evaluation rubrics consistently.
- Independently submit completed deliverables on a rolling basis while meeting requirements for written and verbal clarity.
Qualifications
- Hands-on experience with AI model training, annotation, RLHF, or evaluation.
- Deep expertise in history, archival or library science, law, including paralegal, legal academic, or compliance work, finance, including equity research, private equity, venture capital, or forensic accounting, journalism, or technology and industry history.
- Advanced research skills and a record of verifying complex claims using authoritative sources.
- Exceptional written and verbal communication skills for articulating findings and feedback.
- Ability to follow detailed instructions and evaluation frameworks precisely.
- Experience designing adversarial tasks intended to challenge model performance is advantageous.
- Graduate-level research experience, experience at a Fortune 100 company, or professional fluency in a non-English language is a plus.
Work Terms
- Remote contract engagement.
- Approximately 15 hours per week, with flexible scheduling.
- Work is submitted on a pay-per-task basis, and completion time varies by expertise and workflow.
- Minimum weekly submission requirements apply.
- Selected candidates should be ready to begin and complete their first tasks within 24 to 48 hours after onboarding.
Compensation
- $30 to $60 per hour.
- Payment is output-based for tasks that meet project specifications.
Application Process
Apply for consideration. Selected candidates complete onboarding before beginning their first assigned tasks.