Data Analyst, AI Evaluation

Role Overview

Apply data analysis and review expertise to help train next-generation AI systems through accurate, high-quality evaluations of real-world data and task outputs. No prior AI experience is required.

Key Responsibilities

• Review datasets and detailed task outputs for accuracy, completeness, and adherence to guidelines.
• Identify and flag inconsistencies, errors, and outputs that do not follow instructions.
• Apply structured rubrics and evaluation criteria consistently across large datasets.
• Use sound judgment and document rationale when guidance is unclear or incomplete.
• Escalate recurring data-quality patterns for further analysis rather than handling them only as individual issues.
• Provide clear written and verbal feedback that identifies issues and recommends resolution paths.
• Maintain close attention to detail while completing high-volume, repetitive review assignments.

Qualifications

• 3 to 5+ years of experience in data review, data quality, quality assurance, or an analytical role.
• Experience reviewing, validating, and critiquing data or deliverables against explicit criteria.
• Ability to follow detailed written instructions and apply complex evaluation rubrics precisely.
• Comfort making defensible decisions when guidelines are ambiguous or incomplete.
• Strong error-spotting skills, attention to detail, and written and verbal communication.
• Working knowledge of spreadsheets, including sorting, filtering, and simple formulas.
• SQL or hands-on data-querying experience is a plus.
• Experience with data annotation, labeling, or rubric-based evaluation is advantageous.

Work Terms

• Independent contractor engagement.
• Remote role for candidates based in the United States.

Compensation

• $30 to $60 per hour.

Application Process

• Apply using an email address or a supported Google or LinkedIn sign-in method. Continuing confirms agreement to the candidate terms and privacy policy.