About this role
This position offers a unique opportunity to collaborate with a leading AI research organization, focusing on the evaluation of AI systems in real-world design contexts. You will not be creating deliverables directly; instead, your role will involve defining excellence in design by developing specific grading criteria and rigorously assessing completed work samples with well-reasoned justifications.
Key Responsibilities- Design precise, task-specific grading criteria for various design deliverables, including brand identity systems, marketing visuals, presentation layouts, packaging concepts, and digital assets.
- Score both AI-generated and human-created work samples against established criteria, providing detailed written justifications for each score.
- Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate quickly on your evaluations.
- 5+ years of professional experience in graphic, brand, or visual design.
- Experience at top design studios, creative agencies, or in senior design roles at leading consumer or technology brands.
- Deep understanding of design fundamentals, including typography, layout, visual hierarchy, color theory, composition, and brand systems.
- Exceptional written communication skills, with the ability to articulate why a design meets or does not meet established standards.
- Detail-oriented and consistent, with a willingness to have your judgments reviewed and calibrated against peers.
- Prior experience with AI training, evaluation, or human-data projects is a strong plus.
This is a remote, hourly position.
CompensationHourly rate ranges from $80 to $150.
EligibilityQualified applicants may be asked to complete a brief assessment or provide additional information during the application process.