Psychologist for AI Benchmark Development

Role Overview

Create and evaluate rigorous academic psychology assessments that support AI research benchmarks. This fully remote, asynchronous opportunity focuses on challenging multiple-choice content across specialized psychology domains.

Key Responsibilities

• Work in one of two assignments: author original, challenging multiple-choice questions in your psychology specialty and submit them for review, or verify existing questions for accuracy, clarity, and rigor.
• Develop questions that assess deep conceptual understanding rather than surface-level recall.
• Ensure every question is unambiguous, self-contained, precisely defined, and includes all information needed to solve it.
• Rate question difficulty as Medium, introductory undergraduate; Hard, advanced undergraduate; or Expert, postgraduate and above.
• Provide one correct answer and nine plausible, subtly incorrect alternatives designed to challenge expert-level solvers.
• Write clear, concise, step-by-step reasoning solutions in Markdown.
• Include one to five academic references per question from reputable sources, including peer-reviewed journals and university repositories.
• For verification assignments, identify issues involving clarity, completeness, precision, or solvability; edit questions as needed and document the rationale for changes.

Subject Areas

Assignments may cover Neuromorphic Engineering, Human Factors and Engineering Psychology, Consumer and Market Psychology, Psychometrics, Digital Health, and psychedelic-assisted therapy.

Qualifications

• PhD, PsyD, or current doctoral candidacy in Psychology or a closely related field.
• A master''s degree may be considered for candidates with exceptional depth in a specific subdomain.
• Strong knowledge of graduate-level psychological theory, research methodology, and empirical literature.
• Clinical licensure or psychology research publications are strong advantages.
• Excellent written English, with the ability to communicate complex ideas clearly and concisely.

Work Terms

• Fully remote, asynchronous hourly work.
• Expected commitment of at least 10 hours per week.

Compensation

$50 to $63 per hour.