Psychologist for AI Model Evaluation

Psychology experts apply clinical and research knowledge to design and evaluate psychology-focused prompts and to assess large language model outputs for accuracy, reasoning, and domain-appropriate communication. This role involves contributing subject-matter expertise to AI evaluation projects that probe models'' understanding of psychological concepts, methods, and explanations.

• Develop domain-specific prompts and tasks that target psychology concepts, assessments, interventions, and research methods.
• Evaluate and annotate LLM responses for correctness, nuance, and suitability for psychological audiences.
• Conduct literature review and targeted research to inform prompt design and evaluation criteria.
• Collaborate asynchronously with AI research teams and peers to refine tasks and share domain insights.
• Learn and apply new skills related to AI evaluation while contributing to project deliverables.

• Open to doctoral students and graduates in Psychology; applicants must hold at minimum a master’s degree from a US institution.
• Demonstrated subject-matter expertise sufficient to outperform current AI systems in explaining and applying key psychology concepts.
• Experience with research methods, scientific writing, or clinical reasoning in psychology preferred.
• Ability to work primarily asynchronously and communicate results clearly to researchers and engineers.
• Selection for each project is limited to a small cohort chosen for disciplinary and research expertise; applicants should be prepared to compete on the basis of domain knowledge and research experience.

• Part-time, remote, asynchronous engagement.
• Flexible hours with no minimum weekly commitment required.
• Placement into projects depends on project availability; the program runs year-round but openings vary by domain and time.
• Work is performed independently from any location that permits remote participation.

• Up to $85/hr (depending on the project).

• F-1 students who are eligible for CPT or OPT may be eligible to participate. Consult your Designated School Official to confirm eligibility before accepting work.
• If your school requires enrollment in a CPT course for authorization, this program may not meet that requirement.
• STEM OPT is not supported for this program.
• Additional documentation or identity verification may be required during onboarding; check the program help documentation for full details.

• Create an account on the program platform and complete the requested profile information.
• Complete identity verification as instructed.
• Apply to or join relevant projects and complete the project-specific onboarding steps.
• Begin assigned project work and receive payment according to the stated compensation terms.