STEM Expert for AI Model Evaluation

Drive the creation and evaluation of challenging STEM problems used to fine-tune and benchmark large language models. You will design multi-step physics and math problems, produce clear step-by-step solutions with rigorous reasoning, and collaborate with researchers to build evaluation benchmarks that probe model limitations. This role is fully remote, contract-based, and ideal for candidates currently engaged in advanced STEM study or research.

Based in San Francisco, the company accelerates frontier AI research and helps enterprises deploy reliable, high-impact AI systems. Work here means contributing high-quality data, advanced training pipelines, and domain expertise to support cutting-edge LLM research and production deployments.

• Design and solve challenging STEM problems to probe the limitations of large language models, with an emphasis on areas where models struggle, such as abstraction, multi-step reasoning, and symbolic manipulation.
• Create clear, high-quality, step-by-step solutions with well-articulated reasoning suitable for evaluation and training use.
• Collaborate with LLM researchers to align problem sets and solutions with evaluation goals and to define success criteria.
• Help develop new evaluation benchmarks based on Physics curricula spanning early undergraduate through PhD-level topics.
• Provide constructive feedback and detailed annotations on model outputs and dataset items.

• Education and experience, preferred: currently pursuing or holding a Master’s, PhD, or Postdoctoral degree in STEM, Applied Physics, or a closely related field.
• Analytical skills: strong research aptitude and the ability to analyze and solve complex physics and STEM problems using a structured, logical approach.
• Communication: excellent structured written communication, ability to explain STEM concepts clearly in simple language, and to use visuals and physics reasoning where appropriate.
• Creative thinking: capacity for creative and lateral thinking when designing problem prompts and solutions.
• Feedback and annotation: experience or aptitude for providing detailed, constructive annotations and review notes.
• Remote work skills: self-motivated, able to work independently, and effective at collaborating in a distributed environment.
• Technical setup: access to a desktop or laptop with a reliable internet connection.

• Location: Remote.
• Engagement: Contract, contractor assignment or freelancer status.
• Benefits: This engagement does not include medical or paid leave.
• Duration and extension: Contracts may be extended based on performance and project needs.

• Work fully remotely on cutting-edge AI projects.
• Opportunity to contribute to leading LLM research and enterprise AI deployments.
• Gain experience leveraging AI tools to strengthen analytical skills and future-proof your career.

Candidates currently pursuing or holding a Master’s, PhD, or Postdoctoral degree in STEM, Applied Physics, or a related field are eligible and encouraged to apply.

If you meet the eligibility above and are interested in contributing to LLM evaluation and benchmark creation, please submit an application. Eligible applicants will be considered based on their qualifications and fit for current project needs.