Physics Expert for AI Model Evaluation

As a Physics STEM Expert, you will play a pivotal role in designing and solving complex Physics problems to evaluate the limitations of large language models. This position offers the opportunity to develop high-quality, step-by-step solutions with precise reasoning, and to collaborate closely with LLM researchers to align problem design with evaluation objectives, particularly in challenging areas such as abstraction, multi-step reasoning, and symbolic manipulation.

Your contributions will help shape new evaluation benchmarks based on Physics curricula, ranging from early undergraduate concepts to advanced PhD-level topics. You will analyze model outputs, identify reasoning gaps, and provide insights that enhance model accuracy and performance, thereby supporting the advancement of AI research through your domain expertise in Physics.

• Design and solve challenging Physics problems to evaluate and identify the limitations of large language models.
• Develop clear, high-quality, step-by-step solutions with precise and well-articulated reasoning.
• Collaborate closely with LLM researchers to align problem design with evaluation objectives.
• Contribute to the development of new evaluation benchmarks based on Physics curricula.
• Analyze model outputs, identify reasoning gaps, and provide insights to improve model accuracy and performance.
• Support the advancement of AI research by combining domain expertise in Physics with analytical methodologies.

• Analytical Skills: Strong research and analytical abilities, with the capability to evaluate complex information and identify meaningful insights.
• Feedback Skills: Ability to provide constructive feedback, detailed annotations, and clear evaluations.
• Creative Thinking: Strong creative and lateral thinking skills to develop innovative solutions.
• Communication: Excellent written and verbal communication skills for effective collaboration in a remote environment.
• Independence: Self-motivated and capable of managing tasks and deadlines efficiently in a remote setting.
• Technical Setup: Access to a reliable desktop or laptop and a stable high-speed internet connection.

• Candidates with a Master’s, Ph.D., or Postdoctoral degree in Physics, Applied Physics, or a closely related field are strongly encouraged to apply.
• Strong ability to analyze and solve complex Physics problems using a structured approach.
• Ability to explain Physics concepts clearly through simplified explanations and visual representations.

• Commitments Required: at least 4 hours per day and up to 40 hours per week with 4 hours of overlap with PST.
• Engagement type: Contractor.

• Shortlisted candidates will be sent a Job Interest Form.
• After the profile review, an assessment will be shared, which must be completed within 24 hours.
• Based on the assessment outcomes, shortlisted candidates will be contacted to schedule an interview.