Computational Astrophysicist & Cosmologist for AI Benchmark Design

Design graduate level computational science challenges that evaluate whether advanced AI systems can perform research oriented astrophysics and cosmology workflows. This role focuses on creating original problems that require real scientific software, including simulations, result interpretation, experiment design, and extracting hidden information from data.

You will build and refine benchmark tasks that test more than pattern matching. Problems may require an AI system to calculate an exact result from a fully specified setup or strategically plan queries and experiments to identify information that is not directly observable. You will test each task with advanced AI models and adjust its design until it reaches the intended level of difficulty.

• Create original computational problems based on real research workflows in astrophysics and cosmology.
• Develop tasks requiring skilled use of specialized scientific software, including multi step calculations, simulations, data interpretation, experiment design, and information discovery.
• Design problems where careful reasoning, efficient measurement choices, interpretation of partial results, and elimination of plausible alternatives are essential.
• Write Python based problem setups, oracle functions, and solution validators.
• Test tasks against advanced AI models and refine them using feedback and observed performance.

• Graduate level training in a relevant STEM field, such as an MS, PhD, or equivalent research experience.
• Deep hands on experience in astrophysics and cosmology, including astropy and related tools for cosmological calculations, angular power spectra, galaxy survey analysis, and observational data reduction pipelines.
• Demonstrated proficiency with at least one relevant scientific software library through research publications, open source contributions, or professional work.
• Strong Python programming skills and practical experience using scientific libraries to solve research problems.
• Understanding of software limitations, edge cases, and what makes a scientific problem meaningfully difficult.
• Ability to work independently and improve task designs based on feedback.
• Comfort working in a Linux terminal environment with remote compute sandboxes.

• Experience with additional specialized astrophysics or cosmology software.
• Experience across multiple relevant scientific domains or tools.
• Familiarity with benchmark or evaluation design.
• Experience teaching scientific subjects or designing exams and problem sets.
• Experience with computational reproducibility and containerized environments.

• Remote, hourly engagement.
• Availability of at least 15 to 20 hours per week is required.

$70 to $100 per hour.