Computer Scientist for AI Benchmarking

Develop rigorous academic assessment content that helps evaluate and advance AI capabilities across specialized computer science domains. This fully remote, asynchronous role involves either creating original expert-level multiple-choice questions or verifying existing questions for accuracy, clarity, and rigor.

You will contribute to benchmark development in areas including Accelerator and GPU Kernel Engineering, Formal Methods and Automated Reasoning, Computer Architecture and Accelerators, Distributed Systems, DevOps and Site Reliability, Data Engineering and Databases, Cloud and Infrastructure, Operating Systems and Kernel Engineering, Machine Learning Engineering, Web and API Development, Embedded Systems Engineering, Computer Graphics and Game Development, and Mobile Engineering.

• Author original questions that assess deep conceptual understanding rather than surface-level recall.
• Ensure every question is unambiguous, self-contained, precisely defined, and includes all information needed to solve it.
• Assign a difficulty rating of Medium, introductory undergraduate level; Hard, advanced undergraduate level; or Expert, postgraduate level and above.
• Provide one correct answer and nine plausible, subtly incorrect alternatives designed to challenge expert solvers.
• Create clear, concise, step-by-step solution rationales in Markdown.
• Include one to five academic references for each question from reputable sources, such as peer-reviewed journals or university repositories.
• For verification assignments, identify issues involving clarity, completeness, precision, or solvability; edit questions as needed and document the reasoning for each change.
• For authoring assignments, create challenging questions within your area of computer science expertise, rate their difficulty, and submit them for review.

• PhD or current doctoral candidacy in Computer Science, Electrical Engineering, or a closely related field.
• A master''s degree may be considered for candidates with exceptional depth in a particular subdomain.
• Strong graduate-level knowledge of computer science theory, algorithms, systems design, and/or machine learning.
• Research publications, experience at leading technology companies, or a competitive programming background are strong advantages.
• Excellent written English and the ability to communicate complex ideas clearly and concisely.

• Fully remote, asynchronous hourly work.
• Expected commitment of at least 10 hours per week.

$66 to $84 per hour.