About this role
Role Overview
Create challenging codebase exploration tasks that help train AI agents to investigate and reason about unfamiliar C software repositories. You will develop high level engineering questions that require evidence from runtime behavior and investigation across multiple files, similar to the work a senior engineer performs when diagnosing a foundational system.
The work covers production C codebases including PostgreSQL, Redis, git, OpenSSH, OpenSSL, curl, libuv, libgit2, libarchive, Open vSwitch, tcpdump, tmux, and zstd.
Key Responsibilities
- Select assignments from a prevalidated set of C repositories, each fixed to a specific git commit and supplied with an engineering question, positive and negative rubrics, foils, and a reference solution.
- Investigate the relevant subsystem and associated pull request or issue history until you understand its behavior.
- Rewrite questions to challenge advanced coding agents while keeping them precise and fairly answerable.
- Refine rubrics and foils to recognize sound reasoning and reject credible but incorrect answers.
- Complete the required validation workflow, Check, Golden, Run, and Analyze, revising the task until each stage passes.
- Document work in the task README, submit it for expert review, and address feedback on returned tasks.
Qualifications
- 3+ years of professional software engineering experience, including substantial production C experience.
- Ability to navigate large, unfamiliar C codebases and explain runtime behavior, not only source code structure.
- Comfort with preprocessor-heavy builds, platform conditionals, manual memory management, and autotools, CMake, or Meson build graphs.
- Experience with databases, cryptography, networking protocols, or systems programming is highly relevant. The task mix emphasizes architecture and system design, 39%, and code onboarding, 34%, with a meaningful security component.
- Ability to use debuggers, sanitizers, tracing tools, and profilers to establish actual program behavior.
- Precise written English, as the questions and rubrics you produce are central deliverables.
Work Terms
- Remote, asynchronous, per-task contract work with expert team leads across US, India, and Nigeria time zones.
- Work is completed in a designated online workspace; time is tracked with Insightful and access is provisioned through Okta.
- Before receiving task access, candidates must complete identity verification, a background check, signed Terms of Work and CIIAA, a tax form, and a short calibration quiz.
- Each completed task is reviewed by a second engineer and a senior reviewer before acceptance.
Compensation
$130 per approved task, paid after the task is accepted through review.
Application Process
Submit a resume or relevant technical background. Qualified applicants may be asked to complete a brief technical assessment or provide additional information.