About this role
Role Overview
You will create evaluation environments and automated checkers that test AI models on complex competitive programming problems. The role focuses on designing original contest-style problems not present on existing online judges, producing reference solutions, and building robust test suites and checkers in C++ that validate multiple valid outputs and edge cases. Your work will directly shape how models learn, reason, and perform.
Key Responsibilities- Design and implement checkers for competitive programming problems that allow multiple valid outputs, ensuring correctness and robustness.
- Create unique custom problems that are not available on existing online judges, including full problem statements and constraints.
- Produce reference solutions and develop comprehensive test suites to exercise edge cases and varied solution approaches.
- Evaluate and validate participant submissions against complex problem statements sourced from platforms such as Codeforces.
- Collaborate with the customer team to refine problem requirements and clarify ambiguous scenarios.
- Write clear, maintainable C++ code to assess diverse outputs and edge cases efficiently.
- Document checker logic, explain edge cases and implementation details in writing and verbally.
- Provide feedback to problem setters and reviewers on improving test cases and checkers.
- Keep up to date with competitive programming standards and incorporate best practices into checker development.
- Expertise in competitive programming and coding problem analysis.
- Advanced proficiency in C++, demonstrated by the ability to implement robust checker solutions.
- Strong analytical skills to interpret complex problem constraints and multiple solution paths.
- Exceptional written and verbal communication skills, able to articulate technical details clearly.
- High attention to detail for code validation and output consistency.
- Experience working independently in a remote, collaborative environment.
- Ability to produce high-quality, well-documented code under tight deadlines.
- Preferred but not required, prior experience as a problem setter, tester, or checker on major competitive programming platforms.
- Engagement type, independent contractor.
- Minimum commitment, 15 hours per week or more.
- Location, remote.
- Listed rate, $45 to $65 per hour.
- Compensation structure, output-based. Experts are paid per task that meets the project specifications.
- Time to complete tasks may vary depending on the expert''s experience and workflow.
- Minimum submission requirements apply. Experts must submit a minimum of tasks per week.
- No prior experience in AI is required, domain expertise in competitive programming and C++ is the primary qualification.
- Roles are typically filled within 48 hours, and selected experts are expected to start their first tasks within 24 to 48 hours of completing onboarding.