About this role
Role Overview
Help advance large language models by creating high-quality software engineering datasets, benchmarks, and evaluations alongside AI researchers. You will work across Python, JavaScript including ReactJS, C/C++, Java, Rust, and Go to develop reference solutions, improve code samples, and assess AI-generated output for real-world engineering quality.
Key Responsibilities
- Curate code examples, build solutions, and correct code across Python, JavaScript including ReactJS, C/C++, Java, Rust, and Go for AI model training initiatives.
- Evaluate and refine AI-generated code for efficiency, scalability, and reliability.
- Partner with cross-functional teams to improve AI-driven coding solutions against industry performance benchmarks.
- Build agents that verify code quality and identify recurring error patterns.
- Develop and assess model capabilities across the software engineering lifecycle, including prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, operations, and maintenance.
- Design automated verification mechanisms for solutions to software engineering tasks.
Qualifications
- Several years of software engineering experience, including at least 2 years of continuous full-time experience at a top-tier product company.
- Strong full-stack application development experience and the ability to deploy scalable, production-grade software with modern languages and tools.
- Deep knowledge of software architecture, design, development, debugging, and code-quality and code-review assessment.
- Excellent written and verbal communication skills, with the ability to provide clear, structured evaluation rationales.
Work Terms
- Remote contract engagement.
- Flexible schedule of at least 10 hours per week, with up to 40 hours per week available.
- Some overlap with Pacific Time is required.
- Independent contractor role, with no medical benefits or paid leave.