About this role
Role Overview
Help build and evaluate the datasets and tooling that advance large language models for real-world software engineering. You will collaborate with researchers to curate code examples, author precise solutions, and correct AI-generated code across multiple languages. Your work will support frontier AI research as well as enterprise deployments that require reliable, production-grade coding intelligence.
Key Responsibilities- Curate code examples, implement solutions, and correct code in Python, JavaScript including ReactJS, C and C++, Java, Rust, and Go.
- Evaluate and refine AI-generated code for efficiency, scalability, reliability, and maintainability.
- Collaborate with cross-functional research and engineering teams to improve AI-driven coding solutions and measure them against industry performance benchmarks.
- Design and build agents or automated systems that verify code quality and detect recurring error patterns.
- Formulate hypotheses about steps in the software engineering lifecycle, including prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, and operations, then evaluate model capabilities on those steps.
- Create verification mechanisms that can automatically validate solutions to software engineering tasks.
- Several years of professional software engineering experience, including at least 2 years of continuous full-time experience at a top-tier product company, for example Google, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, or IBM Research.
- Proven ability to build full-stack applications and to deploy scalable, production-grade software using modern languages and tooling.
- Deep knowledge of software architecture, design patterns, debugging practices, and code quality and review processes.
- Strong oral and written communication skills, with the ability to produce clear, structured evaluation rationales.
- Location: Remote.
- Engagement type: Contractor, independent contractor status, no medical or paid leave provided.
- Commitment: Flexible, minimum 10 hours per week up to 40 hours per week. Partial overlap with Pacific Time is required.
- Duration: 1 month, starting next week, with potential extensions based on performance and fit.