About this role
Role Overview
Validate browser-based evaluation workflows for AI-generated web applications to ensure tests are technically sound, deterministic, and able to produce clear pass or fail outcomes before being included in benchmark datasets.
Key Responsibilities- Review browser-based testing workflows for AI-generated web applications.
- Assess whether browser interactions are technically feasible, reliable, and reproducible.
- Validate test isolation, fixtures, setup, and execution flow to ensure consistent behavior.
- Confirm assertions and validation logic produce deterministic, unambiguous pass or fail results.
- Identify flaky tests, hidden dependencies, ambiguous validation logic, or other reliability risks.
- Provide structured technical feedback in line with project review rubrics.
- Minimum 3 years of experience in Software Engineering, QA Engineering, SDET, or Test Automation.
- Proven experience testing modern web applications.
- Familiarity with browser automation frameworks such as Playwright, Cypress, Selenium, or similar tools.
- Experience writing automated UI and end to end tests.
- Strong understanding of test isolation, fixtures, reproducibility, and reliable browser automation practices.
- Excellent debugging skills and strong attention to detail.
- Contribute to building reliable evaluation workflows used to assess AI generated software quality.
- Work on realistic web application testing scenarios involving automation, test design, and software quality engineering.
- Collaborate with a high caliber team helping establish quality standards for AI generated software evaluation.
- Location: Remote.
- Engagement type: Hourly.
$30 to $60 per hour.