About this role
Role Overview
As an IT Support Expert, you will play a crucial role in advancing frontier agent evaluations in IT operations by creating long-horizon IT tasks that reflect real-world scenarios. Your work will involve developing tasks that are paired with a deterministic rubric to evaluate agent performance against verifiable ground truth, ensuring that all tasks have checkable answers.
Key Responsibilities- Design helpdesk scenarios with tickets that have known correct resolutions and required diagnostic steps.
- Create identity and access tasks involving provisioning and deprovisioning with defined role-based access outcomes, as well as SSO/MFA issues with known causes.
- Develop endpoint and SaaS scenarios, including device setups against baseline specifications and license management with specified target states.
- Engage in challenging tasks that require long sessions of focused work.
- Minimum of 2 years of experience in IT support, helpdesk, or system administration.
- Expertise in one or more areas: identity and access (Okta, Active Directory, Entra ID), endpoint management (Jamf, Intune), M365 or Google Workspace administration, or ticketing operations (Jira, ServiceNow).
- Ability to read and produce IT artifacts such as tickets, runbooks, access-change requests, and incident write-ups.
- Strong written communication skills, with the ability to articulate reasoning step by step and encode it into deterministic rubrics.
- CompTIA, ITIL, or equivalent certifications are a plus.
- Must be located in the United States or Canada.
This is a remote position with an hourly employment type.
CompensationCompensation ranges from $50 to $70 per hour, depending on domain depth and prior experience. Strong contributors will have opportunities for promotion based on task quality and throughput.