About this role
As an Analytics & Business Intelligence Expert, you will play a crucial role in advancing frontier agent evaluations in analytics by developing long-horizon analytics tasks. These tasks will be designed to reflect your existing work, each accompanied by a deterministic rubric that assesses agent performance against verifiable ground truth. Your focus will be on creating scenarios that yield checkable answers, avoiding open-ended questions and subjective assessments.
Key Responsibilities- Develop ad-hoc analysis tasks that address business questions with known correct answers, utilizing SQL queries to achieve specified result sets.
- Create experimentation scenarios, including A/B test readouts that provide lift, significance, and guardrails with established ground-truth values.
- Define metrics with precise SQL, and conduct funnel and cohort analyses that yield expected numerical outputs.
- Engage in challenging tasks that require long sessions of focused work.
- Bachelor''s or Master''s degree in a quantitative field.
- Minimum of 3 years of experience as a data/BI analyst in a data-mature company.
- Proficiency in one or more areas: advanced SQL, experimentation and A/B testing, metric design, or BI-tool development (Looker, Tableau, Mode, Hex, Metabase).
- Ability to read and produce analytics artifacts, including SQL queries, dashboards, experiment readouts, and metric specifications.
- Strong written communication skills, with the ability to articulate reasoning clearly and encode it into deterministic rubrics.
- Must be located in the United States or Canada.
This is a remote position with an hourly employment type.
CompensationThe compensation for this role ranges from $70 to $100 per hour, depending on domain expertise and prior experience. High-performing contributors will have opportunities for promotion based on the quality and throughput of their tasks.
EligibilityApplicants must be located in the United States or Canada.