Driving Scene Caption Analyst for AI Evaluation

Role Overview

As a Driving Scene Caption Analyst, you will play a crucial role in the development of autonomous driving technology by analyzing sensor feeds and ensuring the accuracy and clarity of AI-generated content. This position requires a keen understanding of U.S. road rules and traffic laws, as well as strong analytical skills to interpret driving scenes and evaluate the reasoning behind AI outputs.

Key Responsibilities

• Review 30-second autonomous vehicle sensor feeds and interpret driving scenes from a first-person perspective.
• Evaluate and edit LLM-generated thinking traces for logical consistency and clarity.
• Ground entities and events to scene coordinates and object references.
• Revise driving plans to reflect correct actions, justifications, maneuvers, and speed.
• Identify and remove hallucinated objects or events from AI-generated text.
• Distinguish between actual sensor data and fabricated model outputs.
• Maintain a zero-error standard in safety-critical judgments under time pressure.
• Validate that outputs align with observed trajectory, maneuver, and speed.
• Apply U.S. traffic laws and driving logic to evaluate scenarios.

Qualifications

• Experience in roles requiring precise writing and structured analytical thinking.
• Documented driving experience in the U.S. with strong knowledge of traffic laws.
• Professional fluency in English with high clarity and precision.
• Ability to evaluate logical coherence in multi-step reasoning.
• Strong attention to detail and ability to maintain quality under time constraints.

Nice to Have Skills

• Familiarity with LLM-generated content and common failure modes.
• Understanding of hallucinations, reasoning inconsistencies, and instruction-following errors in AI systems.

Work Terms

• Commitments Required: at least 4 hours per day and up to 30-40 hours per week with 4 hours of overlap with PST.
• Engagement type: Contractor.
• Engagement Length: 12 weeks.

Evaluation Process

• Shortlisted candidates will be onboarded.