Driving Scene Caption Analyst for AI Evaluation

Role Overview

As a Driving Scene Caption Analyst, you will play a critical role in enhancing the accuracy and clarity of AI-generated content related to autonomous driving. This position combines analytical skills with a deep understanding of U.S. road rules and traffic laws. You will review short sensor feeds, evaluate reasoning generated by language models, and refine outputs to ensure they meet high standards of precision and coherence.

Key Responsibilities

• Review 30-second autonomous vehicle sensor feeds and interpret driving scenes from a first-person perspective.
• Evaluate and edit LLM-generated thinking traces for logical consistency and clarity.
• Ground entities and events to scene coordinates and object references.
• Revise driving plans to reflect correct actions, justifications, maneuvers, and speed.
• Identify and remove hallucinated objects or events from AI-generated text.
• Distinguish between actual sensor data and fabricated model outputs.
• Maintain a zero-error standard in safety-critical judgments under time pressure.
• Validate that outputs align with observed trajectory, maneuver, and speed.
• Apply U.S. traffic laws and driving logic to evaluate scenarios.

Qualifications

• Experience in roles requiring precise writing and structured analytical thinking.
• Documented driving experience in the U.S. with strong knowledge of traffic laws.
• Professional fluency in English with high clarity and precision.
• Ability to evaluate logical coherence in multi-step reasoning.
• Strong attention to detail and ability to maintain quality under time constraints.

Nice to Have Skills

• Familiarity with LLM-generated content and common failure modes.
• Understanding of hallucinations, reasoning inconsistencies, and instruction-following errors in AI systems.

Work Terms

• Commitments Required: at least 4 hours per day and up to 30-40 hours per week with 4 hours of overlap with PST.
• Engagement type: Contractor.
• Engagement Length: 12 weeks.

Evaluation Process

• Shortlisted candidates will be onboarded.