Hindi Audio Evaluator for AI Research

Lead transcription, annotation, and evaluation of Hindi audio and video to help train and benchmark advanced language models. This short term, structured engagement focuses on producing high quality Hindi outputs, defining evaluation standards, and testing model responses, with remote work and occasional in-person collaboration in the San Francisco Bay Area.

• Listen to, analyze, and transcribe Hindi audio and video following detailed instructions and formatting constraints.
• Produce accurate, high quality written Hindi outputs, and provide supporting work in English when required.
• Capture linguistic nuance, including tone, intent, formal versus informal register, regional expressions, dialectal variation, code switching, and contemporary Hindi usage.
• Define evaluation expectations for consumer audio contexts, and create detailed rubrics and grading guidelines in both Hindi and English.
• Document standards to ensure consistent grading across reviewers and model evaluations, and identify edge cases such as grammatical complexities and colloquialisms.
• Run prompts through language models, assess generated outputs against predefined criteria for accuracy, completeness, fluency, and clarity, and supply structured feedback to improve model performance.
• Participate in quality assurance and review cycles to validate task design, rubrics, and outputs before datasets are integrated into benchmarks.
• Collaborate with project leads to resolve ambiguities and refine task instructions and evaluation criteria.

• Strong writing, editing, and critical thinking skills.
• Ability to work independently, manage time effectively, and meet deadlines.
• Native or near native fluency in Hindi, both spoken and written, and professional fluency in English.
• Strong familiarity with spoken Hindi, regional vocabulary, dialects, and contemporary usage.
• Proven ability to accurately transcribe and analyze Hindi audio across general consumer contexts.
• Available to commit 10 to 20 hours per week.

• College students or recent graduates.
• Backgrounds such as linguistics, humanities, social sciences, journalism, translation or localization, or technical disciplines.
• Prior experience with transcription, annotation, localization, evaluation, or research workflows in Hindi.
• Familiarity with Hindi English code switching, including Hinglish, and regional language variation.
• Interest in AI, language models, or applied research environments.

Remote engagement with the expectation of occasional in person collaboration if required in the San Francisco Bay Area. This is a short term, structured project role paid hourly. Commitment is 10 to 20 hours per week.

Hourly rate: $50 per hour.

• Must be based in the San Francisco Bay Area, able to attend occasional in person sessions if requested.
• Fluent Hindi and professional English required.

• Complete a short AI led interview, approximately 15 minutes in length.
• If selected, you will be onboarded and invited to begin project work following the interview.