Telugu Language Specialist for AI Evaluation

Evaluate Telugu AI-generated responses to identify factual errors, reasoning gaps, clarity and tone issues, and opportunities for improvement. Produce clear, reproducible English-language analyses that will be used to define ideal model replies and improve user experience.

• Review AI-generated responses written in Telugu, and assess their factual accuracy, reasoning quality, clarity, tone, and completeness.
• Conduct fact-checking using trusted public sources and external tools to verify claims made in responses.
• Document strengths, specific areas for improvement, and factual inaccuracies in English, producing high-quality human evaluation data.
• Compare outputs against expected conversational behavior and system guidelines, ensuring alignment with desired standards.
• Produce consistent, reproducible evaluation artifacts that can guide model improvements.

• Bachelor''s degree.
• Native fluency in Telugu.
• Strong proficiency in written English, able to articulate nuanced feedback clearly.
• Significant experience using large language models, with an understanding of how and why people use them.
• Strong attention to detail, able to spot subtle issues others may miss.
• Background or experience in structured analytical thinking, for example research, policy, analytics, linguistics, or engineering.

• Experience with RLHF, model evaluation, or data annotation projects.
• Experience writing or editing high-quality written content.
• Experience comparing multiple outputs and making fine-grained qualitative judgments.

• Identification of factual inaccuracies, reasoning errors, and communication gaps in model responses.
• Delivery of clear, consistent, and reproducible evaluation artifacts.
• Feedback that leads to measurable improvements in response quality and user experience.

• Engagement type, contract work with hourly compensation.
• Remote role, open to candidates globally.
• Work is paid hourly, contractor relationship.

• Pay rate: $15 to $20 per hour.

• Must be a native Telugu speaker.
• Must have strong English writing skills.
• Must hold at least a Bachelor''s degree.

• To be considered, you must complete a bilingual competency interview conducted in Telugu. Candidates who do not complete this interview will not be considered for the role.
• The evaluations you produce for this role must be written in English.