About this role
Role Overview
This position involves assessing AI-generated responses in Tamil, identifying strengths and areas for improvement. Your insights will contribute to developing optimal AI-generated responses, with your analysis documented in English.
Key Responsibilities- Conduct fact-checking using trusted public sources and external tools.
- Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
- Assess reasoning quality, clarity, tone, and completeness of responses.
- Ensure model responses align with expected conversational behavior and system guidelines.
- Bachelor''s degree.
- Native speaker of Tamil.
- Significant experience using large language models (LLMs) and understanding their applications.
- Excellent writing skills in English, capable of articulating nuanced feedback.
- Strong attention to detail, with the ability to notice subtle issues.
- Background or experience in fields requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering).
- Prior experience with RLHF, model evaluation, or data annotation work.
- Experience writing or editing high-quality written content.
- Experience comparing multiple outputs and making fine-grained qualitative judgments.
- Identifying factual inaccuracies, reasoning errors, and communication gaps in model responses.
- Producing clear, consistent, and reproducible evaluation artifacts.
- Your feedback leads to measurable improvements in response quality and user experience.