About this role
Role Overview
Evaluate AI-generated responses written in Gujarati, identify factual errors and improvement opportunities, and deliver clear, structured analysis in English that will be used to develop higher-quality AI responses.
Key Responsibilities- Conduct fact-checking using trusted public sources and external tools.
- Produce high-quality human evaluation data by noting response strengths, areas for improvement, and factual inaccuracies.
- Assess reasoning quality, clarity, tone, and completeness of model outputs.
- Ensure model responses align with expected conversational behavior and system guidelines.
- Write evaluation artifacts and feedback in English, with clear, reproducible judgments.
- Bachelor''s degree.
- Native-level fluency in Gujarati.
- Strong proficiency in English, with excellent writing skills to communicate nuanced feedback.
- Significant experience using large language models and familiarity with how and why people use them.
- Strong attention to detail, able to notice subtle issues.
- Experience in domains that require structured analytical thinking, such as research, policy, analytics, linguistics, or engineering.
- Prior experience with RLHF, model evaluation, or data annotation.
- Experience writing or editing high-quality written content.
- Experience comparing multiple outputs and making fine-grained qualitative judgments.
- Consistently identifying factual inaccuracies, reasoning errors, and communication gaps in model responses.
- Producing clear, consistent, and reproducible evaluation artifacts.
- Delivering feedback that leads to measurable improvements in response quality and user experience.
- Location: Remote, global.
- Engagement: Contract work, paid on an hourly basis.
- Hourly rate: 15 - 20 hourly.
- Fluent language requirement: native Gujarati and strong English proficiency.
- To be considered for this role, you must complete a bilingual competency interview in Gujarati. Your competency interview must be taken in Gujarati.
- Bachelor''s degree is required.