Punjabi Language AI Response Evaluator

Evaluate Punjabi AI-generated responses to identify factual errors, reasoning gaps, tone and clarity issues, and areas for improvement. Your written analyses will be in English and will help create ideal AI responses later in the project.

• Fact-check model responses using trusted public sources and external tools.
• Identify factual inaccuracies, reasoning errors, and missing context in Punjabi-generated outputs.
• Describe response strengths, and provide actionable suggestions for improvement in English.
• Assess clarity, tone, completeness, and alignment with expected conversational behavior and system guidelines.
• Produce consistent, reproducible human-evaluation artifacts that can be used to refine model outputs.

• Bachelor''s degree.
• Native-level fluency in Punjabi.
• Strong written English skills, able to articulate nuanced feedback clearly.
• Significant experience using large language models and familiarity with common LLM use cases.
• Strong attention to detail, with the ability to detect subtle issues in responses.
• Background or experience in structured analytical thinking, for example research, policy, analytics, linguistics, or engineering.

• Experience with RLHF, model evaluation, or data annotation.
• Experience writing or editing high-quality written content.
• Experience comparing multiple outputs and making fine-grained qualitative judgments.

• Consistently identify factual inaccuracies, reasoning problems, and communication gaps in model outputs.
• Deliver clear, consistent, and reproducible evaluation artifacts.
• Provide feedback that leads to measurable improvements in response quality and user experience.

• Location: Remote, global.
• Engagement type: Contract work, hourly.

• Pay rate: 15 to 20 per hour.

• Fluent language requirement: native Punjabi, strong English proficiency.
• You must complete the Bilingual Competency interview in Punjabi to be considered for this role. The interview is a required step in the application process.

Complete the required Bilingual Competency interview in Punjabi as part of your application. Only candidates who take this interview will be considered. Your evaluation work will be written in English, so be prepared to demonstrate your English writing skills during the process.