Bilingual Portuguese Language Model Evaluator

This role offers a unique opportunity for native Portuguese speakers from Portugal or Brazil to engage in a high-impact AI research project. As a contributor, you will create Portuguese/English prompt-golden answer pairs that are essential for training and evaluating advanced language models. This position is ideal for individuals who possess exceptional writing skills and have a deep understanding of local language usage, tone, and cultural context.

• Multilingual Prompt Design & Optimization: Create detailed prompts in Portuguese and/or English that are naturally phrased and relevant to Portuguese-speaking users in Portugal and Brazil.
• Define and Document Evaluation Standards: Establish expectations for correct responses in consumer contexts specific to Portugal and Brazil, and develop comprehensive rubrics that reflect linguistic nuance and cultural conventions.
• Model Testing and Grading (Bilingual): Assess model outputs for accuracy and cultural fit in Portuguese, comparing results with English as necessary.
• Benchmarking & Quality Assurance: Collaborate in QA processes to ensure consistency and reliability across Portuguese-language benchmarks before official evaluations.

• Native-level fluency in Portuguese (written), specific to Portugal or Brazil, with strong reading and writing ability in English.
• Must be native to Portugal or Brazil and have lived in or spent significant time in the country, demonstrating deep cultural and linguistic familiarity.
• BS or BA from a reputable institution (completed or in progress).
• Strong writing and critical thinking skills.
• Ability to work independently and meet deadlines.
• Familiarity with ChatGPT or similar tools for personal decision-making, hobbies, or general interests.
• Based in Portugal or Brazil, or able to produce culturally accurate Portuguese specific to these regions.

• Experience in teaching, research, editing, or academic writing.
• Experience creating evaluation criteria, rubrics, or grading guidelines.
• Familiarity with LLMs, prompting, or model evaluation is helpful but not required.

• Commit to at least 20 hours per week.
• Engagement expected for approximately 2–4 months.
• Work in a structured project environment with clear goals and tools.

• Hourly rate ranging from $12 to $30.

• Strictly limited to candidates who are native to Portugal or Brazil.