About this role
Help identify weaknesses in conversational AI by testing model responses, uncovering risks, and creating actionable safety data in both English and Tamil.
Role OverviewThis remote, text-based role focuses on adversarial testing of AI models and agents. You will evaluate outputs for issues involving bias, misinformation, and harmful behavior. Participation in projects involving higher-sensitivity content is optional; topics are communicated in advance, with clear guidelines and wellness resources available.
Key Responsibilities- Test conversational AI models and agents for jailbreaks, prompt injections, misuse cases, bias exploitation, and multi-turn manipulation.
- Review AI responses for accuracy, completeness, appropriateness, and safety concerns.
- Annotate failures, classify vulnerabilities, and flag systemic risks.
- Follow established taxonomies, benchmarks, playbooks, and quality standards to conduct consistent testing.
- Create reproducible reports, datasets, and attack cases that teams can use to strengthen AI systems.
- Native fluency in English and Tamil is required.
- Strong judgment when assessing language and content, with the ability to explain findings clearly to technical and non-technical audiences.
- Careful, structured work habits and the ability to identify subtle errors, inconsistencies, and gaps.
- Adaptability across projects, task types, and customer needs.
- Adversarial machine learning, including jailbreak datasets, prompt injection, RLHF or DPO attacks, or model extraction.
- Cybersecurity, including penetration testing, exploit development, or reverse engineering.
- Socio-technical risk assessment, including harassment or misinformation testing, abuse analysis, or conversational AI testing.
- Creative adversarial thinking informed by psychology, acting, or writing.
- Remote, hourly engagement.
- $16 to $22 per hour.