AI Rater Guidelines Writer

Create precise, unambiguous rater guidelines and rubrics that enable consistent human judgments used to train and evaluate advanced large language models. You will work directly with program and research teams to turn ambiguous specifications into rater-ready instructions across domains such as finance, retail, insurance, legal, and sports, with a strong focus on removing contradictions and defining edge-case behavior.

• Close the gap between ambiguous program requirements and operational rater instructions by guiding research and program teams.
• Design clear, non-contradictory rating guidelines and rubrics that raters can apply consistently, including handling edge cases.
• Evaluate draft guidelines for ambiguity, internal contradiction, and coverage gaps, and iterate until raters can apply them without escalation.
• Translate program specifications from domain areas such as finance, retail, insurance, legal, and sports into precise, discipline-specific instructions.
• Collaborate with subject matter experts and program leads to ensure consistency and accuracy across multiple guideline sets.

• 3+ years of professional experience in linguistics, instructional design, technical writing, or a closely related field.
• Direct experience writing or refining guidelines or rubrics for human raters, specifically in a GenAI or RLHF context.
• Proven ability to work across multiple subject-matter domains, and to translate domain-specific nuance into unambiguous instructions.
• Demonstrable track record of resolving ambiguity and contradiction in written specifications, with concrete before and after examples.
• Evidence of career progression in relevant roles.
• Strong written communication skills, able to explain complex or nuanced guidance clearly and precisely.
• Ability to engage reliably for at least 35 hours per week during weekdays.

This role is a W-2 employment position with Cincinnatus LLC, and candidates will be placed as part of an extended workforce at a leading AI laboratory. Employment includes payroll, benefits administration, and compliance provided by Cincinnatus LLC, while you are assigned to work directly within client teams. The role requires reliable weekday availability for a minimum of 35 hours per week.

Hourly pay range: 45 to 65 per hour.

• Location: United States. Candidates must be eligible to work in the United States under W-2 employment through Cincinnatus LLC.
• Cincinnatus LLC is the employer of record for this engagement and will manage employment, payroll, benefits, and compliance.
• Cincinnatus is an Equal Employment Opportunity employer. They do not discriminate based on race, religion, color, national origin, sex including pregnancy, childbirth, reproductive health decisions, or related medical conditions, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or any other legally protected characteristic.