Accountant for AI Model Evaluation

Evaluate how well AI systems perform real-world accounting tasks by defining professional quality, creating task-specific grading criteria, and scoring work samples. This role does not produce accounting deliverables, it sets the standards and documents rigorous, evidence-based justifications for scores on AI-generated and human work.

• Design precise, task-specific grading criteria for real-world accounting deliverables, including reconciliations, close packages, financial statements and disclosures, workpapers, and technical accounting memos
• Score AI-generated and human work samples against those criteria, providing detailed written justifications for every score
• Apply consistent, evidence-based judgment so that scores are reproducible and defensible
• Incorporate structured feedback from senior reviewers and iterate quickly on grading criteria and scoring approach

• At least 5 years of professional accounting experience
• Experience in audit, assurance, or advisory at leading global accounting firms, or in senior accounting roles at large public companies
• CPA or equivalent professional certification strongly preferred
• Deep practical knowledge of US GAAP application, month-end close, account reconciliations, financial statement preparation and disclosures, and audit-ready workpapers
• Exceptionally strong written communication, with the ability to explain precisely why a piece of work does or does not meet professional standards
• Detail-oriented, consistent, and comfortable having professional judgment reviewed and calibrated against peers
• Prior experience with AI training, evaluation, or human-data projects is a strong plus

• Remote engagement
• Hourly employment type

$130 to $160 per hour

No specific work-authorization or sponsorship details were provided. Applicants should ensure they are legally authorized to work in their country or jurisdiction of residence.

• Submit your resume or a summary of relevant professional background to begin
• Qualified applicants may be asked to complete a brief technical assessment or provide additional information for evaluation