Content Marketer for AI Agent Evaluation

Role Overview

Help create realistic simulated marketing workplaces that evaluate and improve AI agents performing real content marketing work. These environments mirror a marketing organization''s tools, including email, storage, CRM, project management, social media management, web analytics, AEO/SEO, advertising, CMS, product analytics, and support, using realistic documents, dashboards, personas, and intentionally planted issues.

Key Responsibilities

• Design and pressure-test Content Marketing scenarios, including briefs, supporting artifacts, practitioner judgment calls, and mistakes less experienced practitioners might overlook.
• Create task briefs based on content marketing work you have personally performed, identifying the underlying issue being tested and the conclusion a strong practitioner should reach.
• Define the documents, dashboards, personas, and tool states needed to make each scenario realistic.
• Write or review reference answers and scoring criteria that distinguish strong responses from plausible but incorrect ones.
• Evaluate AI agent attempts against the standards you establish.
• Contribute scenarios involving messy or conflicting data diagnosis, prioritization and tradeoffs, planning and execution, QA and reconciliation, triage and escalation, research and evaluation, reporting, and multi-step work across several tools.

Content Marketing Focus

Work may cover research and discovery, editorial prioritization, content production, and content quality assurance and audits.

Qualifications

• Hands-on content marketing experience, including end-to-end ownership of an editorial program rather than solely managing others who perform the work.
• Comfort using a CMS such as Contentful, Google Analytics 4, and a research platform such as Semrush.
• Experience building AI training environments, reinforcement learning environments, or simulated case studies is strongly preferred.
• Rubric Academy or Rubric Bootcamp certification is strongly preferred.
• Strong written reasoning skills, with the ability to clearly explain why a decision is correct.

Work Terms

Hourly engagement.

Compensation

$60 to $100 per hour.

Application Process

Applicants complete a brief multiple-choice knowledge screener for this content marketing specialty before their application is reviewed.