AI Policy Reviewer (Remote)

About the role

The AI Policy Reviewer ensures that AI‑generated and user‑generated content complies with internal governance, regulatory standards and responsible AI principles. Working remotely, you will act as a quality‑assurance gatekeeper, safeguarding model integrity by identifying safety risks, bias, misinformation and policy violations.

Key responsibilities

• Review and score AI‑generated responses against detailed policy rubrics, assessing safety, truthfulness, fairness and alignment with community guidelines.
• Identify over‑sensitive or under‑sensitive outputs, handle ambiguous edge cases, and make nuanced judgement calls on context, satire or emerging risks.
• Analyze data to spot systematic flaws, report patterns of bias, hallucination or policy gaps to Product and Engineering teams.
• Collaborate with Policy teams to test and refine evaluation rubrics and provide feedback on teachability of policies.
• Participate in adversarial testing (red‑team) to uncover vulnerabilities before launch.
• Work with Machine Learning Engineers to explain rating decisions and help adjust model behaviour.
• Create high‑quality prompt‑response examples for “golden sets” used in training.

Required profile

• Minimum 3 years of experience in Trust & Safety Operations, Content Policy, Risk Analysis or Legal/Compliance review.
• Deep knowledge of content moderation policies (hate speech, harassment, misinformation, graphic violence).
• Strong analytical ability to deconstruct AI responses and detect logical flaws, hallucinations or subtle biases.
• Excellent written communication to explain findings to engineers and policy experts.
• Emotional resilience and self‑care strategies for exposure to disturbing content.
• Familiarity with global cultural and political nuances for diverse international audiences.

Required skills

• Experience with large language models (e.g., ChatGPT, Gemini).
• Proficiency using dashboards and spreadsheet software for data analysis.
• Use of specialized content review tools.