AI QA Trainer - LLM Evaluation - Freelance Project

AI QA Trainer – LLM Evaluation

Are you an AI QA expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into enterprise-grade platforms. With rigorous evaluation data, tomorrow’s AI can democratize world-class education, keep pace with cutting-edge research, and streamline workflows for teams everywhere. That quality begins with you—we need your expertise to harden model reasoning and reliability.

Responsibilities

On a typical day you will converse with the model on real-world scenarios and evaluation prompts, verify factual accuracy and logical soundness, design and run test plans and regression suites, build clear rubrics and pass/fail criteria, capture reproducible error traces with root‑cause hypotheses, and suggest improvements to prompt engineering, guardrails, and evaluation metrics (e.g., precision/recall, faithfulness, toxicity, and latency SLOs). You’ll also partner on adversarial red‑teaming, automation (Python/SQL), and dashboarding to track quality deltas over time.

Qualifications

A bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or a related field is ideal; shipped QA for ML/AI systems, safety/red‑team experience, test automation frameworks (e.g., PyTest), and hands‑on work with LLM eval tooling (e.g., OpenAI Evals, RAG evaluators, W&B) signal fit. Skills that stand out include evaluation rubric design, adversarial testing/red‑teaming, regression testing at scale, bias/fairness auditing, grounding verification, prompt and system‑prompt engineering, test automation (Python/SQL), and high‑signal bug reporting. Clear, metacognitive communication—“showing your work”—is essential.

Pay & Benefits

We offer a pay range of $6‑to‑$65 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.

Employment Type

Contract

Workplace

Remote

Seniority Level

Mid‑Senior Level