Senior Python AI Trainer & Evaluator (Remote) in Bristol

Prolific is not just another player in the AI space – we are building the biggest pool of quality human data in the world. Over 35,000 AI developers, researchers, and organizations use Prolific to gather data from paid study participants with a wide variety of experiences, knowledge, and skills.

We're looking for Senior Python Developers to join our Expert Network to help train and evaluate cutting-edge AI models. If you have a background in Software Engineering, we'll send you a quick 10- to 15-minute test to assess your skills. If successful, you'll be invited to join Prolific as a participant, where you'll get paid to help AI understand and summarize complex scientific data. Researchers looking for your skills tend to pay up to $50/hr, depending on skills and experience level. You must be prepared to complete paid tasks that require one hour of uninterrupted work, though many are shorter.

What you'll bring

• Educational Background: a BSc or higher in Computer Science, Software Engineering, or closely related technical field
• Professional Experience: real world experience as a Senior Software Engineer/ Developer specializing in Python
• Coding Proficiency: ability to solve LeetCode Medium to Hard-level problems in Python independently.
• Deep Domain Knowledge: expert understanding of the GIL (Global Interpreter Lock), decorators, generators, memory management, and asynchronous patterns (asyncio, async/await).
• Code Quality: a high standard for clean code, including modularity, readability, and adherence to modern Pythonic standards (PEP 8, PEP 20).
• Attention to Detail: ability to spot subtle logical flaws, memory leaks, or security vulnerabilities in model-generated code.

What you'll be doing in the role

• Evaluate Code Accuracy: review AI-generated Python code for functional correctness and adherence to best practices.
• Validate Logic & Reasoning: audit the step-by-step explanations provided by AI for complex algorithmic solutions to ensure they are logically sound.
• Conduct Execution Testing: execute model-generated scripts in appropriate environments to verify performance and output.
• Annotate Model Performance: identify areas where a model provides inefficient solutions, deprecated syntax, or hallucinated library methods.
• Refine Technical Logic: provide structured feedback on how models reason through backend architecture, data pipelines, or API design discussions.

Key Technologies

• Core Language: expert mastery of Python 3.x, including type hints and modern language features.
• Frameworks: extensive experience with Django, FastAPI, or Flask.
• Testing & Tools: proficiency with pytest, unittest, coverage.py, and tox.
• Backend & API: strong understanding of RESTful APIs, GraphQL, and server-side logic in a Python environment (e.g., with FastAPI or Django REST Framework).

Why Prolific is a great platform to join as a Participant

Joining our platform as a Prolific participant will give you the chance to influence the AI models of the future using professional legal expertise. Once you pass our assessment, you can join Prolific in just 15 minutes, and start enjoying competitive pay rates, flexible hours, and the ability to work from home.

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioural data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviours into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breadth and the best of humanity.