Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Lead AI evaluation frameworks and ensure quality measures for GenAI outputs.
- Company: Join the FCA, a key player in UK financial regulation.
- Benefits: Competitive salary, flexible working, and opportunities for professional growth.
- Other info: Be part of a diverse team driving innovation in a digital-first environment.
- Why this job: Make a real impact on the future of financial services and protect millions of consumers.
- Qualifications: Experience in analytics, data science, and leading teams is essential.
The predicted salary is between 72100 - 100000 £ per year.
**Division:** Data, Technology and Innovation
**Department:** AI Product Delivery
**Salary:** National (Edinburgh and Leeds) ranging from £72,100 to £100,000 and London from £79,300 to £110,000 per annum (salary offered will be based on skills and experience)
**This role is graded as:** Technical Specialist – Regulatory
**Your external recruitment contact is** Benjamin via [contact information]
**Your internal recruitment contact** is Lauren via [contact information]
Applications must be submitted through our online portal. Applications sent via social media or email will not be accepted.
**About the FCA and team** We regulate financial services firms in the UK, to keep financial markets fair, thriving and effective. By joining us, you’ll play a key part in protecting consumers, driving economic growth, and shaping the future of UK finance services. The Data, Technology and Innovation (DTI) division enables the FCA to be a digital-first, data-led smart regulator by delivering a secure, agile, and cost-effective technology and data ecosystem that drives better decisions, transparency, and operational efficiency. Working alongside the wider AI Programme (which will continue to oversee/coordinate AI activity across the FCA), the department will partner with business leads to shape and deliver work in priority areas — Authorisations, SPC, EMO and Anti‐Money Laundering.
**Role responsibilities**
- Define and own evaluation frameworks for GenAI outputs covering quality measures such as accuracy, relevance, robustness and hallucination rates
- Design, curate and govern test datasets and benchmarks to ensure consistent model and solution performance assessment over time
- Support development of automated evaluation pipelines and operational reporting to embed assurance into delivery
- Identify, assess and mitigate risks in model behaviour for example bias, errors, safety concerns and edge cases with clear escalation and control recommendations
- Manage delivery through well-defined work packages setting priorities, operating standards and performance objectives including line management of Business Analysts
- Engage senior stakeholders to understand strategic priorities, build a pipeline of scoped and prioritised projects and translate needs into analytics led solutions with clear business value
- Work in the public interest protecting 40 million UK consumers who rely on financial services and supporting long term economic growth from an industry contributing 12% of UK economic output
- Manage digital and data-led change by encouraging innovative experiments and working with senior stakeholders while empowering a diverse team to collaborate openly
**Skills required**
**Minimum:** Experience delivering analytics, data science and AI/ML initiatives, including defining success measures, evaluating model/product performance, and applying innovative problem-solving approaches. Demonstrated experience leading people (line management).
AI Evaluation Lead in Edinburgh employer: PSR Limited
Contact Detail:
PSR Limited Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land AI Evaluation Lead in Edinburgh
✨Tip Number 1
Network like a pro! Reach out to people in your industry on LinkedIn or at events. A friendly chat can lead to opportunities that aren’t even advertised yet.
✨Tip Number 2
Prepare for interviews by researching the company and its culture. Tailor your answers to show how you fit in and can contribute to their goals.
✨Tip Number 3
Practice makes perfect! Do mock interviews with friends or use online platforms. The more comfortable you are, the better you'll perform when it counts.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who take that extra step!
We think you need these skills to ace AI Evaluation Lead in Edinburgh
Some tips for your application 🫡
Tailor Your Application: Make sure to customise your CV and cover letter for the AI Evaluation Lead role. Highlight your experience in analytics, data science, and AI/ML initiatives, as well as any relevant projects that showcase your skills.
Showcase Your Achievements: Don’t just list your responsibilities; focus on what you’ve achieved in previous roles. Use metrics and specific examples to demonstrate how you've defined success measures and evaluated model performance.
Be Clear and Concise: Keep your application straightforward and to the point. Use clear language and avoid jargon where possible. This will help us quickly understand your qualifications and fit for the role.
Apply Through Our Website: Remember, applications must be submitted through our online portal. Don’t send your application via email or social media, as we won’t be able to consider it. We can’t wait to see what you bring to the table!
How to prepare for a job interview at PSR Limited
✨Know Your AI Inside Out
Make sure you’re well-versed in the latest trends and technologies in AI, especially those relevant to evaluation frameworks. Brush up on concepts like accuracy, relevance, and robustness, as these will likely come up during your chat.
✨Prepare Real-World Examples
Think of specific projects where you've successfully implemented AI/ML initiatives. Be ready to discuss how you defined success measures and evaluated model performance. This will show your practical experience and problem-solving skills.
✨Engage with Stakeholders
Since the role involves engaging senior stakeholders, prepare to discuss how you’ve previously translated business needs into analytics-led solutions. Highlight your communication skills and ability to build relationships.
✨Show Your Leadership Skills
As this position includes line management, be prepared to talk about your experience leading teams. Share examples of how you’ve motivated your team and managed delivery through well-defined work packages.