Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Develop and maintain Python-based automation frameworks for AI systems.
- Company: Join Resillion, a passionate tech company leading in AI innovation.
- Benefits: Flexible hybrid work, competitive salary, and skill development opportunities.
- Other info: Dynamic environment with excellent career growth and collaborative culture.
- Why this job: Be at the forefront of AI technology and make a real impact.
- Qualifications: Strong Python skills and experience in test automation for AI systems.
The predicted salary is between 55000 - 65000 £ per year.
At Resillion, our culture is built on a mindset of "if you see something, say something" — taking responsibility and speaking up. We expect everyone to adapt and embrace change as the company grows. We recognise the value of every employee and invest in developing their skills so that, as an organisation, we continue to lead our industry. Above all, it’s a culture driven by passion for what we do and commitment to the bigger picture of the company. If you want to be part of our journey, this role may be the right fit for you.
Location: Glasgow (Hybrid – 3 days per week on‑site)
Start Date: Immediate
Right to Work: Must have full UK working rights (no sponsorship available due to start dates)
Additional Requirement: Must have 3 years of UK address history due to immediate onboarding
Programme: AI / Agentic AI
Seniority: Senior Automation Engineer
Overview
We are seeking a Senior Automation Engineer / SDET with strong Python skills and hands‑on experience testing AI systems, including LLM‑driven applications and agentic AI workflows. You will design and build a scalable automation framework, implement modern AI testing strategies, and ensure the quality and reliability of next‑generation conversational and agent‑based AI products. This role is ideal for someone who is passionate about automation engineering and has a solid understanding of how AI/LLM systems behave in real‑world scenarios.
Key Responsibilities
- Develop and maintain a Python‑based automation framework for validating AI agents (chatbots, voicebots, agentic workflows).
- Design and execute automated tests for:
- LLM responses
- Conversational flows
- Retrieval‑augmented generation (RAG)
- End‑to‑end agentic scenarios
Required Skills & Experience
- Strong Python development skills, particularly for automation (PyTest preferred).
- Solid background in test automation and framework design.
- Experience testing or validating AI/LLM‑based systems (agentic workflows, conversational AI, RAG pipelines, etc.).
- Understanding of AI evaluation concepts — familiarity with tools like DeepEval or RAGAS is a bonus, not a requirement.
- Experience implementing CI/CD pipelines and automated batch executions.
- Strong understanding of source code management, Git workflows, and collaborative engineering practices.
- Ability to design structured, scalable, and maintainable automation solutions.
Preferred Qualities
- Quick learner with a proactive, ownership‑driven mindset.
- Strong problem‑solving skills and ability to work in fast‑moving AI environments.
- Comfortable collaborating with cross‑functional teams (AI/ML, engineering, product).
- Passion for AI technologies and modern testing methodologies.
Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) in Edinburgh employer: Resillion
Contact Detail:
Resillion Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) in Edinburgh
✨Tip Number 1
Network like a pro! Reach out to folks in the industry on LinkedIn or at meetups. A friendly chat can sometimes lead to job opportunities that aren’t even advertised yet.
✨Tip Number 2
Prepare for those interviews! Research common questions for automation testers and practice your answers. We all know that confidence is key, so get comfortable talking about your Python skills and AI testing experience.
✨Tip Number 3
Show off your work! If you’ve got a GitHub profile or a portfolio of projects, make sure to share it. It’s a great way to demonstrate your skills and passion for automation and AI.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are genuinely interested in joining our journey.
We think you need these skills to ace Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) in Edinburgh
Some tips for your application 🫡
Tailor Your CV: Make sure your CV highlights your Python skills and experience with AI systems. We want to see how your background aligns with the role, so don’t be shy about showcasing relevant projects or achievements!
Craft a Compelling Cover Letter: Your cover letter is your chance to shine! Share your passion for automation testing and AI technologies. Let us know why you’re excited about this role and how you can contribute to our journey at Resillion.
Showcase Your Problem-Solving Skills: In your application, give examples of how you've tackled challenges in previous roles. We love candidates who can think on their feet and adapt to fast-moving environments, especially in AI!
Apply Through Our Website: We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for the role. Plus, it’s super easy!
How to prepare for a job interview at Resillion
✨Know Your Python Inside Out
Make sure you brush up on your Python skills, especially around automation frameworks like PyTest. Be ready to discuss your past projects and how you've used Python to solve real-world problems in AI testing.
✨Familiarise Yourself with AI Testing Strategies
Since the role involves testing AI systems, it’s a good idea to understand modern AI testing strategies. Even if you’re not familiar with tools like DeepEval or RAGAS, showing that you’ve researched them will impress the interviewers.
✨Showcase Your CI/CD Knowledge
Be prepared to talk about your experience with CI/CD pipelines and how you’ve integrated automated tests into them. Mention specific tools like Jenkins or GitHub Actions that you’ve used, as this will demonstrate your hands-on experience.
✨Emphasise Collaboration Skills
This role requires working closely with cross-functional teams, so highlight any experiences where you collaborated with engineering, AI/ML, or product teams. Share examples of how you contributed to team success and adapted to changes in fast-paced environments.