At a Glance
- Tasks: Design and build automation frameworks for testing AI systems using Python.
- Company: Join Resillion, a passionate tech company leading in AI innovation.
- Benefits: Hybrid work model, skill development, and a supportive culture.
- Other info: Dynamic environment with opportunities for growth and collaboration.
- Why this job: Be at the forefront of AI technology and make a real impact.
- Qualifications: Strong Python skills and experience in test automation required.
The predicted salary is between 50000 - 65000 € per year.
At Resillion, our culture is built on a mindset of “if you see something, say something” — taking responsibility and speaking up. We expect everyone to adapt and embrace change as the company grows. We recognise the value of every employee and invest in developing their skills so that, as an organisation, we continue to lead our industry. Above all, it’s a culture driven by passion for what we do and commitment to the bigger picture of the company. If you want to be part of our journey, this role may be the right fit for you.
Location: Glasgow (Hybrid preferred but not essential)
Start Date: Immediate
Right to Work: Must have full UK working rights (no sponsorship available due to start dates)
Additional Requirement: Must have 3 years of UK address history due to immediate onboarding
Programme: AI / Agentic AI
Seniority: Senior Automation Engineer
Overview
We are seeking a Senior Automation Engineer / SDET with strong Python skills and hands‑on experience testing AI systems, including LLM‑driven applications and agentic AI workflows. You will design and build a scalable automation framework, implement modern AI testing strategies, and ensure the quality and reliability of next‑generation conversational and agent‑based AI products. This role is ideal for someone who is passionate about automation engineering and has a solid understanding of how AI/LLM systems behave in real‑world scenarios.
Key Responsibilities
- Develop and maintain a Python‑based automation framework for validating AI agents (chatbots, voicebots, agentic workflows).
- Design and execute automated tests for:
- LLM responses
- Conversational flows
- Retrieval‑augmented generation (RAG)
- End‑to‑end agentic scenarios
- Apply modern AI testing strategies and evaluation methods (e.g., DeepEval, RAGAS — beneficial but not essential).
- Integrate automated tests into CI/CD pipelines (Jenkins, GitHub Actions, Azure Repos).
- Build and manage batch and scheduled executions using Docker, Kubernetes, cron jobs, or similar tooling.
- Manage code repositories using Git, including branching strategies, version control, and collaborative workflows.
- Collaborate closely with engineering, AI/ML, and product teams to ensure high‑quality delivery of AI‑powered features.
Required Skills & Experience
- Strong Python development skills, particularly for automation (PyTest preferred).
- Solid background in test automation and framework design.
- Experience testing or validating AI/LLM‑based systems (agentic workflows, conversational AI, RAG pipelines, etc.).
- Understanding of AI evaluation concepts — familiarity with tools like DeepEval or RAGAS is a bonus, not a requirement.
- Experience implementing CI/CD pipelines and automated batch executions.
- Strong understanding of source code management, Git workflows, and collaborative engineering practices.
- Ability to design structured, scalable, and maintainable automation solutions.
Preferred Qualities
- Quick learner with a proactive, ownership‑driven mindset.
- Strong problem‑solving skills and ability to work in fast‑moving AI environments.
- Comfortable collaborating with cross‑functional teams (AI/ML, engineering, product).
- Passion for AI technologies and modern testing methodologies.
Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) in Birmingham employer: Resillion
At Resillion, we pride ourselves on fostering a dynamic and inclusive work culture that encourages innovation and personal growth. Our commitment to employee development is reflected in our investment in training and skill enhancement, ensuring that every team member can thrive in their role while contributing to cutting-edge AI solutions. Located in Glasgow, we offer a hybrid working model that promotes flexibility, allowing you to balance your professional and personal life effectively.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) in Birmingham
✨Tip Number 1
Network like a pro! Reach out to folks in the industry, especially those at Resillion. A friendly chat can sometimes lead to opportunities that aren’t even advertised yet.
✨Tip Number 2
Show off your skills! If you’ve got a portfolio or GitHub with projects related to Python automation or AI testing, make sure to highlight it during interviews. It’s a great way to demonstrate your expertise.
✨Tip Number 3
Prepare for the interview by brushing up on your knowledge of AI systems and testing strategies. Be ready to discuss how you’d tackle real-world scenarios, as this shows you’re not just about theory but practical application too.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive about their job search.
We think you need these skills to ace Senior Automation Tester (Python) – Agentic AI & LLM Systems (Hybrid) in Birmingham
Some tips for your application 🫡
Show Your Passion for AI:When you're writing your application, let your enthusiasm for AI and automation shine through. We love candidates who are genuinely excited about the technology and its potential, so share any relevant projects or experiences that highlight your passion!
Tailor Your CV and Cover Letter:Make sure to customise your CV and cover letter to match the job description. Highlight your Python skills and any experience with AI systems, as this will show us that you’re a great fit for the Senior Automation Tester role.
Be Clear and Concise:Keep your application clear and to the point. Use bullet points where possible to make it easy for us to see your key achievements and skills. Remember, we want to quickly understand why you’d be a fantastic addition to our team!
Apply Through Our Website:Don’t forget to apply through our website! It’s the best way for us to receive your application and ensures you’re considered for the role. Plus, it shows us you’re keen on joining our journey at Resillion!
How to prepare for a job interview at Resillion
✨Know Your Python Inside Out
Make sure you brush up on your Python skills, especially around automation frameworks like PyTest. Be ready to discuss your past projects and how you've used Python to solve real-world problems in AI testing.
✨Familiarise Yourself with AI Testing Strategies
Since the role involves testing AI systems, it’s a good idea to understand modern AI testing strategies. Even if you’re not familiar with tools like DeepEval or RAGAS, showing that you’ve researched them can impress the interviewers.
✨Show Your Collaborative Spirit
Resillion values teamwork, so be prepared to talk about your experiences working with cross-functional teams. Highlight any instances where you’ve collaborated with engineering, AI/ML, or product teams to deliver high-quality features.
✨Embrace Change and Adaptability
With a culture that encourages speaking up and taking responsibility, demonstrate your adaptability. Share examples of how you’ve embraced change in previous roles and how you can contribute to Resillion's growth mindset.
