Senior QA Engineer — AI/LLM Systems & Quality Leadership

About us
We’re Dayshape—an award-winning software scale-up with big ambitions and the momentum to match. Trusted by Big Four and many other top professional services firms globally, our AI-powered resource management platform is helping organisations to achieve extraordinary results. Our platform stands apart as the only solution that combines advanced AI, real-time project financials, and firm-wide insights to elevate resource management to a strategic function. By driving profitable growth, powering confident decisions, and ensuring satisfied clients and teams—we're helping our customers build strong organisations and careers for the long term.

Why our customers love Dayshape:
We help professional firms optimise margins and increase revenue, unlocking access to more profitable work. We provide complete operational visibility today and the tools to confidently predict tomorrow. We empower firms to become the places where top talent wants to work - and the best clients want to work with.

Recognised as Scotland’s fastest-growing tech company in the Deloitte Technology Fast 50 for three consecutive years and more recently as the Enterprise Digital Tech Business of the Year at the 2024 ScotlandIS awards, we’ve consistently proven our ability to innovate and deliver real impact—and we’re always looking for like-minded people to join us. At Dayshape, our purpose is to improve people's working lives, and our culture is an important driving force in helping us to do just that. We're a friendly, inclusive, and ambitious team—driven by our values and a shared commitment to success. If you’re ready to join a fast-growing, high-impact company that’s reimagining resource management, then let’s talk.

About the Role
We’re seeking a Senior Quality Engineer to join our Data Insights LLM team and lead quality assurance for large language model‑powered systems. This role is ideal for an experienced Senior Quality Engineer who has demonstrated the ability to build, extend, or contribute to test frameworks from scratch, and who is comfortable working close to the codebase in complex, data‑driven environments. You’ll collaborate closely with Software Engineers and Product to embed quality throughout the LLM feature lifecycle and help shift testing earlier in the development process.

As a Senior Quality Engineer, you’ll own and shape your team’s quality strategy — from test approach and automation to risk‑based decision‑making — while staying hands‑on and adapting testing methods for probabilistic and evolving systems such as LLMs. Our focus is on component‑ and integration‑level testing, using automation to provide fast, reliable feedback, rather than end‑to‑end UI testing. Beyond functional testing, you’ll help define and validate aspects such as accuracy, reliability, and user trust, supporting confident, risk-aware decisions. For the right candidate, there’s room to grow into wider technical and strategic ownership, influencing quality approaches across the business. This is a hands-on role where you’ll contribute directly to test design and automation, shaping quality practices as our use of LLMs evolves.

What You'll Do:

• Improve and maintain regression testing (manual and automated) for reliability and clarity, coaching others as needed.
• Shape and own quality approaches from discovery to release, championing risk-based practices.
• Design and update test automation and tooling, collaborating with engineers on architecture and testability.
• Manage test infrastructure and data, ensuring suitability for data-driven or probabilistic systems.
• Diagnose and triage test failures, reduce flakiness, and improve diagnostics.
• Support testing for customer hotfixes, leading reviews and embedding preventative actions.
• Plan and execute test strategies, including nondeterministic scenarios (e.g. LLM features).
• Maintain meaningful test artefacts and evidence, adapting for fast-moving or nondeterministic contexts.
• Contribute to clear and actionable root cause analysis for bugs and defects.
• Participate in planning and refinement to clarify intent, identify risks, and define success criteria.
• Support release activities from a quality perspective, focusing on risk and readiness.
• Advise engineers on effective, risk-based testing approaches.
• Build strong domain knowledge to inform quality decisions and proactively address gaps.

About You
Experienced Senior Quality Engineer with expertise in building and enhancing test frameworks, working closely with the codebase for early, reliable feedback. Collaborates with Software Engineers on testability, architecture, and design, with working knowledge of object‑oriented programming languages (e.g. C# or Java) to support component‑ and integration‑level testing. Owns team quality and embeds it throughout the process, influencing by collaboration and evidence. Proven at shaping team-level quality approaches in changing or uncertain environments. Applies a risk-based, pragmatic approach to testing and communicates coverage decisions clearly. Strong in exploratory testing, informing automation and design in complex systems. Quickly develops deep product understanding to inform quality decisions. Enjoys coaching and sharing knowledge to improve team quality mindset. Shows initiative and helps resolve obstacles for themselves and others to maintain progress.

Bonus Points For:

• ISTQB® Certified Tester – AI Testing (CT‑AI), or equivalent experience/training in testing AI or ML‑based systems.
• Hands-on experience testing AI‑powered or non‑deterministic systems in production.
• Experience acting as the sole or primary Quality Engineer on a team, owning quality outcomes end‑to‑end.
• Strong experience influencing engineers and product decisions without formal people‑management authority.
• Experience testing distributed or cloud‑based systems, with an emphasis on diagnostics, reliability, and observability.

What you’ll get:
Starting salary circa £55,000, depending on experience. At least £1,000 per year to spend on professional and personal development. 33 days' holiday per year (including bank holidays), increasing by 1 day each year to a maximum of 40 days. Paid four-week sabbatical in your fifth anniversary year on top of your holiday entitlement. Enhanced family leave policies. Private healthcare and rewards through AXA. Income protection and death in service cover. Matched 5% auto-enrolment workplace pension scheme. Access to wellbeing offerings, such as our Employee Assistance Programme and a dedicated counselling service. Volunteering time – up to 20 hours a year to participate in volunteer work. Regular All Hands meeting for inspiration and over-communication. Time out of the working week for team socials each month, with a mix of in-person and virtual options: past events include hiking, family BBQs, board games and at-home cocktail classes! Genuinely nice, smart people to work with, who are excited about growing our company.

Working Details
This is a full-time role (37.5 hours per week). We typically work from 09:00 – 17:30 from Monday to Friday, though we can be a little flexible around this, just let us know. We are ideally looking for someone based in or around Edinburgh who can work regularly from our office. We support flexible hybrid working and do not expect daily office attendance, however this role involves close, high‑touch collaboration during a period of change, so regular in‑person time with the team is important, and candidates must be able to travel to our Edinburgh office as part of the role.

Join the team!
Equality of opportunity is more than just a responsibility: we believe it is a huge advantage to welcome a variety of experiences and perspectives into the team. Diversity is a great asset and, as such, we strongly encourage applications from any background. This is your opportunity to really influence how we get things done, and how we measure and improve our capability to deliver software with high quality in a scalable way. We are doing well, but there are lots more to do to maintain the high bar and pace that we have set. Everyone here is growing personally as the company grows, so if that sounds like something you would like to be part of, we would love to see your application.

The deadline for applications is Monday 27th April at 12pm. Initial screens and interviews will be conducted around these dates. Please note the successful candidate for this role will be subject to background checks and will have an opportunity to declare anything to us beforehand.