Senior Applied AI Engineer - Prompting & Evaluation in Lamesley

Role Accountabilities

• Practise the craft at a high level - designing, building and refining prompts, context strategies and agentic workflows for AI features in our products.
• Grow the engineers around you - coaching, pairing, workshops and building shared assets that spread good practice across hundreds of engineers.
• Set standards and keep the organisation honest - defining what good looks like for prompting, evaluation and context engineering, and maintaining those standards as tools change.

Key Activities and Contributions

• Design, build and refine prompts, context strategies and agentic workflows for AI features in Klipboard products, where a confidently wrong price, part match or stock answer is worse than no answer.
• Build evaluation into everything - test prompts against real business cases at scale, measure quality honestly, and treat "it looked fine when I tried it" as the start of the work, not the end.
• Work across models and providers, choosing pragmatically on quality, cost and latency rather than loyalty, and keeping up as models, tools and providers change quickly.
• Get deep into the domain detail with product teams and subject matter experts, because a prompt is only as good as the understanding of the business problem behind it.
• Coach engineers across teams to get dramatically more from AI coding tools and LLMs, through pairing, reviews, workshops and patient one-to-one help.
• Build and maintain shared prompt patterns, reusable templates, evaluation harnesses and internal guides, so good practice spreads as assets rather than folklore.
• Run an internal community of practice where engineers share what is working, what failed and why, and make it somewhere people actually want to show up.
• Help define what good looks like for prompting, context engineering and evaluation across R&D, and keep those standards current as tools and models change.
• Share measurable results with engineering leadership, including where AI tooling is genuinely paying off and where it is not.

Systems, Tools and Technology

• Large language model APIs across multiple providers (GPT, Claude, Llama and equivalents).
• AI coding tools: GitHub Copilot, Cursor or equivalents.
• Prompt engineering, context design and agentic workflow frameworks.
• Evaluation harnesses, evaluation datasets and automated test suites for AI outputs.
• Retrieval-augmented generation (RAG), vector search, embeddings and retrieval pipelines (desirable).
• LLM orchestration frameworks (LangChain, Semantic Kernel or equivalents - desirable).

Deliver AI features in Klipboard products with proper evaluation and measurable quality standards. Embed prompt engineering and evaluation best practice across the R&D organisation. Be the sceptical voice when something demos well but will not hold up in production; be the encouraging one when a genuinely good idea needs backing. Maintain high standards for safety, accuracy and data handling in everything that ships. Build and sustain a living library of prompt patterns, templates and guides. Meet engineers where they are, including those working in long-established codebases.

Customer Experience

• Ensure AI features in Klipboard products give accurate, safe and trustworthy answers - customers run their businesses on our output.
• Work with product teams and subject matter experts to understand the business problem behind each feature properly before building.
• Prevent confidently wrong answers - incorrect prices, part matches or stock levels - from reaching customers.

Key Outcomes and Activities

• Engineers across several teams can point to specific things they now do better because of coaching and knowledge-sharing.
• At least one AI feature or workflow shipped per quarter with proper evaluation behind it.
• A living, used library of prompt patterns, templates and guides in place and growing.
• Community of practice running, well attended, and producing outputs teams actually reuse.
• Engineering leadership has a clearer, evidence-based picture of where AI is paying off.

People, Collaboration & Culture

• Generous with knowledge and genuinely energised by other people getting better.
• Rigorous and honest about results - measures, shows working, and says so when data disagrees with the enthusiasm.
• Patient with engineers at every level of AI maturity, from those who live in these tools to those who are skeptical.
• Curious about the trades Klipboard customers work in, because the best prompts are full of domain understanding.
• Calm about the pace of change - treats model and tooling shifts as the interesting part of the job, not a threat.
• Works closely with engineering leadership, architecture, product and cross-site engineering teams across the UK and South Africa.

Additional Responsibilities

• Some travel between Klipboard's UK and South Africa offices as part of the coaching and community-building remit.
• Keep standards current as models, tools and providers change rapidly.
• Represent Klipboard's engineering practice externally where appropriate - talks, writing or open-source contributions.

Key Relationships (Internal and External)

• Internal: Engineering leadership and architecture teams; R&D engineering teams across Newcastle, Tankersley, Hungerford, Johannesburg and Cape Town / Stellenbosch; Product managers and product owners; Subject matter experts in Klipboard's product verticals (distributive trades, rental, automotive).
• External: AI and LLM platform and tooling providers.

Technical and Professional Expertise

• Strong software engineering background with several years building and shipping production software.
• Substantial hands-on experience with large language models: prompt design, context engineering and structured outputs, in real work rather than personal experiments.
• Experience building evaluations or test harnesses for LLM outputs, and using them to improve quality measurably.
• Clear evidence of growing other engineers - through mentoring, teaching, internal training, communities of practice or similar.
• Daily working fluency with AI coding tools such as GitHub Copilot, Cursor or equivalents, with a practical view of where they help and where they mislead.
• Retrieval-augmented generation, agentic workflows, tool use or fine-tuning in production settings.
• Experience with LLM APIs, orchestration frameworks, vector search, embeddings, retrieval pipelines, evaluation datasets or automated test suites for AI outputs.
• Experience across more than one model provider, with a feel for their different strengths.
• Domain exposure to Klipboard sectors: distributive trades, rental, retail, automotive aftermarket parts or garage management.
• Experience helping teams adopt AI tooling in established or legacy codebases, not just greenfield projects.
• Public sharing of your work: internal talks given externally, writing or open-source contributions.