Lead Software Engineer - LLM Ops Platform Reliability in Glasgow

Lead Software Engineer - LLM Ops Platform Reliability in Glasgow

Glasgow Full-Time 60000 - 80000 £ / year (est.) On-site
JPMorganChase

At a Glance

  • Tasks: Build and operate AI infrastructure for large language models, ensuring reliability and performance.
  • Company: Join JPMorgan Chase's innovative AI and Machine Learning Platform team.
  • Benefits: Competitive salary, health benefits, and opportunities for professional growth.
  • Other info: Be part of a diverse team that values inclusion and drives technological advancements.
  • Why this job: Make a real impact on cutting-edge AI systems and solve challenging production problems.
  • Qualifications: Experience in software engineering, Python, AWS, and site reliability practices.

The predicted salary is between 60000 - 80000 £ per year.

Description

Help shape how AI systems run reliably in production at scale.

In this role, you’ll build and operate large language model serving infrastructure, bringing strong engineering fundamentals and site reliability practices to cutting-edge AI platforms.

You’ll work hands-on with cloud and Kubernetes-based deployments, deep observability, and cost-aware performance tuning.

If you enjoy solving hard production problems and making platforms measurably better, you’ll find meaningful impact and growth here.

As a Lead Software Engineer at JPMorgan Chase in the AI and Machine Learning Platform team, you will build and scale AI infrastructure that modernizes traditional infrastructure management and site reliability engineering through applied AI.

You will own the reliability, performance, and cost-efficiency of the LLM inference platform end to end.

You will operate large language model serving stacks (such as v LLM and llm-d) in production at scale, with deep instrumentation and strong operational rigor.

You will partner across engineering to deliver secure software, improve stability, and lead incident response and continuous improvement.

  • Job Responsibilities
  • Design, develop, troubleshoot, and deliver secure, high-quality production software and services for AI infrastructure
  • Build backend services and APIs that enable reliable operation of AI infrastructure in production
  • Operate and scale LLM serving infrastructure (such as v LLM and llm-d), including model hosting, request routing, continuous batching, and KV-cache optimization
  • Deploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon EKS and Amazon Sage Maker, as well as on-prem and local GPU clusters, using reproducible infrastructure as code and continuous delivery pipelines
  • Implement observability (logs, metrics, traces) with dashboards and actionable alerting, including Prometheus metrics and Grafana/Alertmanager integration for LLM and GPU workloads
  • Tune GPU and accelerator capacity, autoscaling, and cost efficiency for LLM inference workloads using performance and optimization techniques (e. g., quantization, parallelism, speculative decoding)
  • Lead reliability engineering for LLM endpoints through capacity planning, load/soak testing, safe rollouts (blue/green, canary), failover, and incident response for outages and model-quality regressions
  • Participate in an on-call rotation, lead incident triage and mitigation, and produce clear post-incident root-cause analyses and follow-ups
  • Identify recurring operational issues and automate remediation to improve platform stability and developer experience
  • Build and maintain multi-agent systems with strong orchestration (planning, coordination, tool-calling, state/memory, and workflow control) where appropriate
  • Contribute to an inclusive team culture grounded in diversity, opportunity, inclusion, and respect, and help drive adoption of leading-edge technologies through communities of practice
  • Required Qualifications, Capabilities, and Skills
  • Formal training, certification, or equivalent practical experience in software engineering concepts
  • Hands-on experience with system design, application development, testing, and operational stability in production environments
  • Advanced proficiency in Python for building production-grade services and tooling
  • Proficiency with automation and continuous delivery methods
  • Hands-on experience with AWS and Terraform for infrastructure delivery and lifecycle management
  • Strong understanding of site reliability engineering practices, including incident management, root-cause analysis, runbooks, and reliability patterns
  • Practical knowledge of observability and instrumentation across metrics, logs, and traces
  • Comfort with on-call operations and production troubleshooting
  • Hands-on production experience operating LLM inference servers such as v LLM and llm-d (or directly equivalent serving stacks)
  • Hands-on experience hosting and serving LLMs on Amazon EKS and/or Amazon Sage Maker, and on local GPU infrastructure
  • Knowledge of LLM reliability and risk considerations, including latency/throughput trade-offs, model and weight versioning, prompt/response logging, and safe rollout patterns
  • Preferred Qualifications, Capabilities, and Skills
  • Experience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patterns
  • Experience building AI agents using frameworks such as Lang Chain, Crew AI, Lang Graph, or similar orchestration platforms
  • Experience operating or integrating serving platforms such as KServe, Ray Serve, NVIDIA Triton Inference Server, Text Generation Inference (TGI), alongside v LLM/llm-d
  • Familiarity with Amazon Sage Maker Jump Start, Sage Maker Endpoints, and Amazon Bedrock for managed model hosting
  • Experience with online LLM quality monitoring (e. g., hallucination, toxicity, drift detection) and tracing via Open Telemetry conventions
  • Contributions to open-source LLM serving or inference projects (e. g., v LLM, llm-d, Ray, KServe, Triton)

Lead Software Engineer - LLM Ops Platform Reliability in Glasgow employer: JPMorganChase

JPMorgan Chase is an exceptional employer, offering a dynamic work environment where strategic thinkers can thrive. With a strong commitment to employee growth, comprehensive benefits, and a culture that values diversity and inclusion, you will have the opportunity to influence leadership effectiveness and organizational performance in one of the largest technology organisations. Join us to be part of a legacy that spans over 200 years while shaping the future of financial services.

JPMorganChase

Contact Details:

JPMorganChase Recruitment Team

We think you need these skills to ace Lead Software Engineer - LLM Ops Platform Reliability in Glasgow

Site Reliability Engineering
Python
AWS
Terraform
Kubernetes
Observability
Incident Management