Applying for this role is straight forward Scroll down and click on Apply to be considered for this position.
hackajob is partnering directly with JPMorganChase to hire for this role.
JOB DESCRIPTION
Help shape how AI systems run reliably in production at scale. In this role, you'll build and operate large language model serving infrastructure, bringing strong engineering fundamentals and site reliability practices to cutting-edge AI platforms. You'll work hands-on with cloud and Kubernetes-based deployments, deep observability, and cost-aware performance tuning. If you enjoy solving hard production problems and making platforms measurably better, you'll find meaningful impact and growth here.
As a Lead Software Engineer at JPMorgan Chase in the AI and Machine Learning Platform team, you will build and scale AI infrastructure that modernizes traditional infrastructure management and site reliability engineering through applied AI. You will own the reliability, performance, and cost-efficiency of the LLM inference platform end to end. You will operate large language model serving stacks (such as vLLM and llm-d) in production at scale, with deep instrumentation and strong operational rigor. You will partner across engineering to deliver secure software, improve stability, and lead incident response and continuous improvement.
Job Responsibilities
- Design, develop, troubleshoot, and deliver secure, high-quality production software and services for AI infrastructure
- Build backend services and APIs that enable reliable operation of AI infrastructure in production
- Operate and scale LLM serving infrastructure (such as vLLM and llm-d), including model hosting, request routing, continuous batching, and KV-cache optimization
- Deploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon EKS and Amazon SageMaker, as well as on-prem and local GPU clusters, using reproducible infrastructure as xgikmsk code and continuous delivery pipelines
- Implement observability (logs, metrics, traces) with dashboards and actionable alerting, including Prometheus metrics and Grafana/Alertmanager integration for LLM and GPU workloads
- Tune GPU and accelerator capacity, autoscaling, and cost efficiency for LLM inference workloads using performance and optimization techniques (e.g., quantization, parallelism, speculative decoding)
- Lead reliability engineering for LLM endpoints through capacity planning, load/soak testing, safe rollouts Please click on the apply button to read the full job description
Lead Software Engineer - LLM Ops Platform Reliability in Glasgow employer: Hackajob
Joining Google as a Security Platform Engineer in the UK Public Sector means becoming part of a dynamic and innovative team dedicated to delivering secure private cloud services for critical customers. With a strong emphasis on employee growth, you will have access to cutting-edge technology and collaborative opportunities that foster professional development. The inclusive work culture at Google encourages creativity and teamwork, making it an exceptional employer for those seeking meaningful and impactful work in a supportive environment.
