At a Glance
- Tasks: Lead the design and optimisation of a large-scale observability platform.
- Company: Innovative tech company in Cambridge with a focus on cloud infrastructure.
- Benefits: Competitive salary, flexible working options, and opportunities for professional growth.
- Other info: Join a dynamic team tackling complex engineering challenges.
- Why this job: Be the expert shaping impactful infrastructure solutions at scale.
- Qualifications: 15+ years in distributed systems, strong Kubernetes and Golang experience.
The predicted salary is between 80000 - 100000 £ per year.
Our client is scaling a large, distributed cloud platform and is looking for a Principal Engineer to act as the Subject Matter Expert (SME) across observability and cloud infrastructure. You will be working at serious scale managing thousands of Kubernetes nodes, handling tens of terabytes of logs daily, and supporting millions of real-time metrics across a highly distributed environment.
This is a senior, hands-on role where you will own the technical direction and standards of the observability ecosystem. As the SME, you will define best practice, guide architectural decisions, and act as the go-to expert across engineering teams, ensuring scalable, cost-efficient, and high-performance systems.
Key Responsibilities
- Act as the SME for observability and cloud infrastructure across the organisation
- Lead architecture across metrics, logs, and tracing systems
- Design and optimise high-throughput data pipelines and storage layers
- Implement strategies such as sampling, aggregation, and down-sampling
- Extend and enhance open-source observability tools at scale
- Partner with engineering teams to standardise tooling and improve adoption
- Drive reliability, scalability, and cost optimisation across the platform
- Define and promote best practices aligned with OpenTelemetry and modern observability standards
- Mentor engineers and elevate engineering quality across teams
Tech Environment
- Kubernetes at scale (thousands of nodes)
- High-volume telemetry (hundreds of thousands of events per second)
- Observability stack: Prometheus, Grafana, Loki, Tempo, ELK/OpenSearch, ClickHouse
- Multi-cloud (AWS, GCP)
- Infrastructure as code (Terraform), CI/CD pipelines
What We're Looking For
- 15+ years building and scaling distributed systems
- Strong hands-on experience with Golang (plus Python or Shell)
- Deep expertise in observability at scale
- Strong Kubernetes and cloud infrastructure experience
- Proven ability to design systems for performance, scale, and cost efficiency
- Experience with service mesh technologies (e.g. Istio/Envoy)
- Ability to operate as a technical authority and trusted advisor across teams
Nice to Have
- Open-source or CNCF contributions
- Experience using AI tools to improve engineering efficiency
Why Join
- Be the go-to expert shaping a large-scale observability platform
- Work on complex, high-impact infrastructure challenges
- Strong ownership and influence at Principal level
Principal Engineer employer: Synergetic Recruitment Group Limited
Join a forward-thinking company in Cambridge that values innovation and expertise, offering a dynamic work culture where your contributions directly impact the scalability of a large, distributed cloud platform. With a strong focus on employee growth, you will have the opportunity to mentor fellow engineers and shape best practices in observability, all while enjoying a collaborative environment that encourages technical excellence and creativity.
Contact Details:
Synergetic Recruitment Group Limited Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land Principal Engineer
✨Tip Number 1
Network like a pro! Reach out to folks in your industry on LinkedIn or at meetups. We all know that sometimes it’s not just what you know, but who you know that can land you that Principal Engineer role.
✨Tip Number 2
Show off your skills! Create a portfolio or GitHub repository showcasing your work with Kubernetes, observability tools, and any open-source contributions. We want to see your hands-on experience in action!
✨Tip Number 3
Prepare for those technical interviews! Brush up on your knowledge of cloud infrastructure and distributed systems. We recommend practicing common interview questions and even doing mock interviews with friends or colleagues.
✨Tip Number 4
Apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive about their job search!
We think you need these skills to ace Principal Engineer
Some tips for your application 🫡
Tailor Your CV:Make sure your CV reflects the skills and experiences that align with the Principal Engineer role. Highlight your expertise in observability, cloud infrastructure, and any relevant projects you've led.
Craft a Compelling Cover Letter:Use your cover letter to tell us why you're the perfect fit for this role. Share specific examples of how you've tackled challenges in distributed systems and your approach to mentoring engineers.
Showcase Your Technical Skills:Don’t shy away from listing your technical proficiencies, especially in Kubernetes, Golang, and observability tools. We want to see how you’ve applied these skills in real-world scenarios.
Apply Through Our Website:For the best chance of getting noticed, make sure to apply through our website. It helps us keep track of your application and ensures it reaches the right people quickly!
How to prepare for a job interview at Synergetic Recruitment Group Limited
✨Know Your Tech Inside Out
Make sure you’re well-versed in the technologies mentioned in the job description, especially Kubernetes, Golang, and observability tools like Prometheus and Grafana. Brush up on your knowledge of distributed systems and be ready to discuss your hands-on experience with them.
✨Showcase Your Problem-Solving Skills
Prepare to share specific examples of how you've tackled complex engineering challenges in the past. Think about situations where you had to optimise performance or scale systems effectively, and be ready to explain your thought process and the outcomes.
✨Demonstrate Leadership and Mentorship
As a Principal Engineer, you'll need to guide others. Be prepared to discuss how you've mentored junior engineers or led teams in previous roles. Highlight any best practices you've implemented and how they improved team performance or project outcomes.
✨Align with Their Vision
Research the company’s approach to observability and cloud infrastructure. Be ready to discuss how your vision aligns with theirs, particularly around cost efficiency and scalability. Showing that you understand their goals will set you apart as a candidate who’s genuinely interested in contributing.
