At a Glance
- Tasks: Build and run an evaluation framework for cutting-edge software benchmarks.
- Company: Join a pioneering tech company focused on innovative computer-use agents.
- Benefits: Enjoy a competitive salary, hybrid work model, and opportunities for travel to Paris.
- Other info: Dynamic team environment with excellent career growth and mentorship opportunities.
- Why this job: Make a real impact by enhancing software evaluation processes and collaborating with top engineers.
- Qualifications: 5+ years in backend development, strong Python skills, and experience with distributed systems.
The predicted salary is between 63000 - 77000 £ per year.
About H
When we released Holo4 on 28 September, we published every trajectory behind its public benchmark scores at trajectories. hcompany. ai.
For OSWorld that's 369 desktop tasks, each run 3 times, with the steps, tokens and time of every attempt.
This role builds and runs the evaluation framework that produces runs like these.
H builds computer-use agents and the models behind them. Developers use them through a managed API, and our forward deployed engineers take them into enterprise workflows.
What this team owns
The evaluation framework: orchestration, runtimes and observability.
Researchers and forward deployed engineers bring the benchmarks, across web apps, desktop applications and the command line.
Your job is to make the framework that runs them reliable, fast and cheap, and to make adding a new one quick.
Research uses the results to choose checkpoints and decide whether a model ships.
Product and the forward deployed engineers use them to measure agents on customer workflows.
It carries roughly 50 benchmarks now.
That number should be between 100 and 200 soon, and the framework has to keep up.
- What you'd be doing
- Integration support for researchers and forward deployed engineers bringing in a benchmark, with a shorter path each time.
- Setting the standard for how a benchmark enters the framework, and building the checks that enforce it.
- Scheduling and observability, so cluster capacity isn't left idle while evaluation jobs queue.
- Reproducible results across trials, so a release decision rests on numbers that hold.
- Whatever stack a benchmark calls for. One week that's cluster tuning; the next it's a browser extension or desktop environments.
- Time with customers, from single developers to large companies, to find out what they want measured, then automating it so the results flow back into our harnesses and models.
- The first few months
By 3 months you'll have helped researchers or forward deployed engineers integrate 5 benchmarks, and started fixing what slows the framework down.
By 6 months one part of it is yours, for example scaling the runs, observability, or a group of related benchmarks, and a release will have gone out on your numbers.
By 12 months you'll know the design and trade-offs of the whole evaluation system, and be the person the rest of H asks about evaluations.
Who you'd work with
Ceiran Chapman, our VP Engineering, is hiring for this role.
You'd join the evaluation team.
The people relying on your work day to day are H's researchers and forward deployed engineers.
- What we think it takes
- Likely a good fit if you
- Have spent 5+ years in backend development, with production Python at the core, and use coding agents to go faster without letting quality drop.
- Have built test, QA or evaluation tooling that other teams depended on, and care whether a number is right.
- Have operated distributed systems on Kubernetes in a public cloud. AWS experience helps most.
- Have built and shipped systems end to end, including APIs (REST or Graph QL) and integrations with outside services.
- Know relational and non-relational databases, and message queues such as SQS, Rabbit MQ or Kafka.
- Instrument what you build, with metrics, tracing and monitoring from the start.
- Stronger still if you have
- Measured LLM quality before, or built agents yourself.
- Packaged and run workloads in Docker and on virtual machines.
- Used Temporal, Dask, Fast API, Postgre SQL, Grafana or Datadog.
- Automated web or desktop software with Playwright, Selenium or a browser extension you wrote.
- Set standards other engineers follow, through code review, design review or mentoring.
You do not need a background in machine learning. We'll work that out with you.
How we hire
A 30 minute call with our Talent team, a 60 minute technical challenge, a 60 minute system design interview, and a 30 minute final conversation with Ceiran.
About 3.5 hours in total.
Practicalities
London posting
Hybrid in London. That means 3 office days a week and a Paris trip about once every 4 to 6 weeks. There is a Paris posting for the same role. We offer a competitive package.
Senior Software Engineer - London employer: H Company
H Company is an exceptional employer, offering a dynamic work culture that fosters innovation and collaboration in the fast-paced AI sector. With a strong emphasis on employee growth, you will have access to cutting-edge projects and the opportunity to work alongside talented professionals in either Paris or London, all while enjoying the flexibility of a hybrid work model. Join us to be part of a mission-driven team that values your contributions and supports your professional development.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Software Engineer - London
✨Join Local Tech Meetups
Get out there and mingle with fellow developers by joining local tech meetups. It’s a fantastic way to meet people who might be working at H Company or know someone who does. Plus, you can pick up some trendy tech skills and trends while you're at it!
✨Contribute to Open Source Projects
Show off your coding chops by jumping into open-source projects. Not only does this give you practical experience, but it also gets you noticed in the dev community. You'll create a killer portfolio that speaks volumes about your skills to H Company.
✨Tap into Online Developer Communities
Don’t underestimate the power of online developer communities like GitHub, Stack Overflow, and even Reddit. Participate in discussions, share your projects, and build your visibility. We can often find opportunities through these channels that can lead to a full-time gig at companies like H Company.
✨Explore Job Boards Specifically for Tech Roles
Keep your eyes peeled on job boards that focus on tech roles. Sites like TechCareers or Stack Overflow Jobs can often have listings for companies like H Company that might not show up on broader job sites. Make it a habit to check these regularly, and don’t hesitate to apply directly through our website!
We think you need these skills to ace Senior Software Engineer - London
Some tips for your application 🫡
Show off your coding skills:When applying for a software engineering role, it's super important to showcase your coding skills. Make sure your CV includes your tech stack, any relevant programming languages you’re comfortable with, and examples of projects you've worked on. If you have a GitHub profile, link it up! We love to see code in action.
Tailor your portfolio:For a full-time role, we’d expect to see some solid examples of your work in your portfolio. Make sure to include at least two or three projects that highlight your problem-solving skills and your ability to work with different technologies. Focus on the projects that are most relevant to the position at H Company.
Craft a killer cover letter:Your cover letter is your chance to stand out—make it personal! Explain why you want to work at H Company and how your skills align with the role. Show us your passion for software development. We dig enthusiastic candidates who understand the value of collaboration and continuous learning!
Be clear and concise:When it comes to writing your CV and cover letter, clarity is key. Avoid jargon that could confuse us and stick to simple, direct language. Highlight your achievements with quantifiable results where possible, and keep everything easy to read. A well-organised application goes a long way!
How to prepare for a job interview at H Company
✨Brush Up on Your Coding Skills
For a full-time software engineering role, it's crucial that we stay sharp with our coding abilities. Expect technical questions that might involve solving problems on the spot or discussing algorithms. Practise on platforms like LeetCode or HackerRank to get comfortable with the types of questions that often come up.
✨Know Your Tools and Frameworks
Make sure we’re well-acquainted with the tools and technologies listed in the job description. Familiarise ourselves with any specific frameworks or programming languages mentioned. If H Company uses React or Node.js, for instance, be ready to discuss how we’ve used them in previous projects or coursework.
✨Showcase Your Projects
Bring along a portfolio that highlights our best work. This could be code samples, GitHub repositories, or any side projects we’ve built. Make sure we can talk through our thought process for each project, especially the challenges we faced and how we solved them—this shows our problem-solving skills in action.
✨Prepare for Behavioural Questions
While technical skills are key, full-time positions also require cultural fit. Be ready to discuss our previous experiences and how we handle teamwork, conflict, and deadlines. Brush up on the STAR method—Situation, Task, Action, Result—to clearly articulate our past experiences when discussing how we've contributed to a team.