At a Glance
- Tasks: Build a cutting-edge trading data platform from scratch using open-source technologies.
- Company: Dynamic London trading firm focused on innovation and collaboration.
- Benefits: Competitive salary, hands-on experience, and mentorship from senior engineers.
- Other info: 3 days on-site in London, excellent growth opportunities in a fast-paced environment.
- Why this job: Join a team where you can make a real impact and learn rapidly.
- Qualifications: Strong SQL and Python skills, with experience in building production data pipelines.
The predicted salary is between 59400 - 72600 £ per year.
Senior Data Engineer | Trading Analytics | London, 3 days on site
A London Trading firm is building a trading data platform from scratch, in house, on open source. Spark, Kafka, Polaris, Ranger, Airflow, Min IO. This is the person who builds it.
Why now
The analytics function is running on legacy warehouses that were never built for the volume or the velocity of trading data.
They are consolidating them into a single lakehouse handling streaming and batch together, and building it in house rather than buying it.
A Principal Data Engineer is being hired alongside this role to own the architecture.
You will be the one turning that design into a working platform.
- What you would do
- Build ingestion pipelines for both streaming and batch sources
- Write the Spark jobs that clean, transform and land data in the lakehouse
- Build and own Airflow orchestration
- Work on cataloguing and access control with Polaris and Ranger
- Help migrate data off the legacy warehouses and retire them
- What we need
- Strong SQL and Python. These are the working languages of the platform.
- Real experience building production data pipelines, not just maintaining them
- Spark, and some exposure to Kafka or another streaming technology
- Comfort with infrastructure that someone runs themselves rather than only managed cloud services
- Financial markets background, any asset class. Commodities not required.
- Useful, not essential
- Java or another JVM language
- Hadoop, HDFS, S3-compatible object storage, open table formats
- Kubernetes, Terraform, CI/CD
- Who this suits
Someone two to six years into a data engineering career who wants to build a platform from the ground up rather than add features to one that already exists.
You will be closer to the metal here than in most roles at this level, and you will learn more in the first year than in three years of pipeline maintenance somewhere else.
If you want a title and a team, the Principal role is the one to look at instead.
The honest version
It is 3 days a week on site in London.
Some of the estate is on premise and some of what you inherit is legacy that needs retiring.
In exchange you get to build a modern lakehouse at, on open source, with someone senior to learn from.
Process
Three stages: an introductory call, a technical discussion built around real scenarios rather than a timed coding test, and a conversation with the CTO.
Moving quickly, filling before year end.
Data Engineer in London employer: Bourne Search Ltd
Join a dynamic London trading firm where you will have the unique opportunity to build a cutting-edge trading data platform from the ground up. With a strong emphasis on innovation and collaboration, our work culture fosters continuous learning and professional growth, allowing you to develop your skills alongside experienced professionals in the field. Enjoy the benefits of a flexible work schedule with three days on-site, while contributing to a transformative project that will redefine our analytics capabilities.
