At a Glance
- Tasks: Lead integration of barge data into our global cargo intelligence platform using innovative tech.
- Company: Join Kpler, a dynamic company transforming global trade information since 2014.
- Benefits: Enjoy competitive salary, inclusive culture, and opportunities for professional growth.
- Other info: Collaborative environment focused on mentorship and career development.
- Why this job: Make a real impact in commodities and energy sectors while working with cutting-edge technology.
- Qualifications: 5+ years in data engineering, advanced Python & SQL skills, and Kafka streaming experience.
The predicted salary is between 72000 - 88000 £ per year.
At Kpler, we are dedicated to helping our clients navigate complex markets with ease. By simplifying global trade information and providing valuable insights, we empower organisations to make informed decisions in commodities, energy, and maritime sectors. Since our founding in 2014, we have focused on delivering top-tier intelligence through user-friendly platforms. Our team of over 850 experts from 69 countries works tirelessly to transform intricate data into actionable strategies, ensuring our clients stay ahead in a dynamic market landscape. Join us to leverage cutting-edge innovation for impactful results and experience unparalleled support on your journey to success.
About the Role
As a Senior Software Engineer within our Commodities tribe, you will lead the end-to-end integration of US inland waterway barge data into Kpler’s global cargo intelligence platform. Operating under a "you build it, you run it" philosophy, you will design robust ingestion paths, stream real-time data using Kafka, and connect barge movement models directly to our customer-facing surfaces. This role carries genuine design authority, empowering you to solve complex entity resolution problems while mentoring peers and elevating technical craft across our crews.
Key Responsibilities
- Lead end-to-end integration architecture: Drive the full engineering lifecycle—from external feed ingestion and domain modeling to streaming, entity resolution, and UI distribution.
- Design robust ingestion & entity-resolution systems: Map incoming barge data onto Kpler reference data (vessels, zones, installations, products) using existing Elasticsearch and PostgreSQL matching patterns.
- Build scalable Kafka streaming pipelines: Construct event-driven contracts, schemas (Avro), deduplication mechanisms, and dead-letter queue replays with Python and Scala.
- Deliver data across customer surfaces: Surface integrated barge data cleanly across Elasticsearch read models, external APIs, data warehouse feeds, and the Kpler Terminal (TypeScript/Vue).
- Champion production quality & observability: Maintain full operational ownership of your services, setting SLOs, participating in on-call rotations, and driving post-incident improvements.
- Ensure data integrity & analyst alignment: Validate external feeds and integrate human-in-the-loop analyst workflows so corrections seamlessly apply to barge movements.
- Mentor and elevate engineering craft: Guide and mentor crew members, document architectural designs, and actively contribute to decomposing legacy estates into event-driven services.
Experience & Background
What you'll need (Must-haves)
- Production data engineering experience: ~5+ years of experience designing, operating, and maintaining data-intensive production systems with full ownership.
- Advanced Python & SQL expertise: Proven ability to build clean, well-bounded services within large, complex, and evolving codebases.
- Event-driven streaming proficiency: Hands-on experience with Kafka streaming in production, including schema management (Avro/Schema Registry), deduplication, and replay patterns.
- Third-party integration & entity resolution: Demonstrated background integrating external data feeds, managing schema mapping, and building data quality monitoring frameworks.
- Operational mindset: Direct experience managing production observability, writing documentation, and resolving incidents in a distributed system environment.
Nice-to-haves
- Functional programming & Scala: Experience or familiarity with Scala (Kafka Streams) or a track record of rapidly mastering new languages.
- Geospatial & domain knowledge: Familiarity with geospatial data tools (PostGIS) or exposure to maritime, logistics, or commodities domains.
- Modern data stack exposure: Experience with Elasticsearch, Airflow, Snowflake/lakehouse table formats, Kubernetes, and GitOps deployments.
- UI/Frontend collaboration: Working knowledge of TypeScript or Vue to collaborate on user-facing terminal features.
We are a dynamic company dedicated to nurturing connections and innovating solutions to tackle market challenges head-on. If you thrive on customer satisfaction and turning ideas into reality, then you’ve found your ideal destination. Are you ready to embark on this exciting journey with us?
Our People Pledge
Don’t meet every single requirement? Research shows that women and people of colour are less likely than others to apply if they feel like they don’t match 100% of the job requirements. Don’t let the confidence gap stand in your way, we’d love to hear from you! We understand that experience comes in many different forms and are dedicated to adding new perspectives to the team.
Kpler is committed to providing a fair, inclusive and diverse work-environment. We believe that different perspectives lead to better ideas, and better ideas allow us to better understand the needs and interests of our diverse, global community. We welcome people of different backgrounds, experiences, abilities and perspectives and are an equal opportunity employer.
Back-end Engineer employer: Kpler
Kpler is an exceptional employer located in the vibrant city of London, offering a dynamic work culture that fosters collaboration and innovation. Employees benefit from comprehensive growth opportunities, including professional development and the chance to work alongside talented teams across various functions. With a focus on data-driven decision-making, Kpler empowers its workforce to make impactful contributions in a fast-paced environment, making it an ideal place for those seeking meaningful and rewarding careers in the tech industry.
StudySmarter Expert Advice🤫
We think this is how you could land Back-end Engineer
✨Join Local Tech Meetups
Get out there and mingle with fellow developers by joining local tech meetups. It’s a fantastic way to meet people who might be working at Kpler or know someone who does. Plus, you can pick up some trendy tech skills and trends while you're at it!
✨Contribute to Open Source Projects
Show off your coding chops by jumping into open-source projects. Not only does this give you practical experience, but it also gets you noticed in the dev community. You'll create a killer portfolio that speaks volumes about your skills to Kpler.
✨Tap into Online Developer Communities
Don’t underestimate the power of online developer communities like GitHub, Stack Overflow, and even Reddit. Participate in discussions, share your projects, and build your visibility. We can often find opportunities through these channels that can lead to a full-time gig at companies like Kpler.
✨Explore Job Boards Specifically for Tech Roles
Keep your eyes peeled on job boards that focus on tech roles. Sites like TechCareers or Stack Overflow Jobs can often have listings for companies like Kpler that might not show up on broader job sites. Make it a habit to check these regularly, and don’t hesitate to apply directly through our website!
We think you need these skills to ace Back-end Engineer
Some tips for your application 🫡
Show off your coding skills:When applying for a software engineering role, it's super important to showcase your coding skills. Make sure your CV includes your tech stack, any relevant programming languages you’re comfortable with, and examples of projects you've worked on. If you have a GitHub profile, link it up! We love to see code in action.
Tailor your portfolio:For a full-time role, we’d expect to see some solid examples of your work in your portfolio. Make sure to include at least two or three projects that highlight your problem-solving skills and your ability to work with different technologies. Focus on the projects that are most relevant to the position at Kpler.
Craft a killer cover letter:Your cover letter is your chance to stand out—make it personal! Explain why you want to work at Kpler and how your skills align with the role. Show us your passion for software development. We dig enthusiastic candidates who understand the value of collaboration and continuous learning!
Be clear and concise:When it comes to writing your CV and cover letter, clarity is key. Avoid jargon that could confuse us and stick to simple, direct language. Highlight your achievements with quantifiable results where possible, and keep everything easy to read. A well-organised application goes a long way!
How to prepare for a job interview at Kpler
✨Brush Up on Your Coding Skills
For a full-time software engineering role, it's crucial that we stay sharp with our coding abilities. Expect technical questions that might involve solving problems on the spot or discussing algorithms. Practise on platforms like LeetCode or HackerRank to get comfortable with the types of questions that often come up.
✨Know Your Tools and Frameworks
Make sure we’re well-acquainted with the tools and technologies listed in the job description. Familiarise ourselves with any specific frameworks or programming languages mentioned. If Kpler uses React or Node.js, for instance, be ready to discuss how we’ve used them in previous projects or coursework.
✨Showcase Your Projects
Bring along a portfolio that highlights our best work. This could be code samples, GitHub repositories, or any side projects we’ve built. Make sure we can talk through our thought process for each project, especially the challenges we faced and how we solved them—this shows our problem-solving skills in action.
✨Prepare for Behavioural Questions
While technical skills are key, full-time positions also require cultural fit. Be ready to discuss our previous experiences and how we handle teamwork, conflict, and deadlines. Brush up on the STAR method—Situation, Task, Action, Result—to clearly articulate our past experiences when discussing how we've contributed to a team.