Senior Data Engineer (Data EcoSystems)

At Kpler, we are dedicated to helping our clients navigate complex markets with ease. By simplifying global trade information and providing valuable insights, we empower organisations to make informed decisions in commodities, energy, and maritime sectors. Since our founding in 2014, we have focused on delivering top-tier intelligence through user-friendly platforms. Our team of over 700 experts from 35+ countries works tirelessly to transform intricate data into actionable strategies, ensuring our clients stay ahead in a dynamic market landscape. Join us to leverage cutting-edge innovation for impactful results and experience unparalleled support on your journey to success.

Kpler is hiring for a Senior Data Engineer to own the design, delivery, and reliability of platform capabilities that ingest and deliver data at scale. You’ll lead complex, high-impact initiatives that support critical business outcomes, shape engineering best practices across data ingestion, governance and distribution, while partnering closely with other teams to improve developer experience and platform adoption. The role also includes contributing to client‑facing solutions and data access features that enable customers to reliably consume Kpler’s data.

• Design and implement Snowflake ingestion pipelines in Java, sourcing data from upstream systems and delivering high-quality solutions for high-impact, ambiguous, and complex problem spaces.
• Own the reliability and operational excellence of ingestion services, including monitoring and observability, stability, performance, scalability, and on‑call readiness, while making sound engineering trade‑offs aligned with business needs.
• Maintain and evolve existing pipelines by identifying bottlenecks, reducing failure modes, and proactively addressing issues before they impact users or downstream systems.
• Lead technical design for complex initiatives, driving end‑to‑end architecture and implementation while coordinating across teams and stakeholders.
• Drive continuous improvement to developer experience for data ingestion by contributing to platform standards, improving usability and documentation, and partnering with other teams to enable adoption and consistent best practices.
• Refactor and modernise complex systems pragmatically, balancing technical debt against delivery timelines and proposing scope changes as priorities shift.
• Lead and debug production issues and major incidents under pressure, ensuring rapid mitigation, clear communication, and follow‑through on corrective actions.
• Contribute to the platform roadmap and cross‑organisational technical direction by participating in architecture forums and broader engineering initiatives that strengthen the discipline.
• Collaborate on platform‑adjacent projects as needed, including client‑facing data access and delivery solutions.
• Partner on data governance and risk communication, working effectively with functions such as compliance and clearly articulating technical risk and trade‑offs to non‑technical stakeholders.

• Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
• 5+ years of professional experience in software engineering and/or data engineering roles, with sustained ownership of production systems.
• Demonstrated experience building, operating, and evolving data ingestion pipelines (batch and/or streaming) for analytics platforms.
• Strong hands‑on proficiency in Java/Scala, Python and SQL, with solid data modelling fundamentals.
• Proven experience in software and systems design, applying best practices for performance, testability, maintainability, and operational robustness.
• Strong database and analytics warehouse fundamentals, including schema and table design for analytical workloads, query performance concepts, and practical experience with Snowflake or equivalent cloud data warehouses.
• Demonstrated ownership of non‑functional requirements, making sound engineering decisions that balance system design, technical debt, reliability, observability, and business needs.
• Solid DataOps and operational ownership mindset, including running production data services, implementing observability (metrics, logs, alerts), and improving reliability through automation and standard practices.
• Experience working with cloud platforms (AWS), including operating data platforms and services in modern environments.
• Exposure to deploying and operating services using Kubernetes and CI/CD pipelines, including GitOps‑style workflows with PR‑based deployments, environment promotion, and reproducible releases.
• Track record of delivering well‑designed, maintainable systems and owning reliability and observability for production workloads.

• Experience operating Snowflake at scale, including warehouse and resource sizing, cost and performance trade‑offs, workload optimisation, and operational best practices.
• Familiarity with data governance concepts such as access control, auditability, data retention, and lineage.
• Experience contributing to platform developer experience through tooling, documentation, and paved paths, and influencing engineering practices beyond the immediate team.
• Experience with TypeScript and Vue.js, integrating with APIs and presenting data within end‑user workflows such as Excel add‑ins.
• Experience with Python and boto3, and with AWS‑based client delivery patterns including S3 and sFTP‑style data distribution, relevant to data‑sharing platforms.

We make things happen. We act decisively and with purpose, going the extra mile. We build together. We foster relationships and develop creative solutions to address market challenges. We are here to help. We are accessible and supportive to colleagues and clients with a friendly approach.

Our People Pledge: Don’t meet every single requirement? Research shows that women and people of colour are less likely than others to apply if they feel like they don’t match 100% of the job requirements. Don’t let the confidence gap stand in your way, we’d love to hear from you! We understand that experience comes in many different forms and are dedicated to adding new perspectives to the team. Kpler is committed to providing a fair, inclusive and diverse work‑environment. We believe that different perspectives lead to better ideas, and better ideas allow us to better understand the needs and interests of our diverse, global community. We welcome people of different backgrounds, experiences, abilities and perspectives and are an equal opportunity employer.

By applying, I confirm that I have read and accept the Staff Privacy Notice. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analysing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.