Zest is the first AI-driven platform empowering food & drink manufacturers to take action against food waste.
What you’ll work on
- Own the foundation: Take full ownership of our data platform - improving pipeline reliability and scalability today while shaping our long-term architecture.
- Evaluate and improve the platform. Recommend what to keep, change or simplify, explain the trade-offs and deliver agreed improvements.
- Help us scale safely across customers. Work with engineering to improve how we separate customer data and access, provision resources, manage concurrent runs and backfills, and contain the impact of failures.
- Build dependable integrations. Bring different customer systems and delivery formats into tested, maintainable pipelines that fit our shared data model.
- Make processing safe and repeatable. Handle changing schemas, late or corrected data, historical backfills and reruns without losing or duplicating valid data.
- Make problems easier to investigate. Improve testing, monitoring and visibility into how data moves through the pipeline, and help the team resolve failures.
- Build trust in the data. Work with the data science team to turn business rules and validation criteria into automated checks, helping us identify questionable results even when processing succeeds.
The experience we're looking for
- Strong Python and SQL, used to build and maintain production data systems.
- Production Dagster experience. You’ve built, operated and scaled Dagster pipelines across multiple customers or independently owned datasets, with hands-on experience of tenant isolation, resource allocation and managing concurrent workloads.
- Integrating complex source data, understanding inconsistencies and bringing different sources into a clear, usable data model.
- Reliable data processing, including schema changes, incremental updates and safe reprocessing of historical or corrected data.
- An analytical approach to data quality, using exploration, automated tests, monitoring and reconciliation to investigate problems and prevent them recurring.
- Sound engineering judgement and clear communication, including evaluating existing systems, prioritising practical improvements and explaining decisions to technical and non-technical colleagues.
Helpful experience
- Manufacturing or other operational data, including ERP systems, production records or sensor data.
- BigQuery, dbt or GCP.
- Data platforms that provide analytics directly to customer-facing products.
- Infrastructure and delivery tooling such as Terraform, automated deployment and GitHub Actions.
Our Benefits
- Equity package
- Flexible holidays
- Work from abroad (1 month per year)
- Enhanced maternity / paternity packages
- Flexible working hours
- Family friendly benefits
Senior Data Engineer Central London, hybrid · Full-time → employer: ZEST SOLUTIONS
ZEST SOLUTIONS is an attractive employer due to its enhanced benefits package and commitment to employee development. The company fosters a supportive team environment where contributions are valued, and there are ample opportunities for career advancement.