Data Engineer

At UK Tote Group, we’re on a mission to reimagine the future of pool betting — building a modern, data-driven betting experience for millions of racing fans. Our technology powers real-time insights, supports responsible gaming, and helps us deliver trusted, customer-first products across the UK and international markets.

You’ll play a key role in designing, building, and optimising the Databricks-based Lakehouse that drives real-time data, analytics, and reporting across the Tote.

• You’ll design, build, and optimise data pipelines using Databricks, Spark Structured Streaming, and Delta Live Tables to ensure data flows efficiently and reliably across the organisation.
• By applying the Medallion Architecture, you’ll develop robust Bronze, Silver, and Gold Delta tables that serve as the foundation for analytics, APIs, and decision‑making tools.
• You’ll integrate data from Kafka (MSK), AWS S3, and external APIs, ensuring seamless ingestion into the Lakehouse.
• You’ll work closely with BI teams to enable high-performance Power BI dashboards through Databricks SQL Warehouses, making data more accessible and actionable across the business.
• You’ll ensure that all data is governed, discoverable, and secure under Unity Catalog, and you’ll contribute to continuous delivery by implementing and maintaining CI/CD pipelines for Databricks jobs, notebooks, and DLT workflows.
• Monitoring, tuning, and troubleshooting pipeline performance will also be part of your day‑to‑day, using Databricks metrics, CloudWatch, and AWS Cost Explorer to ensure optimal performance and efficiency.
• You’ll document data models, schemas, and lineage, maintaining a clear understanding of data flows and dependencies.
• Alongside Compliance and Technology colleagues, you’ll help ensure the platform remains fully compliant with GDPR and Gambling Commission regulations.
• Finally, you’ll champion best practices in data platform design, observability, and cost management, helping shape the future of the Tote’s data ecosystem.

• We’re looking for an experienced Data Engineer with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark or Scala, including Structured Streaming.
• You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta Lake, Delta Live Tables, and the Medallion Architecture.
• A strong AWS background is important, particularly with services such as S3, Glue, Lambda, Batch, and IAM.
• You’ll be proficient in Python and SQL for data engineering and analytics, and comfortable implementing CI/CD pipelines using tools such as GitHub Actions, Azure DevOps, or Jenkins.
• Solid experience with Git, version control, and Spark performance tuning will help you succeed in this role.
• Most importantly, you’ll bring a collaborative, proactive attitude and an ability to balance platform reliability with the pace of delivery.
• It would be an advantage if you have experience working with streaming architectures or data governance frameworks like Unity Catalog.
• Familiarity with Power BI, Looker, or Tableau is desirable, as is exposure to Databricks REST APIs, Airflow, or Databricks Workflows.
• Knowledge of infrastructure-as-code tools such as Terraform, AWS networking fundamentals, and cost management techniques using Photon and DBU monitoring will also be beneficial.
• You’ll be analytical, detail-oriented, and motivated by solving complex data challenges.
• A self-starter by nature, you take ownership of your work and enjoy designing end-to-end solutions that deliver measurable value.
• You’re a great communicator who can explain data concepts clearly to both technical and non-technical colleagues, and you have a passion for automation, efficiency, and data quality.
• Most of all, you’re curious and committed to continuous learning within a fast-evolving cloud data landscape.

At the Tote you can expect a friendly working environment with a strong sense of teamwork and pride in what we do. Within this role you’ll develop a broad range of skills and experiences that can enhance your career at the Tote. Additionally, our company benefits package includes:

• Competitive Basic Salary
• Discretionary Bonus Scheme
• Company Shares Option Plan
• Contributory pension scheme
• Life insurance (4 x basic salary)
• Simply Health Cash Plan
• Holiday entitlement (33 days inclusive of bank holidays)
• Study Support and opportunity for progression and development
• Confidential 24/7 365 employee assistance helpline
• Agile and collaborative office environment with free parking, fruit, biscuits, and drinks
• Regular social events, charity events and volunteering opportunities