This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
At UK Tote Group, we’re on a mission to reimagine the future of pool betting — building a modern, data-driven betting experience for millions of racing fans. Our technology powers real-time insights, supports responsible gaming, and helps us deliver trusted, customer-first products across the UK and international markets. As a Data Engineer, you’ll play a key role in designing, building, and optimising the Databricks-based Lakehouse that drives real-time data, analytics, and reporting across the Tote. You’ll build streaming and batch data pipelines on AWS using Apache Spark Structured Streaming, Delta Live Tables (DLT), and Kafka (MSK), ensuring our business teams across Liquidity, Product, Marketing, Finance, and Compliance have fast, trusted data at their fingertips. This is a hands-on engineering role where you’ll collaborate across engineering, BI, and product teams to deliver scalable, secure, and governed data solutions under Unity Catalogue.
Job Responsibility:
Design, build, and optimise data pipelines using Databricks, Spark Structured Streaming, and Delta Live Tables
Apply the Medallion Architecture to develop robust Bronze, Silver, and Gold Delta tables
Integrate data from Kafka (MSK), AWS S3, and external APIs
Work closely with BI teams to enable high-performance Power BI dashboards through Databricks SQL Warehouses
Ensure all data is governed, discoverable, and secure under Unity Catalog
Contribute to continuous delivery by implementing and maintaining CI/CD pipelines for Databricks jobs, notebooks, and DLT workflows
Monitor, tune, and troubleshoot pipeline performance using Databricks metrics, CloudWatch, and AWS Cost Explorer
Document data models, schemas, and lineage
Help ensure the platform remains fully compliant with GDPR and Gambling Commission regulations
Champion best practices in data platform design, observability, and cost management
Requirements:
Proven expertise in building pipelines in Databricks
Strong grasp of Apache Spark (PySpark or Scala), including Structured Streaming
Experience with Kafka (MSK) and real-time data ingestion
Deep understanding of Delta Lake, Delta Live Tables, and the Medallion Architecture