Data Engineer – AWS, PySpark & AI in London

Data Engineer – AWS, PySpark & AI in London

London Full-Time No working from home possible
D

Location: London - Three days in the office and two from home

Job reference: J13171

Build the data platform powering a new generation of AI-driven consumer intelligence. You’ll work at the intersection of AWS, large-scale behavioural data and agentic AI, creating scalable pipelines and data products that turn ethically sourced, consented consumer data into meaningful insights.

Joining a collaborative, scaling technology company, you’ll have the freedom to experiment with cutting-edge tools, broaden your experience and make a visible impact on the platform from day one.

The team uses Claude extensively to support development, debugging, testing and documentation. You’ll combine AI-assisted development with strong Python, SQL and Spark fundamentals to produce reliable, well-tested solutions.

What you’ll be doing

  • Designing, building and optimising scalable AWS data pipelines and ETL/ELT processes
  • Developing data solutions using Python, SQL and Spark
  • Processing large and complex behavioural datasets
  • Improving data quality, reliability, performance and monitoring
  • Using Claude and other AI-assisted development tools to support delivery
  • Collaborating with engineers, analysts and data scientists
  • Identifying opportunities to automate and improve data workflows
  • Contributing ideas as the platform and its data products continue to evolve

What we’re looking for

  • Hands‑on data engineering experience using Python and SQL
  • Practical Spark or PySpark experience
  • A good understanding of scalable ETL/ELT design and data quality
  • Experience using Claude or another AI-assisted development tool, with the ability to review and validate generated code
  • Strong communication skills and an interest in working within a small, collaborative team

This is an opportunity to work with cutting-edge data and AI technology, take ownership beyond one narrow area and help shape a platform as the business grows.

Applicants must already have the right to work in the UK, as sponsorship is not available for this position.

#J-18808-Ljbffr

D

Contact Details:

Datatech Analytics Recruitment Team