At a Glance
- Tasks: Design and build scalable data pipelines using AWS, Databricks, and PySpark.
- Company: Join a high-impact engineering team in a collaborative environment.
- Benefits: Competitive salary, hybrid work model, and opportunities for professional growth.
- Other info: Start in August and enjoy a dynamic workplace with excellent career prospects.
- Why this job: Work with cutting-edge technology and make a real impact on marketing insights.
- Qualifications: Experience in cloud environments and strong skills in AWS, PySpark, and SQL.
The predicted salary is between 80000 - 90000 £ per year.
- Senior Data Engineer| AWS/Databricks/Py Spark | London/Glasgow (Hybrid) | August Start
- Role: Senior Data Engineer
Location: This is a hybrid engagement represented by 2 days/week onsite, either in Central London or Glasgow.
About The Role
Our partner is looking for a Senior Data Engineer to join a high-impact engineering team delivering scalable data solutions for complex marketing and customer insight use cases.
This is an opportunity to work on cutting-edge data pipelines, cloud-native platforms and real-time data flows in a collaborative, forward-thinking environment.
You'll be involved in designing and building production-grade ETL pipelines, driving Dev Ops practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem.
Responsibilities
- Architect and build scalable, secure data pipelines using AWS, Databricks and Py Spark.
- Design and implement robust ETL/ELT solutions for both structured and unstructured data.
- Automate workflows and orchestrate jobs using Airflow and Git Hub Actions.
- Integrate data with third-party APIs to support real-time marketing insights.
- Collaborate closely with cross-functional teams including Data Science, Software Engineering and Product.
- Champion best practices in data governance, observability and compliance.
- Contribute to CI/CD pipeline development and infrastructure automation (Terraform, AWS Dev Ops).
- Provide input into technical decisions, peer reviews and solution design.
Requirements
- Proven experience as a Data Engineer in cloud-first environments.
- Strong commercial knowledge of AWS services (e. g. S3, Glue, Redshift).
- Advanced Py Spark and Databricks experience (Delta Lake, Unity Catalog, Databricks Jobs etc).
- Proficient in SQL (T-SQL/Spark SQL) and Python for data transformation and scripting.
- Hands-on experience with workflow orchestration tools such as Airflow.
- Strong version control and Dev Ops exposure (Git, Git Hub Actions, Terraform).
- Familiar with data quality tools and metadata/cataloguing (e. g. Great Expectations, Unity Catalog).
- Beneficial: Mar Tech domain knowledge.
Notable: This is a hybrid engagement represented by 2 days/week onsite, either in Central London or Glasgow. You must be able to start in August.
Senior Data Engineer| AWS/Databricks/Py Spark | London/Glasgow (Hybrid) | August Start
