You're a technically proficient data engineer who thrives on solving complex data challenges and takes pride in building robust, scalable solutions. You have a natural curiosity that drives you to understand not just how things work, but why they work, and you're always exploring ways to improve and optimise systems. You're comfortable working autonomously whilst also being a strong collaborator who values input from colleagues across different teams. You approach problems pragmatically, balancing technical excellence with business needs, and you're not afraid to roll up your sleeves when issues arise. Your attention to detail ensures data quality and reliability, but you also keep the bigger picture in mind, understanding how your work impacts the broader organisation. You communicate technical concepts clearly to non-technical stakeholders and enjoy mentoring others when opportunities arise. Above all, you're adaptable and embrace change in a fast-moving technical landscape. You're eager to learn new tools and technologies, and you bring a positive, solution-oriented mindset to every challenge. About the role ... We are seeking a skilled Data Engineer to join our team and play a key role in designing, building, and maintaining our data infrastructure. You'll work with large-scale datasets and cutting-edge technologies to create robust data pipelines that power business insights and decision-making. Role responsibilities Design, develop, and maintain scalable data pipelines using Apache Spark and AWS Glue Architect and implement data solutions leveraging AWS services including S3, Redshift, EMR, Lambda, and Kinesis Ensure data quality, reliability, and performance across all data systems Monitor and troubleshoot data pipeline performance and resolve issues proactively Document technical designs, data flows, and system architectures Build and optimise ETL/ELT processes to transform raw data into actionable insights Develop Python-based applications and scripts for data processing and automation Collaborate with data analysts, scientists, and business stakeholders to understand data requirements Implement data governance and security best practices in compliance with UK GDPR Role requirements Strong proficiency in Python for data engineering tasks Hands-on experience with Apache Spark for distributed data processing Proven experience with AWS Glue for ETL job orchestration Solid understanding of AWS data architecture and services (S3, Redshift, EMR, Lambda, etc.) Experience designing and implementing data warehouses and data lakes Strong SQL skills and experience with relational and NoSQL databases Understanding of data modelling principles and best practices Experience with version control systems (Git) Right to work in the UK AWS certifications (Solutions Architect, Data Analytics, or similar) Experience with Infrastructure as Code tools (Terraform, CloudFormation) Knowledge of real-time data streaming technologies (Kinesis, Kafka) Familiarity with orchestration tools (Apache Airflow, Step Functions) Experience with data visualisation and BI tools Understanding of machine learning workflows and MLOps practices Knowledge of UK data protection regulations and compliance requirements #J-18808-Ljbffr
