Software Engineer

Senior Python Data Engineer | AWS | Spark | Airflow | Public Sector Location: UK Remote / Hybrid Rate: Competitive Daily Rate Duration: Long-Term Contract Sector: Public Sector We are supporting a major public sector programme and are looking for an experienced Senior Data Engineer to join a high-performing cloud data team delivering business-critical data solutions. About the Role This is an opportunity to work on large-scale data platforms, helping design, build and optimise modern cloud-native data services that support analytics, reporting and operational decision-making. Responsibilities Design, develop and maintain scalable data pipelines and ETL/ELT solutions Build and optimise cloud-native data platforms within AWS Develop Spark and PySpark workloads for large-scale data processing Create and maintain Apache Airflow workflows and orchestration pipelines Implement Infrastructure as Code using Terraform Support batch and real-time data ingestion frameworks Work closely with data architects, analysts, data scientists and engineering teams Drive data quality, automation, monitoring and performance optimisation Contribute to CI/CD processes and engineering best practices Qualifications Experience Required Previous experience delivering within public sector or highly regulated environments Experience working on large-scale cloud data transformation programmes Strong stakeholder engagement and problem-solving skills Ability to work independently and contribute within agile delivery teams Required Skills Python AWS Terraform SQL ETL / ELT Development Data Integration Data Warehousing Preferred Skills AWS Glue EMR Lambda Databricks Kafka Docker Jupyter Notebooks Snowflake Data Governance