Senior Data Engineer Role (Python / Spark / Databricks) - Hybrid - Contract - Banking
Role - Senior Data Engineer
Rate - £850 p/d (Inside IR35)
Duration - 6 months with very likely extension
Location - Hybrid / Liverpool Street (London) - 3 days per week in a Liverpool Street office
About the Role
Global financial institution seeking a highly skilled Senior Data Engineer to join a growing data engineering function responsible for delivering critical data platforms and analytics capabilities across the business.
This is a genuinely hands-on engineering role, requiring strong expertise in Python, Databricks, Spark/PySpark and SQL. The successful candidate will be actively involved in the design, development, optimisation and support of enterprise-scale data platforms, working alongside engineers, architects and business stakeholders to deliver robust and scalable solutions.
The role offers the opportunity to work on complex data challenges within a modern cloud-based environment, leveraging the latest data engineering technologies and best practices.
Key Responsibilities
Data Engineering & Development
- Design, develop and maintain scalable data pipelines using Python, Databricks and PySpark
- Build and optimise ETL/ELT solutions processing large-scale datasets
- Develop high-quality, reusable code and engineering frameworks
- Deliver robust, performant and maintainable data solutions
- Troubleshoot and resolve complex production and performance issues
- Support the ongoing enhancement and modernisation of enterprise data platforms
Databricks & Data Platform Engineering
- Develop and support Databricks-based data solutions and Lakehouse architectures
- Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities
- Optimise distributed data processing workloads for performance and reliability
- Implement data quality controls and monitoring solutions
- Contribute to platform stability, scalability and operational excellence
Solution Delivery
- Work closely with business and technology stakeholders to understand requirements and deliver effective solutions
- Participate in Agile delivery processes including sprint planning, refinement and estimation
- Support the delivery of strategic data initiatives across multiple business areas
- Collaborate with architects, engineers and product teams to deliver high-quality outcomes
- Identify and resolve technical risks and implementation challenges
Engineering Best Practice
- Promote software engineering best practices including testing, code reviews and version control
- Support CI/CD adoption and deployment automation
- Contribute to coding standards and engineering governance
- Drive continuous improvement across development and support processes
- Mentor junior engineers and contribute to knowledge sharing within the team
Required Skills & Experience
- Strong commercial experience as a Data Engineer or Senior Data Engineer
- Expert-level Python development skills
- Extensive hands-on experience with Databricks in enterprise environments
- Strong Apache Spark and PySpark expertise
- Advanced SQL development and optimisation skills
- Experience designing and building large-scale ETL and data integration solutions
- Strong understanding of distributed computing concepts and data processing frameworks
- Experience working with modern cloud platforms including Azure, AWS or GCP
- Knowledge of software engineering principles, testing and code quality practices
- Experience working within Agile development environments
- Excellent problem-solving and troubleshooting capabilities
Desirable Experience
- Experience within investment banking, financial services or other highly regulated environments
- Exposure to risk, trading, treasury or regulatory data domains
- Experience with Kafka, streaming technologies or real-time data processing
- Databricks certifications
- Experience with Airflow, Databricks Workflows or similar orchestration tools
- CI/CD pipeline development and DevOps practices
- Docker and Kubernetes
- Experience supporting data science or machine learning workloads
This role will suit a Senior Data Engineer who enjoys remaining close to the technology and has strong recent experience building production-grade solutions using Python, Databricks, Spark/PySpark and SQL in enterprise environments.
#J-18808-Ljbffr
Senior Data Engineer in London employer: GCS Recruitment Specialists
Join a leading global organisation in London, where you will be at the forefront of a transformative cyber security programme that enhances the resilience of critical manufacturing environments. With a strong emphasis on employee growth and collaboration, the company fosters a dynamic work culture that values innovation and flexibility, offering competitive salaries and comprehensive benefits. This role not only provides the opportunity to lead significant projects but also to engage with diverse teams across the globe, ensuring your contributions have a meaningful impact on operational security.
Contact Details:
GCS Recruitment Specialists Recruitment Team
