PySpark Developer in London

We are looking for an experienced PySpark Developer with strong Microsoft Fabric and Azure engineering skills to join a major transformation programme within the financial-markets domain. This role is fully hands-on, focused on building and optimising large-scale data pipelines, dataflows, semantic models, and lakehouse components. Key Responsibilities * Design, build and optimise Spark-based data pipelines for batch and streaming workloads * Develop Fabric dataflows, pipelines, and semantic models * Implement complex transformations, joins, aggregations and performance tuning * Build and optimise Delta Lake / delta tables * Develop secure data solutions including role-based access, data masking and compliance controls * Implement data validation, cleansing, profiling and documentation * Work closely with analysts and stakeholders to translate requirements into scalable technical solutions * Troubleshoot and improve reliability, latency and workload performance Essential Skills * Strong hands-on experience with PySpark, Spark SQL, Spark Streaming, DataFrames * Microsoft Fabric (Fabric Spark jobs, dataflows, pipelines, semantic models) * Azure: ADLS, cloud data engineering, notebooks * Python programming; Java exposure beneficial * Delta Lake / Delta table optimisation experience * Git / GitLab, CI/CD pipelines, DevOps practices * Strong troubleshooting and problem-solving ability * Experience with lakehouse architectures, ETL workflows, and distributed computing * Familiarity with time-series, market data, transactional data or risk metrics Nice to Have * Power BI dataset preparation * OneLake, Azure Data Lake, Kubernetes, Docker * Knowledge of financial regulations (GDPR, SOX) Details * Location: London (office-based) * Type: Contract * Duration: 6 months * Start: ASAP * Rate: Market rates If you are a PySpark / Fabric / Azure Data Engineer looking for a high-impact contract role, apply now for immediate consideration