At a Glance
- Tasks: Design and optimise data pipelines for financial instruments using cutting-edge technologies.
- Company: Join a global leader in financial markets transformation.
- Benefits: Competitive salary, remote work options, and hands-on experience with Azure and Fabric.
- Other info: Collaborate with experts in a dynamic environment focused on career growth.
- Why this job: Make a real impact in a high-stakes engineering role with modern cloud architecture.
- Qualifications: Strong Python and PySpark skills, experience with data pipelines and Azure Cosmos DB.
The predicted salary is between 60000 - 80000 £ per year.
You will be part of a specialist engineering team responsible for designing, building, and optimising end-to-end financial instrument mastering pipelines. These pipelines span ingestion, normalisation, bi-temporal processing, and publication into enterprise data platforms. You will work closely with data architects, domain experts, and QC engineers to deliver scalable, reliable, and high-performance data solutions across Azure and Microsoft Fabric ecosystems.
Key Responsibilities
- Build and maintain PySpark-based data pipelines for financial instrument mastering across multiple data sources.
- Design and implement bi-temporal data processing models (system time + valid time) including Slice, Resolve, Coalesce, and Diff logic.
- Develop optimised Azure Cosmos DB data models, including partitioning, indexing, change feed processing, and point-read optimisation.
- Integrate external APIs for entity resolution and matching services (PermID / IAAS) with robust retry and batching mechanisms.
- Design publication pipelines to convert bi-temporal data into uni-temporal outputs and publish via Microsoft Fabric / Parquet-based lakehouse architectures.
- Implement data quality frameworks using Great Expectations to ensure accuracy and compliance.
- Build robust unit and integration tests using PyTest for PySpark and Cosmos DB components.
- Support and maintain CI/CD pipelines (GitLab CI) including Python packaging, Artifactory deployment, and ARM-based infrastructure provisioning.
- Work with YAML-driven configuration for mastering rules, schemas, and environment setup.
- Monitor and troubleshoot production pipelines using Eventstream telemetry, KQL, and DataDog observability tools.
- Deliver scalable transformation logic, optimised aggregations, and high-performance data processing workflows.
- Implement data governance controls including data masking, role-based access, and compliance policies.
- Continuously tune and optimise workloads for performance, cost efficiency, and reliability.
Required Skills & Experience
- Strong experience in Python and PySpark (Spark SQL, DataFrame API, Structured Streaming).
- Hands‑on experience building large‑scale ETL / streaming data pipelines.
- Experience working with Azure Cosmos DB (NoSQL) including data modelling and performance tuning.
- Strong knowledge of Azure Data Lake Storage (ADLS / OneLake / ABFS).
- Experience implementing bi-temporal or SCD Type 2 data models.
- Strong understanding of data quality frameworks (e.g., Great Expectations).
- Experience with CI/CD pipelines (GitLab / Azure DevOps) and automated deployments.
- Strong testing discipline using PyTest, mocking, and integration testing approaches.
- Experience working with YAML/JSON configuration and infrastructure-as-code (ARM templates).
- Strong understanding of distributed data processing and Spark-based architectures.
- Experience working with financial or time‑series datasets (market data, reference data, risk data preferred).
- Strong communication skills and ability to work with cross‑functional stakeholders.
Why Join
- Work on a global financial markets transformation programme.
- Hands-on with next‑generation Azure + Fabric data platforms.
- Exposure to bi‑temporal modelling and financial instrument mastering systems.
- High‑impact engineering role with modern cloud and streaming architecture.
- Opportunity to work with leading domain and technical experts in a regulated environment.
Developer employer: Queen Square Recruitment
Join a forward-thinking company that prioritises innovation and collaboration, offering developers the chance to work on cutting-edge Azure and Microsoft Fabric data platforms. With a strong focus on employee growth, you will have access to hands-on experience in a high-impact engineering role, alongside leading domain experts in a dynamic and supportive work culture. Enjoy competitive benefits and the opportunity to contribute to a global financial markets transformation programme, making your work both meaningful and rewarding.
StudySmarter Expert Advice🤫
We think this is how you could land Developer
✨Tip Number 1
Network like a pro! Reach out to folks in the industry, attend meetups, and connect with people on LinkedIn. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving PySpark and Azure. This gives potential employers a taste of what you can do and sets you apart from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on your technical skills and understanding the company’s tech stack. Practice coding challenges and be ready to discuss your past projects in detail—especially those related to financial data processing.
✨Tip Number 4
Don’t forget to apply through our website! We’re always on the lookout for talented developers like you. Plus, it’s a great way to ensure your application gets seen by the right people.
We think you need these skills to ace Developer
Some tips for your application 🫡
Tailor Your CV:Make sure your CV reflects the skills and experiences that match the job description. Highlight your experience with Python, PySpark, and Azure Cosmos DB, as these are key for us at StudySmarter.
Craft a Compelling Cover Letter:Use your cover letter to tell us why you're passionate about data engineering and how your background aligns with our mission. Be specific about your experience with data pipelines and bi-temporal processing.
Showcase Your Projects:If you've worked on relevant projects, whether in a professional or personal capacity, make sure to mention them. We love seeing practical examples of your work with ETL processes and data quality frameworks.
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for this exciting opportunity in our engineering team!
How to prepare for a job interview at Queen Square Recruitment
✨Know Your Tech Stack
Make sure you’re well-versed in Python and PySpark, as these are crucial for the role. Brush up on your knowledge of Azure Cosmos DB and data modelling techniques, especially bi-temporal processing. Being able to discuss your hands-on experience with these technologies will show that you're ready to hit the ground running.
✨Showcase Your Problem-Solving Skills
Prepare to discuss specific challenges you've faced while building ETL or streaming data pipelines. Think about how you optimised performance or resolved issues in production. This will demonstrate your ability to troubleshoot and innovate, which is key for this role.
✨Understand the Business Context
Familiarise yourself with financial datasets and compliance standards like GDPR and SOX. Being able to connect your technical skills to real-world applications in finance will impress the interviewers and show that you understand the bigger picture.
✨Ask Insightful Questions
Prepare thoughtful questions about the team’s current projects, the tools they use, and their approach to data governance. This not only shows your interest in the role but also helps you gauge if the company culture aligns with your values and work style.
