At a Glance
- Tasks: Design and build scalable data pipelines while managing Azure platform components.
- Company: Join a leading company in data and analytics with a focus on innovation.
- Benefits: Enjoy a mostly remote role with competitive salary and career growth opportunities.
- Other info: Collaborative environment with a commitment to service excellence and continuous improvement.
- Why this job: Make an impact by optimising data solutions and mentoring junior engineers.
- Qualifications: 10+ years in data engineering, strong Python/PySpark skills, and Azure expertise.
The predicted salary is between 70000 - 90000 £ per year.
We are seeking a skilled and experienced Senior Data / Platform Engineer to join our Data & Analytics team. This hybrid role combines hands-on data engineering on Databricks and Azure Synapse with platform administration responsibilities across our cloud data estate. The role holder will design, build, and operate scalable data pipelines while also maintaining the underlying Azure platform — including infrastructure-as-code (Pulumi), CI/CD automation, monitoring, security, and Databricks workspace administration. The ideal candidate combines strong Python/PySpark engineering skills with deep Azure platform knowledge and a service-excellence mindset, ensuring the Allianz Data Technology platform remains secure, reliable, and aligned to Allianz standards.
Duties and Responsibilities
- Lead solution design activities, collaborating with peers and mentoring junior colleagues to define and execute the team backlog.
- Develop, test, and document scalable ETL/ELT data pipelines and workflows using Databricks and Azure Synapse to ingest and transform data from a variety of sources.
- Administer and maintain Azure data platform components including Synapse, Databricks, ADLS Gen2, Key Vault, networking (VNets, NSGs, Managed Private Endpoints) and access control (RBAC, ACLs).
- Manage infrastructure-as-code across Dev, Staging, and Production environments using Pulumi (and equivalents such as Terraform / Bicep).
- Design and operate CI/CD pipelines using GitHub Actions (with OIDC federation) and/or Azure DevOps, supporting trunk-based development practices.
- Administer Databricks workspaces — cluster policies, Secret Scopes, Repos/Git integration, Workflow job health, and Unity Catalog governance.
- Monitor platform and pipeline health using Azure Monitor, Log Analytics, KQL, and Azure Dashboards; triage and resolve incidents.
- Implement robust data security and ensure compliance with data privacy regulations; manage service principals, Managed Identities, and least-privilege access.
- Carry out routine platform operations: patching, backups, storage lifecycle, tagging, access reviews, DR readiness, and runbook execution.
- Identify and address performance bottlenecks and data quality issues to ensure data accuracy and reliability.
- Work with testers to ensure automated test plans are in place and agree test packs for UAT; review peers' work and take accountability for the quality of squad deliverables.
- Collaborate with stakeholders and analysts to understand data requirements and deliver clean, reliable, accessible data.
- Ensure solution designs align with the Allianz Data Technology Strategy, maintaining and enhancing Allianz standards and Service Excellence.
- Maintain technical documentation to Allianz standards (e.g., Grimlock) and stay current with industry trends, driving continuous improvement and innovation.
Qualification, Experience, Technical and Functional Skills
- Bachelor's or Master's degree in Computer Science, Information Systems, or a related field, with 6–10 years of relevant experience in data engineering and Azure platform administration.
Must Have
- Databricks: hands-on experience building and optimizing pipelines, managing Delta Lake, and administering workspaces (cluster policies, Unity Catalog, Secret Scopes, Workflows).
- Python / PySpark: strong programming skills for data processing, automation, and scripting.
- Azure data stack: Synapse, Databricks, ADLS Gen2, Key Vault — including Linked Services, Managed Identity, and Spark Pool configuration.
- Azure platform fundamentals: compute, storage, networking (VNets, NSGs, Private Endpoints), identity and RBAC.
- CI/CD: GitHub Actions (with OIDC federation) and/or Azure DevOps for data and platform deployments.
- Infrastructure-as-Code: Pulumi (or Terraform / Bicep) across multiple environments.
- Scripting: PowerShell and Bash for platform automation.
- Monitoring & observability: Azure Monitor, Log Analytics, KQL.
- Big data file formats: Parquet and Delta Lake.
- Cloud-native data modelling and ETL/ELT frameworks on Azure.
Good to Have
- Azure certifications: AZ-104, AZ-400, DP-203.
- Azure Data Factory (ADF), Microsoft Purview, Microsoft Fabric.
- Data governance: lineage, cataloguing, sensitivity labels.
- Event-driven architectures: Kafka / Azure Event Hubs.
- Delta Live Tables, MLflow.
- IDMC Secure Agent, Power Automate flows.
- Security baselines: CIS / NIST.
- Observability tooling: OpenTelemetry, Datadog.
- AI tools and their application in data engineering.
- Knowledge of the UK Insurance Market.
- Agile / Scrum delivery experience.
Senior Data Platform Engineer in Northampton employer: N Consulting Global
At Allianz, we pride ourselves on being an exceptional employer, offering a dynamic work culture that fosters collaboration and innovation. Our Senior Data Platform Engineer role provides the unique opportunity to work mostly remotely while engaging in meaningful projects that drive data excellence. With a strong focus on employee growth, we support continuous learning and development, ensuring our team members thrive in their careers while enjoying the benefits of a flexible work environment and a commitment to service excellence.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Data Platform Engineer in Northampton
✨Tip Number 1
Network like a pro! Reach out to your connections on LinkedIn or at industry events. A friendly chat can lead to opportunities that aren’t even advertised yet.
✨Tip Number 2
Show off your skills in real-time! Consider participating in hackathons or coding challenges related to data engineering. It’s a great way to demonstrate your expertise and meet potential employers.
✨Tip Number 3
Don’t just apply, engage! When you find a role that excites you, reach out to the hiring manager or team members. A quick message expressing your enthusiasm can make you stand out.
✨Tip Number 4
Keep learning and sharing! Join online forums or communities focused on Azure and data engineering. Sharing your knowledge not only helps others but also showcases your passion and expertise.
We think you need these skills to ace Senior Data Platform Engineer in Northampton
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Senior Data Platform Engineer role. Highlight your experience with Databricks, Azure Synapse, and any relevant projects that showcase your skills in Python/PySpark and infrastructure-as-code.
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you're the perfect fit for our team. Mention specific experiences that align with the job description and show us your passion for data engineering.
Showcase Your Problem-Solving Skills:In your application, don’t just list your skills—give us examples of how you've tackled challenges in previous roles. We love seeing how you approach problem-solving, especially in data pipeline management and platform administration.
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way to ensure your application gets into the right hands. Plus, it shows us you’re keen on joining the StudySmarter family!
How to prepare for a job interview at N Consulting Global
✨Know Your Tech Inside Out
Make sure you brush up on your Databricks and Azure Synapse skills. Be ready to discuss how you've built and optimised data pipelines in the past, and have examples of your work with infrastructure-as-code using Pulumi or Terraform. This role is all about hands-on experience, so show them you know your stuff!
✨Showcase Your Problem-Solving Skills
Prepare to talk about specific challenges you've faced in data engineering and how you tackled them. Whether it’s addressing performance bottlenecks or ensuring data quality, having a few solid examples will demonstrate your analytical thinking and service-excellence mindset.
✨Collaboration is Key
This role involves mentoring junior colleagues and collaborating with stakeholders. Think of instances where you've led solution design activities or worked closely with others to meet data requirements. Highlighting your teamwork skills will show that you're not just a tech whiz but also a great colleague.
✨Stay Current with Industry Trends
Familiarise yourself with the latest trends in data engineering and Azure technologies. Being able to discuss recent developments or innovations in the field will impress your interviewers and show that you're committed to continuous improvement and innovation.
