At a Glance
- Tasks: Transform analytics infrastructure into a modern Azure-based data platform with DevOps.
- Company: Join a forward-thinking company focused on innovative data solutions.
- Benefits: Flexible remote work, competitive salary, and opportunities for professional growth.
- Other info: Be a senior technical lead in a dynamic team with excellent career progression.
- Why this job: Shape the future of data platforms and drive impactful business decisions.
- Qualifications: 5+ years in data engineering with expertise in Azure and DevOps methodologies.
The predicted salary is between 60000 - 80000 £ per year.
We’re seeking a Senior Data Engineer to transform our analytics infrastructure from monolithic SQL/PostgreSQL systems into a modern Azure-based data platform with DevOps at its core. This permanent position focuses on building an automated, self-service architecture designed for continuous evolution.
You’ll bring 5+ years of data engineering experience with proven expertise in implementing enterprise-scale data platforms and DevOps methodologies. You’ll architect solutions using modern patterns, establish automated data pipelines with CI/CD practices, implement infrastructure-as-code, and mentor our team while building systems that adapt to emerging technologies.
We need someone who prioritises maintainable, scalable, and adaptable systems. We’re committed to Azure but flexible on specific products – we value pragmatic solutions based on actual needs.
- Structure & Expectations
- Reports to: Product organisation – you’ll work closely with product teams to align data capabilities with business objectives
- Location: London office 3 days per week, flexible remote work the rest of the time
- Team: You’ll be the senior technical lead and engineer for data platform initiatives, working with business analysts, engineers, and product stakeholders
- Timeline: Initial focus on migration (6 months), then platform evolution, DevOps maturity, and capability expansion
Core Technical Requirements
- Azure Data Platform Architecture
- Microsoft Azure data stack expertise: ADF, Delta Lake, ADLS Gen2, Synapse, Databricks (with flexibility to integrate non-Microsoft solutions)
- Experience with PostgreSQL and SQL Server, including data replication/integration patterns for migration
- Dimensional and relational data modeling expertise
- Medallion architecture implementation (bronze→silver→gold)
- Data Engineering & Pipelines
- Proficiency in Python and SQL for ELT/ETL pipelines
- Batch, streaming, and micro-batching architectures
- CDC patterns and incremental load strategies at scale
- Experience with Delta Change Data Feed or equivalent real-time sync mechanisms
- Experience building ingestion pipelines from diverse external APIs (CRM, analytics, finance, observability tools) into a unified lakehouse
- DevOps & Infrastructure
- Infrastructure as Code (Terraform preferred)
- CI/CD with Azure DevOps or GitHub Actions
- Test automation for data pipelines (unit, integration, regression)
- Azure networking and security best practices
- Container orchestration (Kubernetes/AKS) experience beneficial
- Event-driven architectures and API integration patterns
- Monitoring and alerting with Azure Monitor and Log Analytics
- Performance tuning and cost optimisation
- Git version control proficiency
- Governance, Compliance & Security
- Strong understanding of information security requirements
- Experience with Microsoft Purview or similar governance frameworks
- GDPR/data protection implementation, including the right-to-be-forgotten
- RBAC, encryption strategies, and audit logging
- Data lineage and quality enforcement
Non-Technical Requirements
- Communication & Documentation
- Strong technical writing – please provide examples
- Clear presentation skills for technical and non-technical audiences
- Experience creating architecture diagrams, data flows, and runbooks
- Leadership
- Track record of training and upskilling teams
- Experience with structured project planning: from high-level initiatives through quarterly planning to executable tasks
- Comfort working with product managers and translating between technical and business contexts
What You’ll Do
- Architect a modern data platform with DevOps principles for long-term business evolution
- Establish automated engineering best practices for data, governance, and operations
- Build robust batch and real-time processing pipelines (integrating web application data with analytics sources and external APIs)
- Create comprehensive documentation and training materials
- Translate business requirements into maintainable technical solutions
- Mentor team members and drive continuous improvement
Why This Matters
You'll establish the foundation for our data platform's next decade. This platform will power our next generation of internal and customer products while empowering our business with real-time insights for faster, smarter decisions.
Application Requirements
Please provide:
- Technical documentation samples
- A description of a challenging data platform project you’ve delivered
- Your approach to implementing GDPR compliance in a lakehouse environment
Senior Data Engineer: Data Platform DevOps & Architecture in London employer: Edgefolio
Join a forward-thinking company in London that prioritises innovation and employee growth, offering a dynamic work culture where your expertise as a Senior Data Engineer will directly influence the evolution of our data platform. With a commitment to flexible remote work and a collaborative environment, you'll have the opportunity to mentor others while working on cutting-edge technologies in Azure, ensuring your contributions lead to meaningful advancements in our analytics infrastructure. We value continuous learning and provide ample opportunities for professional development, making this an excellent place for those seeking a rewarding career in data engineering.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Data Engineer: Data Platform DevOps & Architecture in London
✨Get Involved in Data Science Meetups
Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like Edgefolio!
✨Show Off Your Projects
Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Senior Data Engineer: Data Platform DevOps & Architecture at Edgefolio.
✨Leverage Professional Networks
Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like Edgefolio.
✨Apply Directly through Our Website
When you find a suitable opening like Senior Data Engineer: Data Platform DevOps & Architecture at Edgefolio, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!
We think you need these skills to ace Senior Data Engineer: Data Platform DevOps & Architecture in London
Some tips for your application 🫡
Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!
Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!
Craft a Tailored Cover Letter:For a full-time role at Edgefolio, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.
Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at Edgefolio. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!
How to prepare for a job interview at Edgefolio
✨Brush Up on Your Statistics
For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!
✨Showcase Your Projects
Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!
✨Get Comfortable with Python and R
Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at Edgefolio!
✨Prepare for Case Studies
Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.
