At a Glance
- Tasks: Design and optimise Azure Databricks solutions for enterprise data platforms.
- Company: Global recruitment specialist with a focus on innovative tech roles.
- Benefits: Competitive pay, flexible work arrangements, and opportunities for professional growth.
- Other info: Exciting opportunity for career advancement in a collaborative environment.
- Why this job: Join a dynamic team and make a real impact in data engineering and optimisation.
- Qualifications: Experience with Azure Databricks, Python, and data engineering principles.
The predicted salary is between 51750 - 63250 £ per year.
We are a Global Recruitment specialist that provides support to the clients across EMEA, APAC, US and Canada. We have an excellent job opportunity for you.
Location: London (3 days onsite)
Pay Rate: £540 per day all inc. (PAYE through Umbrella)
Duration: Until 1/2/2027
Role Purpose
We are seeking a highly experienced, hands-on Azure Databricks Platform Engineer/Architect to enhance and optimise an enterprise Data Platform. This role combines architecture, engineering, operational support and platform optimisation. The successful candidate must be capable of designing, configuring, developing, troubleshooting and improving Azure Databricks solutions, rather than operating solely at a strategic or governance level.
The role focuses on three primary outcomes:
- Enabling and optimising Databricks Serverless capabilities.
- Strengthening FinOps practices, cost controls and platform governance.
- Enhancing the Databricks Discovery Zone to support migration and consolidation of workloads currently delivered through POSIT/RStudio.
Key Responsibilities
Databricks Serverless Enablement and Optimisation
- Assess existing workloads and determine the most appropriate compute model, including Serverless, Jobs Compute, Interactive Compute or Classic Clusters, based on workload characteristics, SLA requirements, performance, utilisation and cost.
- Configure and implement Serverless capabilities for notebooks, jobs, SQL workloads, analytical processing and data pipelines.
- Develop workload placement standards and guidance, identifying scenarios where Serverless may not be the most cost-effective solution for predictable, long-running or continuously utilised workloads.
- Implement compute policies, autoscaling configurations, quotas, budget controls and operational guardrails.
- Monitor performance and cost, identifying oversized, underutilised or idle resources and recommending optimisation opportunities.
FinOps and Enterprise Platform Controls
- Define and implement a practical FinOps operating model covering ownership, accountability, environments, projects, applications, cost centres and teams.
- Establish mandatory tagging standards and integrate automated validation into CI/CD pipelines to prevent deployment of non-compliant resources.
- Deliver granular cost attribution and reporting across workspaces, projects, applications, workloads and business teams.
- Configure budgets, spend thresholds, alerts and usage monitoring to proactively manage platform costs.
- Analyse platform usage and billing data to identify cost anomalies, inefficient workloads, excessive storage consumption, unnecessary data movement and underutilised resources.
- Support continuous improvement through cost optimisation recommendations and governance controls.
Discovery Zone Enhancement and POSIT/RStudio Migration
- Enhance the Databricks Discovery Zone to support the migration and modernisation of analytics and data science workloads currently running on POSIT/RStudio.
- Enable capabilities including:
- Secure API and external system integrations
- Application deployment and operationalisation
- External data ingestion
- BI and reporting connectivity
- Scheduling and orchestration
- Local IDE-based development
- LLM and AI integration
- Operational monitoring and reporting
- Define reusable onboarding, migration and delivery patterns that reduce technology sprawl while improving platform security, supportability and delivery speed.
- Collaborate with stakeholders to support migration planning, solution design and adoption.
Data Engineering and Integration
- Design, build and optimise scalable data ingestion and transformation solutions using Python, PySpark, SQL and Delta Lake.
- Implement batch and incremental processing patterns, including Change Data Capture (CDC), schema evolution, reconciliation processes, data quality controls and error handling.
Develop reusable integration frameworks for:
- REST APIs
- SaaS platforms
- Databases
- Files and object storage
- Document repositories
- Enterprise systems
- External and public data sources
Implement secure authentication, secrets management and credential handling practices. Deliver end-to-end data flows from source ingestion through governed and curated data layers, supporting analytics, BI, machine learning and application consumption.
Required Technical Skills
Azure & Databricks Platform
- Deep hands-on Azure Databricks implementation, administration and troubleshooting experience.
- Databricks Serverless architecture, workload placement and compute optimisation.
- Azure networking, identity, security, monitoring, secrets management and private connectivity.
- Databricks SQL, Delta Lake and query/workload performance optimisation.
- Strong understanding of platform governance, operational support and best practices.
Engineering, Delivery & Operations
- Python, PySpark and SQL development expertise.
- Data engineering and pipeline delivery within enterprise-scale environments.
- Jobs, workflows, orchestration, incremental processing and CDC implementation.
- Data quality, reconciliation and operational monitoring.
- REST API integration and external data connectivity patterns.
- FinOps implementation, tagging strategies, cost attribution, monitoring and budget management.
- Experience supporting production platforms and driving continual optimisation.
Experience and Candidate Profile
- Proven experience delivering and supporting enterprise-scale Azure Databricks platforms in production environments.
- Demonstrated ability to work across architecture, engineering, implementation, optimisation and operational support without dependence on specialist teams.
- Strong understanding of security, governance, compliance and controlled delivery within complex or regulated organisations.
- Experience collaborating with data engineers, data scientists, architects, security teams, platform teams and business stakeholders.
- Excellent communication skills with the ability to document standards, patterns, operational procedures and architectural decisions.
Highly Desirable
- Experience migrating workloads from POSIT/RStudio to Databricks.
- Migration of analytical and data science solutions, including conversion of R-based workloads and libraries.
- AI/ML experience, including LLM integration, RAG/vector retrieval, model serving and model life cycle management.
- Experience delivering large-scale platform transformation programmes.
- Strong Azure and Databricks FinOps experience, including cost optimisation, governance and chargeback/showback models.
- Experience operating within highly regulated or complex enterprise environments.
If you are interested in this position and would like to learn more, please send through your CV and we will get in touch with you as soon as possible. Please note, candidates are often Shortlisted within 48 hours.
Platform Architect in London employer: eTeam Workforce Limited
eTeam Workforce Limited is an exceptional employer, offering a dynamic work environment in North Wales that prioritises engineering excellence and compliance within military systems. With a strong focus on employee growth, the company provides ample opportunities for professional development and leadership advancement, fostering a culture of collaboration and innovation. Employees benefit from a supportive atmosphere that values technical integrity and airworthiness, making it a rewarding place to build a meaningful career.
StudySmarter Expert Advice🤫
We think this is how you could land Platform Architect in London
✨Tap into Online Data Science Communities
Join online communities focused on data science like Kaggle, LinkedIn groups, or Reddit threads. These are goldmines for temporary gigs, as you can network with professionals and potentially hear about opportunities at companies like eTeam Workforce Limited before they're even advertised!
✨Show Off Your Skills With Projects
Got some cool data science projects? Showcase them on platforms like GitHub or create a personal portfolio website. This visibility is crucial for landing temporary roles—let recruiters see your actual skills in action, which can set you apart from the crowd.
✨Check Out Specialist Job Boards
For temp roles, hit up job boards dedicated to tech and data science, like Stack Overflow Jobs or DataJobs. These platforms often feature openings that you won’t find on general job sites, including contracts with companies like eTeam Workforce Limited.
✨Leverage University Resources
If you're currently at uni or recently graduated, tap into your school's career services. They often have connections with companies looking for temporary data science interns or contract workers, and they might even host job fairs with employers like eTeam Workforce Limited.
We think you need these skills to ace Platform Architect in London
Some tips for your application 🫡
Highlight Your Data Projects:When applying for a temporary data science role at eTeam Workforce Limited, make sure to showcase any relevant projects you've worked on. Whether it's a personal project, an academic undertaking, or contributions to an open-source initiative, detailing these experiences can really set you apart and demonstrate your practical skills.
Emphasise Your Analytical Skills:In your CV and cover letter, focus on the specific analytical skills that are key to data science. Mention any experience with statistical tools, programming languages like Python or R, and data visualisation software. Don't forget to include any certifications that may bolster your expertise!
Show Your Flexibility:Since this is a temporary role, it's important to convey your adaptability and willingness to learn. In your cover letter to eTeam Workforce Limited, emphasise how quickly you can get up to speed with new tools or projects. Highlight any previous experiences where you've had to adjust to new environments or challenges.
Craft a Unique Data-Driven Cover Letter:Instead of the usual generic cover letter, spice it up with some data! Maybe you’ve improved a process by 20% in a past role or cleaned a dataset with over a million entries. Use these stats to your advantage to grab eTeam Workforce Limited’s attention and show the tangible impact of your work.
How to prepare for a job interview at eTeam Workforce Limited
✨Showcase Your Analytical Skills
For a data science gig, it's crucial to demonstrate your analytical abilities. Be ready to discuss previous projects and the methodologies you used. Think about how you can quantify your impact—did your analysis improve efficiency or save costs? These are the stories that will stick with interviewers at eTeam Workforce Limited.
✨Brush Up on Technical Skills
You might face technical questions on tools relevant to data science, like Python, R, or SQL. Prepare to solve a problem live—perhaps they'll ask you to write a simple query or code snippet. It’s cool to talk about them, but we need to show we can do it in practice, especially in a temporary role where quick results matter.
✨Highlight Your Adaptability
Since this is a temporary position, emphasise your ability to learn quickly and adapt to new tools or workflows. Share examples of how you've thrived in fast-paced environments before, and how you can hit the ground running at eTeam Workforce Limited.
✨Prepare a Portfolio of Your Work
Bring your portfolio to the table—showcase projects where you've leveraged data science techniques to solve problems. Whether it’s a GitHub repository or a set of case studies, having tangible examples of your work will help you stand out and show what you bring to the team at eTeam Workforce Limited.
