At a Glance
- Tasks: Build scalable data pipelines using Azure Databricks for high-quality reporting datasets.
- Company: Join a diverse and inclusive team at Ryan Specialty in London.
- Benefits: Competitive pay, flexible work environment, and opportunities for professional growth.
- Other info: Dynamic role with a focus on innovation and collaboration.
- Why this job: Make an impact by transforming raw data into valuable insights.
- Qualifications: 2+ years of Azure Databricks experience and strong data engineering skills.
The predicted salary is between 50000 - 65000 £ per year.
An experienced data engineer with strong Azure Databricks expertise is required for a 6 month contract to develop and deliver a scalable data pipeline that ingests files and transforms them into standardised, high-quality datasets for reporting. This is a hands-on delivery role focused on building a production-grade pipeline, suited to someone comfortable working independently, defining technical solutions, and rapidly delivering high-quality data engineering components.
Location: London - UK - Fenchurch
What will your job entail?
- Key Responsibilities
- Implement a scalable ETL pipeline in Azure Databricks, that transforms raw data into standardised schemas to support reporting and analytics, following a medallion architecture in line with internal design standards.
- Leverage an existing email ingestion tool to extract multiple format source files from a shared mailbox.
- Develop notebooks, jobs and pipelines in Databricks for ETL orchestration.
- Design and implement robust validation rules, monitor pipeline runs and troubleshoot failures, with logging and notifications to support robust data processes.
- Integrate with CI/CD flows across environments using Azure DevOps.
- Produce clear technical documentation of the solution to ensure maintainability post-delivery.
Experience
- 2+ years hands-on experience with Azure Databricks, including data validation, transformation, and optimization.
- Expertise in PySpark/Spark SQL.
- Knowledge of Unity Catalog and modern Azure Databricks features, as well as data best practices and concepts.
- Proficiency building end-to-end data pipelines with Delta Lake/Lakehouse architecture.
- Experience implementing CI/CD pipelines (Azure DevOps preferred).
- Experience supporting BI reporting layers (Power BI or similar) is nice to have.
- Insurance experience is nice to have.
Data Engineer in London employer: Ryan Specialty
At Ryan Specialty, we pride ourselves on being an exceptional employer, offering a dynamic work culture that fosters innovation and collaboration. Located in the vibrant Fenchurch area of London, our team enjoys access to a wealth of professional growth opportunities, competitive benefits, and a commitment to diversity and inclusion, ensuring every employee feels valued and empowered to contribute meaningfully to our projects.
StudySmarter Expert Advice🤫
We think this is how you could land Data Engineer in London
✨Tip Number 1
Network like a pro! Reach out to your connections in the data engineering field, especially those who work with Azure Databricks. A friendly chat can lead to insider info about job openings or even referrals.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your best projects, especially those involving ETL pipelines and Azure Databricks. This gives potential employers a taste of what you can do and sets you apart from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on common data engineering questions and scenarios. Practice explaining your thought process when building data pipelines and how you tackle challenges—this will help you shine during technical interviews.
✨Tip Number 4
Don’t forget to apply through our website! We’ve got some fantastic opportunities waiting for talented data engineers like you. Plus, it’s a great way to ensure your application gets the attention it deserves.
We think you need these skills to ace Data Engineer in London
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience with Azure Databricks and data pipelines. We want to see how your skills match the job description, so don’t be shy about showcasing your relevant projects!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you’re the perfect fit for this Data Engineer role. Share specific examples of your work with ETL processes and any challenges you've overcome.
Showcase Your Technical Skills:When filling out your application, be sure to mention your expertise in PySpark and CI/CD pipelines. We love seeing candidates who can demonstrate their technical prowess and problem-solving abilities.
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you don’t miss out on any important updates from our team!
How to prepare for a job interview at Ryan Specialty
✨Know Your Tech Inside Out
Make sure you brush up on your Azure Databricks skills, especially around ETL pipelines and PySpark. Be ready to discuss specific projects where you've implemented these technologies, as this will show your hands-on experience.
✨Showcase Your Problem-Solving Skills
Prepare to talk about challenges you've faced in previous roles, particularly around data validation and troubleshooting. Use the STAR method (Situation, Task, Action, Result) to structure your answers and highlight how you resolved issues effectively.
✨Understand the Medallion Architecture
Familiarise yourself with the medallion architecture and be prepared to explain how you've applied it in your work. This will demonstrate your understanding of data engineering best practices and your ability to deliver high-quality datasets.
✨Communicate Clearly and Confidently
Since you'll need to produce clear technical documentation, practice explaining complex concepts in simple terms. This will not only help during the interview but also show that you can communicate effectively with both technical and non-technical stakeholders.
