Data Engineer- Perm

Job Title: Data Engineer (Azure, Databricks, and Microsoft Fabric)

Location: London ideally but open to Birmingham, Manchester or Newcastle

Duration: Perm

Overview:

We are looking for a highly skilled Data Engineer with expertise in Azure, Databricks, and Microsoft Fabric to design, build, and optimize data pipelines and infrastructure. This permanent role involves working with cutting-edge cloud technologies, enabling scalable and efficient data processing, analytics, and business intelligence solutions.

Key Responsibilities:

• Design, develop, and maintain end-to-end data pipelines using Azure Data Services, Databricks, and Microsoft Fabric.
• Implement Lakehouse architecture using Databricks Delta Lake and Microsoft OneLake.
• Develop and optimize ETL/ELT workflows using Azure Data Factory, Databricks, and Fabric Dataflows.
• Work with Azure Synapse Analytics and Fabric Data Warehouses for large-scale data processing.
• Ensure data quality, security, and governance across cloud platforms.
• Develop and manage real-time and batch data processing using Apache Spark and Databricks workflows.
• Collaborate with data analysts, scientists, and business teams to enable data-driven decision-making.
• Automate data pipelines using Python, SQL, and Spark for efficiency and scalability.
• Monitor and troubleshoot data performance issues, ensuring high availability.
• Stay updated with the latest advancements in Azure, Databricks, and Microsoft Fabric technologies.

Required Skills & Qualifications:

• 3+ years of experience in data engineering with Azure and Databricks.
• Expertise in Azure Data Services (Azure Data Lake, Azure Synapse, Azure SQL).
• Hands-on experience with Databricks (Delta Lake, Spark, MLflow, Notebooks, Workflows).
• Strong knowledge of Microsoft Fabric (OneLake, Data Factory, Synapse, Power BI).
• Proficiency in SQL, Python, PySpark, or Scala for data transformation and automation.
• Strong problem-solving skills and ability to work in a fast-paced, agile environment.
• Stakeholder Management

Priyanka Sharma

Senior Delivery Consultant

Office: 02033759240

Email: psharma@vallumassociates.com