At a Glance
- Tasks: Design and build data engineering solutions using Azure Databricks for large-scale datasets.
- Company: Join a leading financial services team in London with a hybrid working model.
- Benefits: Competitive rate of £500 per day, flexible work schedule, and collaborative environment.
- Other info: Opportunity to grow in a dynamic Agile environment with a focus on innovation.
- Why this job: Make an impact by optimising data pipelines and working with cutting-edge Azure technologies.
- Qualifications: 6+ years in data software engineering with expertise in Azure Databricks and Scala.
The predicted salary is between 70000 - 90000 £ per year.
Location: London, UK
Rate: £500 per day, inside IR35
Working Mode: 3 days a week on site
Description
We're looking for a Data Software Engineer - Azure Databricks to join our client (Financial Services) team in London, UK in a hybrid working mode. In this role, you will design and build data engineering solutions leveraging Azure Databricks and related Azure services, enabling efficient processing and transformation of large-scale datasets. You will collaborate with cross-functional teams to ensure the delivery of optimized, reliable, and scalable data pipelines aligned with business requirements.
Responsibilities
- Develop robust, scalable data engineering solutions using Azure Databricks and Azure Data Factory
- Design and optimize data pipelines for processing large volumes of data (multi-million records) with high performance
- Participate in customer discussions, requirement analysis, and technical solution design
- Ensure code quality, performance tuning, and adherence to best practices throughout development
- Communicate project status effectively and elevate critical issues to stakeholders and management
- Actively collaborate with team members to deliver data-driven solutions in an Agile environment
Requirements (Must Have Skills)
- 6+ years of experience in data software engineering with a strong focus on Azure technologies
- Expertise in Azure Databricks development, including efficient handling of large datasets
- Strong working knowledge of Azure Data Factory for building complex ETL workflows
- Experience with performance optimization of data pipelines and distributed processing setups
- Proficiency in Scala development (hands-on experience highly desirable)
- Solid understanding of Azure services relevant to data engineering and cloud architecture
- Strong problem-solving ability with effective communication skills for team collaboration
Nice to Have Skills
- Experience working in Agile teams, including Scrum or Kanban methodologies
- Familiarity with CI/CD practices and tools for data engineering workflows
- Databricks certification or equivalent credentials
- Understanding of data security, compliance, and governance best practices in cloud environments
Data Engineer (Databricks) employer: Ports North
Ports North is an excellent employer, offering a dynamic remote work environment that fosters innovation and collaboration in the field of enterprise analytics. With a strong commitment to employee growth, we provide opportunities for professional development and skill enhancement, particularly in cutting-edge SAP technologies. Our inclusive work culture values diverse perspectives, ensuring that every team member can contribute meaningfully to our major SAP S/4HANA transformation projects.
