Azure Data Engineer - £550 - £590 Inside IR35

Key Responsibilities

My client are expanding their data and technology capabilities and seeking a skilled Data Engineer to join this growing team. The successful candidate will help ingest, cleanse, and model data into unified, analytics-ready datasets supporting Business Intelligence, machine learning, and Data Science initiatives.

Core Duties

• Design and develop cohesive data repositories to support data-driven decision-making.
• Ingest data from a variety of sources, including on-premise SQL databases, REST APIs, and Apache Kafka streams.
• Apply data cleansing rules to ensure data meets quality and consistency standards.
• Model data appropriately for usage scenarios, including designing single-source-of-truth Kimball-style datasets (dimensions and facts).
• Follow DevOps best practices for software development: write clean, testable code with proper linting, unit/integration testing, CI/CD pipelines, and peer reviews.
• Monitor and support data pipelines to ensure timely and accurate data delivery, including real-time incident resolution as needed.
• Actively contribute to team processes such as backlog grooming, sprint planning, demonstrations, and retrospectives.
• Translate business requirements into technical specifications, estimate complexity, and deliver within sprint cycles.
• Conduct ad hoc analyses of structured and unstructured data to guide solution design.
• Maintain comprehensive documentation in the data catalog, including ownership, stewardship, data dictionaries, glossaries, lineage, and data sensitivity.
• Take ownership of assigned work items, collaborating with data owners and stewards to ensure high-quality, compliant deliverables (e.g., GDPR compliance, PII handling, data retention).
• Document solution designs in internal wikis.
• Provide support and maintenance across all data platforms, ensuring their smooth and reliable operation.

Knowledge and Experience

Must Have:

• Minimum 3 years of experience as a Data Engineer or in a related data-focused role.
• Strong hands-on experience with Azure data services, including: Azure Data Factory V2, Azure Data Lake Storage V2, Azure Databricks, Azure Function Apps & Logic Apps, Azure Stream Analytics, Azure Resource Manager tools: Terraform, Azure Portal, Azure CLI, and Azure PowerShell.
• Proficient in PySpark, Delta Lake, Unity Catalog, and Python.
• Ability to write unit and integration tests using unittest, pytest, etc.
• Solid understanding of software engineering principles, including SOLID design, dependency injection, code structuring, and testing.
• Experience with version control systems and CI/CD pipelines.
• In-depth knowledge of Kimball data modeling techniques, such as star and snowflake schemas.
• Strong SQL skills and ability to write performant queries.
• Proficient in data analysis and interpretation.
• Excellent verbal and written communication skills.
• Proven track record of delivering high-quality work under pressure, with strong attention to detail and time management.
• A genuine passion for leveraging technology to create business value.

Nice to Have:

• Experience with Azure DevOps (including Git and multi-stage YAML pipelines).
• Additional programming languages such as C# or PowerShell.
• Infrastructure as Code (IaC) experience, e.g., Terraform, ARM templates, Bicep.
• Familiarity with test-driven development (TDD) methodologies.
• Experience with streaming technologies, e.g., Azure Stream Analytics or Spark Structured Streaming.
• Power BI engineering or integration experience.
• Certified Scrum Developer (CSD) or similar Agile certification.
• Experience with data governance tools, such as Microsoft Purview.
• Exposure to machine learning and artificial intelligence use cases.

Personal Attributes

• Self-motivated and able to work independently without close supervision.
• Organised, methodical, and structured in approach.
• Comfortable navigating fast-paced, dynamic environments with evolving priorities.
• Passionate about both technology and process improvement, with a focus on delivering value.
• Clear and credible communicator, capable of engaging effectively with stakeholders at all levels.
• Strong attention to detail with a commitment to high quality and accuracy.
• Positive and collaborative team player who uplifts those around them.
• Fast learner with the ability to adapt to new technologies and industry trends.
• Excellent time management and organisational skills.