Azure Data Engineer - £550 - £590 Inside IR35

Key Responsibilities

My client are expanding their data and technology capabilities and seeking a skilled Data Engineer to join this growing team. The successful candidate will help ingest, cleanse, and model data into unified, analytics-ready datasets supporting Business Intelligence, machine learning, and Data Science initiatives.

Core Duties

• Design and develop cohesive data repositories to support data-driven decision-making.
• Ingest data from a variety of sources, including on-premise SQL databases, REST APIs, and Apache Kafka streams.
• Apply data cleansing rules to ensure data meets quality and consistency standards.
• Model data appropriately for usage scenarios, including designing single-source-of-truth Kimball-style datasets (dimensions and facts).
• Follow DevOps best practices for software development: write clean, testable code with proper linting, unit/integration testing, CI/CD pipelines, and peer reviews.
• Monitor and support data pipelines to ensure timely and accurate data delivery, including real-time incident resolution as needed.
• Actively contribute to team processes such as backlog grooming, sprint planning, demonstrations, and retrospectives.
• Translate business requirements into technical specifications, estimate complexity, and deliver within sprint cycles.
• Conduct ad hoc analyses of structured and unstructured data to guide solution design.
• Maintain comprehensive documentation in the data catalog, including ownership, stewardship, data dictionaries, glossaries, lineage, and data sensitivity.
• Take ownership of assigned work items, collaborating with data owners and stewards to ensure high-quality, compliant deliverables (e.g., GDPR compliance, PII handling, data retention).
• Document solution designs in internal wikis.
• Provide support and maintenance across all data platforms, ensuring their smooth and reliable operation.

Knowledge and Experience

Must Have:

• Minimum 3 years of experience as a Data Engineer or in a related data-focused role
• Strong hands-on experience with Azure data services, including:
• Azure Data Factory V2
• Azure Data Lake Storage V2
• Azure Databricks
• Azure Function Apps & Logic Apps
• Azure Stream Analytics
• Azure Resource Manager tools: Terraform, Azure Portal, Azure CLI, and Azure PowerShell
• Proficient in PySpark, Delta Lake, Unity Catalog, and Python
• Ability to write unit and integration tests using unittest, pytest, etc.
• Solid understanding of software engineering principles, including SOLID design, dependency injection, code structuring, and testing
• Experience with version control systems and CI/CD pipelines
• In-depth knowledge of Kimball data modeling techniques, such as star and snowflake schemas
• Strong SQL skills and ability to write performant queries
• Proficient in data analysis and interpretation
• Excellent verbal and written communication skills
• Proven track record of delivering high-quality work under pressure, with strong attention to detail and time management
• A genuine passion for leveraging technology to create business value

Nice to Have:

• Experience with Azure DevOps (including Git and multi-stage YAML pipelines)
• Additional programming languages such as C# or PowerShell
• Infrastructure as Code (IaC) experience, e.g., Terraform, ARM templates, Bicep
• Familiarity with test-driven development (TDD) methodologies
• Experience with streaming technologies, e.g., Azure Stream Analytics or Spark Structured Streaming
• Power BI engineering or integration experience
• Certified Scrum Developer (CSD) or similar Agile certification
• Experience with data governance tools, such as Microsoft Purview
• Exposure to machine learning and artificial intelligence use cases

Personal Attributes

• Self-motivated and able to work independently without close supervision
• Organised, methodical, and structured in approach
• Comfortable navigating fast-paced, dynamic environments with evolving priorities
• Passionate about both technology and process improvement, with a focus on delivering value
• Clear and credible communicator, capable of engaging effectively with stakeholders at all levels
• Strong attention to detail with a commitment to high quality and accuracy
• Positive and collaborative team player who uplifts those around them
• Fast learner with the ability to adapt to new technologies and industry trends
• Excellent time management and organisational skills