Lead Data Engineer (Snowflake) in London

Lead Data Engineer (Snowflake) in London

London Full-Time 72000 - 88000 £ / year (est.) Hybrid
F

At a Glance

  • Tasks: Lead the development of a cutting-edge data platform using Snowflake.
  • Company: Join a dynamic banking transformation programme in London.
  • Benefits: Flexible hybrid work, competitive salary, and opportunities for professional growth.
  • Other info: Collaborative environment with a focus on mentorship and career advancement.
  • Why this job: Shape the future of banking with innovative data solutions and lead a talented team.
  • Qualifications: 10+ years in data engineering with strong Snowflake experience.

The predicted salary is between 72000 - 88000 £ per year.

Job Location: London, UK/ Hybrid- 4 days onsite

Job Type: Open for Contract (6 months)/FTC (6 months)/Perm

About the Role

We are looking for a Senior / Lead Data Engineer to join the engineering organisation a multi-year banking transformation programme delivering capabilities across Accounts (Real-Time Ledger), Payments, Foreign Exchange and Client Lifecycle Management, underpinned by an Enterprise Data Platform built on Snowflake. This is a hands-on build role. Working closely with the Enterprise Data Architect, you will turn the target-state data architecture into a production platform: standing up the platform foundations, engineering data acquisition and transformation, and implementing canonical data models and data products. You will act as the engineering counterpart to the Data Architect implementing the design, feeding back on buildability, and setting the engineering standards others follow. At Lead level, you will also shape and lead the data engineering team.

Key Responsibilities

  • Platform foundations
    • Build Snowflake environment structures layered databases and schemas (Raw / Conformed / Consumption), environment separation (development / test / production) and naming conventions
    • Implement role-based access control, service accounts, virtual warehouse design, resource monitors and cost guardrails
    • Manage the platform as code Snowflake objects, roles, grants and warehouses through infrastructure-as-code and version control
    • Establish Git workflows and CI/CD pipelines for data code, with automated build, test and deployment across environments
  • Data acquisition and CDC
    • Assess source systems and recommend fit-for-purpose acquisition patterns full vs incremental, batch vs micro-batch vs event, and file vs API vs direct database vs message hub
    • Advise on where change data capture should be applied and owned log-based CDC at source, application-emitted change events, or warehouse-side change detection and implement the chosen patterns
    • Engineer CDC and replication from relational, document and time-series application databases, handling initial loads, deletes, schema evolution, lag and backfills
    • Ingest event streams from messaging platforms into Snowflake, covering ordering, idempotency, deduplication, replay and error handling
    • Configure and optimise managed ELT connectors (e.g. Fivetran), and build file- and API-based ingestion where required
    • Design and build a metadata-driven ingestion framework sources, objects, load types (full, incremental, CDC), schedules and target mappings held as configuration, with control tables for watermarks, run status, audit and reprocessing so new feeds are onboarded through metadata rather than new code
    • Make the framework reusable across ingestion channels (managed ELT, file, API, database CDC and event feeds), with generic, parameterised loading, schema-drift handling and standard error handling
    • Prove patterns through technical spikes, and record decisions as Architecture Decision Records
  • Transformation and modelling
    • Design and build dbt projects from the ground up structure, layering, conventions, macros, packages, tests and documentation
    • Run dbt both natively in Snowflake (dbt Projects on Snowflake) and on dbt Cloud, building portable, convention-led projects that move between the two with minimal rework
    • Implement canonical data models including industry-aligned models such as BIAN-based business objects from source-to-canonical mapping specifications and governed reference data
    • Build dimensional and domain-oriented data products for reporting, regulatory, analytics and AI consumers
    • Engineer identity resolution, key management and cross-source reconciliation, and feed implementation findings back into the models and mappings
  • Quality, governance and operations
    • Embed data quality as code tests, freshness checks, reconciliations and data contract enforcement
    • Implement Snowflake governance controls tagging and classification, dynamic data masking, row access policies and PII handling aligned to GDPR with lineage that supports BCBS 239
    • Build monitoring and alerting for pipeline failures, freshness, data quality and cost, and define run-books and support processes
  • Orchestration and scheduling
    • Design and implement orchestration and scheduling natively in Snowflake using Tasks and task graphs (DAGs) dependencies, root and finaliser tasks, serverless vs warehouse-backed compute, schedules and event-triggered runs
    • Orchestrate ingestion, dbt execution (e.g. EXECUTE DBT PROJECT), data quality checks and downstream refreshes end to end within Snowflake, with retries, failure handling, alerting and notifications
    • Use Dynamic Tables where declarative, target-lag-driven pipelines are a better fit than task-based scheduling, choosing between incremental and full refresh modes and managing their cost
    • Build operational visibility over task and Dynamic Table runs run history, latency, failures and credit consumption and keep orchestration designs portable to external orchestrators where needed
  • AI-assisted engineering
    • Use Snowflake CoCo (formerly Cortex Code) to accelerate development, with generated code reviewed, tested and deployed through standard CI/CD
    • Apply Snowflake Cortex AI capabilities AISQL functions, Cortex Search, semantic views, Cortex Analyst and Cortex Agents where they add value, and design data products to be AI-ready
  • Leadership and collaboration
    • Work as the engineering counterpart to the Enterprise Data Architect, contributing to design decisions and ADRs
    • Collaborate with Integration, Solution and Application teams on source extracts, event contracts and data contracts
    • Define and own engineering standards, templates and reusable patterns
    • Lead level: lead, mentor and help recruit data engineers, own code review, and plan and report delivery

Primary Skills (Essential)

Candidates are expected to have strong, hands‑on depth in all of the following:

  • SQL: Advanced, production-grade SQL complex joins, window functions, set-based transformation, MERGE / upsert patterns, query profiling and performance tuning
  • Data Warehousing: Solid grounding in warehouse principles layered / medallion architectures, ELT, incremental processing, slowly changing dimensions, history and audit, reconciliation, partitioning and performance
  • Data Modelling: Dimensional modelling (Kimball facts, dimensions, conformed dimensions); implementing canonical / enterprise data models; normalised vs denormalised trade-offs; SCD Types 1/2; working from logical models and mapping specifications
  • Snowflake: Hands‑on platform engineering schema and layer design, RBAC, virtual warehouses, Snowpipe, Streams & Tasks, zero-copy cloning, Time Travel, masking and row access policies, resource monitors, cost optimisation
  • dbt: Building dbt projects from scratch layering, incremental models, snapshots, macros, packages, tests, docs; dbt Projects on Snowflake and dbt Cloud
  • Data Acquisition & CDC: Designing full, incremental and CDC-based acquisition; log-based CDC tooling (e.g. Fivetran / HVR, Debezium, Kafka Connect, Snowflake Openflow); batch, micro-batch and event-driven ingestion
  • Managed ELT: Fivetran or comparable connector configuration, schema drift, history mode, re-syncs, consumption cost management
  • Metadata-driven Ingestion: Designing and building configuration-led ingestion frameworks metadata / control tables, parameterised and reusable load patterns, watermarking, audit, restartability and reprocessing
  • Snowflake Orchestration: Snowflake Tasks and task graphs (DAGs), Streams & Tasks patterns, Dynamic Tables (target lag, incremental vs full refresh), serverless tasks, alerts and notifications, monitoring of task history and cost
  • Engineering Practice: Python for data engineering and automation; Git; CI/CD (e.g. GitHub Actions,); infrastructure-as-code (e.g. Terraform, schemachange); automated testing
  • Snowflake AI: Good working knowledge of Snowflake CoCo (formerly Cortex Code) and Snowflake Cortex AI, and a disciplined approach to AI-assisted development in a regulated environment

Secondary Skills (Desirable)

Experience in any of the following will strengthen an application:

  • Cloud (AWS): S3, IAM, KMS, PrivateLink and networking basics, secrets management
  • Application Databases: Sourcing safely from PostgreSQL, Oracle, MongoDB, TimescaleDB and Redis logical replication, redo logs, change streams, impact on source performance
  • Event Streaming: NATS JetStream or Kafka; Snowpipe Streaming; event schema contracts and replay strategies
  • Advanced Snowflake: Snowpark (Python), Snowflake Horizon catalog, data sharing
  • Modelling Standards: BIAN, ISO 20022 or other industry reference models; awareness
  • Operations: Data observability tooling, FinOps for Snowflake and ELT consumption, external orchestration tools (e.g. Airflow, Dagster)
  • Banking Domain: Core banking / ledger, payments, FX, client onboarding / KYC or regulatory reporting data; GDPR and BCBS 239
  • Certifications: SnowPro Core or SnowPro Advanced: Data Engineer; dbt Analytics Engineering; AWS certification

Experience & Profile

Senior: 10+ years in data engineering, including at least 3 years of hands-on Snowflake delivery in production. Lead: 12+ years, including leading a data engineering team or workstream

Strong track record in data warehousing and data modelling, with experience implementing models designed by architects as well as designing their own

Has stood up a new data platform or major platform workstream environments, access control, automation rather than only extending an existing one

Has built a dbt project from the ground up, including standards, testing and CI/CD

Has designed and built a metadata-driven ingestion framework that has been used to onboard multiple sources in production

Strong hands-hand experience scheduling and orchestrating production pipelines natively in Snowflake with Tasks / task graphs and Dynamic Tables

Has designed and implemented data acquisition across heterogeneous sources, and can clearly explain the trade-offs between full, incremental and CDC approaches

Comfortable reading and implementing architecture artefacts canonical models, mapping specifications, ADRs and offering well‑reasoned implementation feedback

Clear communicator, able to work with architects, application teams and business stakeholders

Senior vs Lead Role Distinction

  • Focus: Hands-on build of pipelines, models and platform components
  • Standards: Applies and contributes to engineering standards
  • People: Peer reviews and supports other engineers
  • Planning: Estimates and delivers own work
  • Architecture: Implements the design; raises issues

Lead Data Engineer (Snowflake) in London employer: FalconSmartIT

FalconSmartIT is an exceptional employer that fosters a collaborative and innovative work culture in the heart of London. With a strong focus on employee growth, we offer extensive training and development opportunities, ensuring our team members are equipped to excel in their roles. Join us to be part of a dynamic environment where your contributions directly impact transformative projects, making your work both meaningful and rewarding.

F

Contact Details:

FalconSmartIT Recruitment Team

StudySmarter Expert Advice🤫

We think this is how you could land Lead Data Engineer (Snowflake) in London

✨Get Involved in Data Science Meetups

Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like FalconSmartIT!

✨Show Off Your Projects

Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Lead Data Engineer (Snowflake) at FalconSmartIT.

✨Leverage Professional Networks

Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like FalconSmartIT.

✨Apply Directly through Our Website

When you find a suitable opening like Lead Data Engineer (Snowflake) at FalconSmartIT, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!

We think you need these skills to ace Lead Data Engineer (Snowflake) in London

Python
Communication Skills
Data Engineering
Data Pipeline Development
Data Governance
Data Quality Assurance
SQL

Some tips for your application 🫡

Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!

Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!

Craft a Tailored Cover Letter:For a full-time role at FalconSmartIT, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.

Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at FalconSmartIT. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!

How to prepare for a job interview at FalconSmartIT

✨Brush Up on Your Statistics

For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!

✨Showcase Your Projects

Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!

✨Get Comfortable with Python and R

Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at FalconSmartIT!

✨Prepare for Case Studies

Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.