Senior Data Engineer in London

Hybrid working within this role enables a balance of 50 per cent of time split between the office and home over a 4-week period. Hybrid working arrangements can evolve subject to business requirements. All offers of employment are subject to satisfactory right‑to‑work checks. Candidates must be able to demonstrate their right to work in the UK. At the present time TfL is unable to offer visa sponsorship for this role.

Overview of project/role

The post holder will have responsibility for the successful development of technical data and analytics solutions providing quality data required to support evidence based business and operational decision making. Forming part of a scrum agile team, the developer will develop to specification, unit test and implement extract/transform/load (ETL) processes using custom ETL or ETL tools.

Key Accountabilities

• To undertake the maintenance of existing and development of new data and analytics solutions to ensure the optimal production of operational dash boarding, reporting and other applications in line with the strategy set by the Chief Data Officer.
• To develop data and analytics solutions using industry best practice so that the code delivered is running, meets business requirements and definitions, conforms to Data & Analytics design specifications and has the controls and checks in place to satisfy audit/finance guidelines.
• To document the technical design of developed components in accordance with Data & Analytics standards.
• To develop to estimated timelines or update management of deviations to these estimates. To raise critical technical issues to the Development Stream Lead/Technical Delivery Manager that may impact on the timeline.
• Working with their scrum agile team members, to undertake and support unit and regression testing of the codebase to ensure it can be delivered and meets business/operational requirements and can be run in a system testing environment.
• To undertake fixes to the code, where necessary. To undertake performance tuning of where required.
• To participate and contribute fully in quality assurance design walk-throughs of developed technical components with colleagues in the Analytics Development and Delivery team.
• When faced with business scenarios which do not fit a standard approach, to collaboratively work through each issue with peers to ensure that the solution meets business/operational requirements and the principles of best practice.
• To ensure that TfL’s policy on privacy and data protection is adhered to in the design of all solutions and that access to data is controlled, personal data is stored securely and is anonymised in accordance with best practice guidelines.
• To provide, where appropriate, user access to Technology & Data information; and to give advice on information and databases, including training and familiarisation with revenue collection and other performance, operational or customer experience data.

Knowledge

• Detailed working knowledge of ETL/ELT, data warehousing/business intelligence methodologies and best practice including dealing with big data, cloud technology and unstructured data and the relative required approaches.
• Knowledge of star schema structure & design, detailed understanding of Kimball & Inmon and hybrid data warehouse design.
• Understanding and experience of Agile development approach and tools.
• A degree in a relevant technology/computer science subject/equivalent technical qualification plus sound on the job experience of the platform specialism.
• Familiar with the variety of SDLC concepts and approaches.
• Knowledge and interest in the latest and emerging technologies likely to be employed in designing and delivering Enterprise Wide quality solutions in the chosen platform.
• Knowledge of best practice data encryption techniques and standards.
• Knowledge of relevant national and international legislation pertaining to the Government/Public Sector IT, e.g. Data Protection.

Skills

• Extensive use of and fully conversant with on premise, cloud based or hybrid Oracle and/or Microsoft database technologies in particular ETL/ELT design, development and testing in Microsoft SQL Server 2008 onwards, TSQL, APS, Azure, SSIS and/OR Oracle 11g/RAC SQL, PL/SQL, ODI and OBIEE.
• Expert at writing/reading complex queries, performance tuning & monitoring - Skilled at solving and communicating problems collaboratively, applying knowledge and technology to the full.
• Extensive use of and fully conversant with cloud database tools such as Azure Databricks and Azure Data Factory.
• Strong analytical skills.
• Communicates all needs, instructions and decisions clearly. Adapts the style of communication to meet the needs of the audience. Checks for understanding.
• Proficient at use of standard computer packages.
• Excellent writing style including translating technical information into ‘plain English’.

Experience

• Has recent experience as a business intelligence developer working in a large scale data warehouse system within the data and analytics industry.
• Recent experience in the latest techniques and methodologies in delivering quality solutions.
• Experienced in large enterprise business/data warehouse implementations.
• Experience of operating in a dynamic working environment, handling multiple tasks and deadlines, delivering to time, budget, and quality.
• Detailed experience of developing and supporting queries on databases that contain Financial, Operational and/or Customer information which is used by many business groups.
• Experience with the Data Protection Act and Data Security Managers.

Equality, diversity and inclusion

We are committed to equality, diversity and inclusion. We want to represent the city we serve, which will help us become a more innovative and efficient organisation. Our goal is to make our recruitment as inclusive as possible. We are a disability confident employer who guarantee an interview to any disabled candidate who meets all of the essential criteria. We also use anonymising software that removes identifying information from CVs and cover letters to make the process fair.

Benefits

In return for your commitment and expertise, you will enjoy excellent benefits and scope to grow. Rewards vary according to the level of role but mostly include the below:

• Final salary pension scheme.
• Free travel for you on the TfL network.
• Reimbursement of 75% of the cost of a standard class Ticket for National Rail travel from home.
• 30 days annual leave plus public holidays.