Senior Data Engineer

Senior Data Engineer Initial 3-month contract £(Apply online only) per day, Outside IR35 Start date: 29th of January 2025 Remote – 1 day per month in Bristol Contractor must be based in the UK (ideally South West or M4 Corridor) Unify are proud to be exclusive representing a cutting edge Startup who are experiencing rapid growth. As the team scales and matures the business is looking to engage the services of a Senior Data Engineer on a contract basis for initially 3 months, to help them build a robust, futureproof data warehouse system to enhance and optimize their Data capabilities. About the Team: They are a fast growing team with 8 engineers and the foundations of a data science team. They are transitioning towards a data-driven culture, focusing on enhancing access to their data and enabling reporting capabilities without extensive engineering involvement. Current Data Infrastructure: – Data Flow: Data is collected primarily through API, but also via client side analytics. – Data Stores: – Primary Store: MongoDB, storing the majority of their data. – Reporting Tool: ElasticSearch, used increasingly for serving reports and analytics through frontend applications. Challenges: – Latency issues with large data volumes. – Difficulty generating ad-hoc reports without engineering support. Objectives: – Unlock access to existing data to allow their data scientist to generate reports independently. – Optimise current data systems and usage. – Support the business in harnessing AI capabilities for data analysis. – Create a robust and efficient data warehouse to streamline reporting processes and accommodate future growth. Scope of responsibilities: 1. System Review: Assess the current data architecture and reporting capabilities. 2. Stakeholder Collaboration: Work closely with the VP of Engineering, senior engineers, tech leads, and data scientists to gather requirements and understand business needs. 3. Design and Proposal: Produce a proposal outlining the architecture and design of the new data warehouse, considering factors such as scalability, performance, and cost. Include recommendations for optimisations of current systems. Present this proposal no later than the end of Month 1. 4. Implementation: Develop and implement the data warehouse using appropriate technology (such as AWS, BigQuery, Snowflake, etc.). Ensure that the architecture is well-documented and meets the identified requirements. 5. Handover: Assist in onboarding a newly recruited mid-level data engineer and provide comprehensive documentation for them to understand the system. Deliverables: – A documented proposal for the data warehouse architecture (Month 1). – Recommendations for improvements and optimisations to current systems (Month 1). – Implementation of the data warehouse, ensuring it meets performance and reporting needs (Month 2). – Comprehensive documentation and training for the new data engineer (Month 3). – Final rollout and verification of the implemented system. Technical Skills, Qualifications and background: – Proven experience designing and implementing data warehouses. – Proficiency in MongoDB and ElasticSearch; familiarity with alternative data storage solutions (e.g., BigQuery, Snowflake etc). – Strong understanding of data modeling and ETL processes. – Ability to conduct performance analyses and troubleshoot latency issues. – Excellent communication skills for collaborating with technical and non-technical stakeholders. – Ideally experience in a similar role in a Startup environment and/or Fintech industry experience. Project Methodology: – The core Engineering team follows a form of the ShapeUp process, though the contractor will not be required to adhere strictly to this framework. – Expected to be visible and present progress updates to the team throughout the contract duration. Virtual interviews will be held on the 22nd of January. The preferred candidate will be able to start the assignment on the 29th of January. Please reach out to our Talent Manager, Mark Brereton for more information. Thanks