Data Platform Tech Lead in Edinburgh

Our Data Platform Team's mission is to empower FreeAgent’s engineers to reliably, securely, and sustainably derive value from our data assets.

We’re looking for a Tech Lead who will play an active part in the design and implementation of our data platform. You will lead a small dedicated team of engineers, ensuring they are well directed, motivated, productive and engaged in their work. You will collaborate closely with the Data Science and Analytics Engineering teams to ensure they have the right tools for their jobs, and enable stakeholders across FreeAgent to turn messy data into valuable insights. This is a team leadership role with a focus on technical contribution.

Our past projects include:

• The design and delivery of an event system which processes over 100 million events per month
• Automating metadata management in our data lake
• Implementing mechanisms for running regular quality checks on our data

Our platform supports our machine learning based automations for document extraction, bill, expense and bank transaction categorisation. We categorise over two million bank transactions every month with accounting grade accuracy. Currently, we’re in the process of migrating to a new orchestration tool, which presents exciting opportunities to shape FreeAgent’s data tech stack and make a positive impact by bringing modern engineering practices to our data pipelines.

Day to day your role will involve:

• Helping us realise our vision for our next generation data platform to make it easy to collect, process, store and analyse customer and business data
• Leading an agile development process for your team
• Collaborating cross-functionally with other engineering teams
• Assisting in producing common development and architectural standards and ensuring they are upheld across engineering teams
• Cutting code alongside your team when required, reviewing pull requests
• Line managing your team, and onboarding, coaching and mentoring any new hires
• Keeping up to date with developments in data, ML and AI technology and techniques
• Working with senior engineering leadership to understand the big picture and the future vision of the FreeAgent stack and drive forward the development of the data platform to meet these goals

Our data platform is built in AWS and some of the specific tools we use are outlined below. However, if you have experience with equivalent tools you should still apply. Recruitment is a great chance for us to bring alternate perspectives into the team!

Ideally you’ll have the following skills and experience:

• You can demonstrate hands-on experience with a modern data pipeline stack. We’re currently migrating to Dagster, dbt and dlt and we work with structured and unstructured data from a variety of sources, as well as internal event data
• You have designed and implemented data lakehouse storage architectures using tools like S3, Redshift, Glue, and Athena together with your team
• You can employ an appropriate data modelling technique for a given problem. As well as working with semi-structured event data we rely on dimensional models for business reporting.
• You have up to date knowledge of data management and retention regulations, security and ethics

• You are comfortable following a modern software development methodology to implement and test changes to complex systems. We use Python, Ruby and Git/Github.

• Experience building and managing infrastructure as code (we use Terraform)
• Experience deploying containerised services with Docker and GitHub Actions
• Experience with machine learning platforms such as SageMaker

• A comprehensive understanding of lean and agile development methodologies
• Experience of or enthusiasm to develop people management skills
• Keeping up to date with new techniques, technologies and resources

We believe in a strong work-life balance, we don’t work long hours (35 hour working week), and we have a fabulous Edinburgh HQ expertly designed for collaboration including high-quality A/V equipment to provide a great experience for distributed teams. At FreeAgent, we support each other and operate a no-blame culture. Our mindset is: work hard, be nice to people, and the rest will sort itself out!

This is a permanent, full time role (Monday to Friday, 9am to 5pm). This role will be based in our Edinburgh office, however there will be an opportunity to adopt hybrid working, under an agreed pattern of 2 days from the office and 3 days at home each week.

• Annual starting salary of £59,000 - £72,000 per annum depending on experience, with room to progress to £80,000 within the band.
• Annual starting salary of £80,000 - £87,000 per annum depending on experience, with room to progress to £95,000 within the band.
• 33 days holiday (and an extra day for each year you’re with us)
• Flexible/hybrid working
• Private health care
• Life assurance
• 5% matched pension contributions
• Summer 4 day weeks (giving you 8 additional days off!)
• Buy-as-you-earn share schemes
• Sabbatical leave (unpaid)
• Cycle to work scheme
• Wellness webinars
• Focused learning time and learning budgets
• Volunteering day