Big Data Engineer (Hybrid or Remote)

At Allstate, great things happen when our people work together to protect families and their belongings from life’s uncertainties. For more than 90 years, our innovative drive has kept us a step ahead of our customers’ evolving needs. Your role in the team is with Arity, a data and analytics company focused on improving transportation. We collect and analyze enormous amounts of data, using predictive analytics to build solutions with a single goal in mind: to make transportation smarter, safer, and more useful for everyone.

As part of that team, your work will showcase both your intelligence and your creativity as you tackle real-world problems and put your talents towards transforming transportation. Arity is committed to supporting an inclusive and diverse environment where you can thrive and learn from others.

This role is responsible for executing multiple tracks of work to deliver Big Data solutions enabling advanced data science and analytics. This includes:

• Working with the team on new Big Data systems for analyzing data
• Coding and development of advanced analytics solutions to make/optimize business decisions and processes
• Integrating new tools to improve descriptive, predictive, and prescriptive analytics

The role contributes to the structured and unstructured Big Data / Data Science tools of Arity from traditional to emerging analytics technologies and methods. The role is responsible for assisting in the selection and development of other team members.

Key Responsibilities

• Participate in the development of moderately complex and occasionally complex technical solutions using Big Data techniques in data & analytics processes
• Develop innovative solutions within the team
• Participate in the development of moderately complex and occasionally complex prototypes and department applications that integrate Big Data and advanced analytics to make business decisions
• Use new areas of Big Data technologies (ingestion, processing, distribution) and research delivery methods that can solve business problems
• Understand the Big Data related problems and requirements to identify the correct technical approach
• Take coaching from key team members to ensure efforts within owned tracks of work will meet their needs
• Execute moderately complex and occasionally complex functional work tracks for the team
• Partner with Arity Technology teams on Big Data efforts
• Partner closely with team members on Big Data solutions for our data science community and analytic users
• Leverage and use Big Data best practices / lessons learned to develop technical solutions

Essential Criteria

• All applicants must demonstrate they have a legal right to work in the UK for employment at Arity. Arity is not providing sponsorship for this vacancy.
• A minimum of 3 years’ experience in Python and/or Scala
• A minimum of 2 years’ experience with building big data ETL pipelines
• A minimum of 2 years’ experience in one or more of the following: Java, Kotlin, GoLang
• Experience in Test Driven Development (TDD)
• Demonstrated experience managing large data sets
• AWS Cloud knowledge
• Clear understanding of data management, process automation, system monitoring, and maintainability

Desirable Skills

• GCP services knowledge
• Experience building Data Lakes (preferably integration with Starburst)

Supervisory Responsibilities

This job does not have supervisory duties.

Why join us?

Allstate NI is proud to be Allstate’s European Digital Centre of Excellence—recent winners of ‘Best Use of Cloud Services’ at the Belfast Telegraph IT Awards 2024, and recognised for our community and sustainability impact at the 2024 Business in the Community Awards and Gold accreditation for Environmental Responsibility. We’re a product-driven, cloud-first organisation delivering real outcomes through modern technology, a digital product-centric talent model, and a culture rooted in engineering excellence. Our teams work in cross-functional structures, guided by an outcome-based delivery approach that accelerates speed, agility, and value.

We offer:

• A generous, flexible benefits package including annual leave, healthcare and dental cover, pension, and lifestyle discounts
• Access to world-class learning platforms and award-winning L&D
• Clear career paths, internal mobility, and a strong focus on growth
• A people-first culture with flexible working options

Be part of a high-performing, socially responsible organisation where your work has purpose, and your growth is supported every step of the way.

Statement on Fair Employment and Equal Opportunities

Allstate NI wishes to ensure equal opportunity is given to all job applicants. This company will not discriminate on the grounds of race, gender (including gender reassignment status), sexual orientation, religious belief, political opinion, marital status, age or disability. We are an equal opportunities employer. We welcome applications from all suitably qualified persons. However, as women are currently under-represented in our workforce, we would particularly welcome applications from women. All appointments will be made on merit. Applicants should note Allstate NI complete AccessNI background checks on all candidates offered a position.