OpenSource Data Platform Solution Engineer
Whitehall Resources are currently looking for a OpenSource Data Platform Solution Engineer in South Yorkshire for an initial 12-month contract.
Job Spec
Cybersecurity Sciences & Analytics (CSA) is a ~$50m and ~150 headcount function, bringing together expertise across cybersecurity, data science, AI, software and cloud engineering, governance, and programme management to:
Main responsibilities
- Deliver cyber analytics capabilities that measurably improve cyber prevention, control effectiveness, and the resilience of operations across the cyber lifecycle
- Engineer, automate, and continuously enhance cyber detection and response capabilities to enable effective, resilient, and scalable security operations
- Provide trusted Cyber risk, control, and regulatory insights that inform executive decision‑making and investment prioritization
- Ensure cyber products, data, and bank‑wide AI use cases comply with cyber risk appetite, regulatory requirements, and data standards, through effective data and AI governance
- Evolve Cyber analytics platforms, products, and engineering practices through a controlled maturity roadmap, to improve platform and operational resilience, efficiency, and sustainability without increasing risk
- Define and maintain architecture artefacts, including HLDs/LLDs, reference architectures, patterns, ADRs (Architecture Decision Records), and reusable templates.
- Contribute to the design and evolution of a petabyte‑scale data platform built on open‑source technologies.
- Evaluate technologies and drive standardisation: assess OSS components and vendors, run PoCs, document trade‑offs, and promote reusable patterns to reduce fragmentation.
- Own the end‑to‑end solution design for custom‑built applications that support the Bank’s cybersecurity ambition.
- Provide architecture and design input to software engineering and platform teams to support implementation and delivery.
- Partner with central architecture and governance teams to ensure solutions align with the Bank’s controls, policies, standards, and guardrails.
- Help ensure solutions remain compliant, operationally supportable, and within agreed cost boundaries.
Person requirements
- Strong team player, able to collaborate effectively with colleagues across geographies, time zones, and cultures.
- Excellent written and verbal communication skills, with the ability to explain complex technical topics clearly.
- Able to influence without authority; confident facilitating design workshops and governance forums.
- Strong prioritisation and pragmatic decision‑making, balancing target architecture with delivery realities.
- Comfortable designing solutions end‑to‑end, while also able to roll up sleeves and contribute hands‑on (including coding where needed).
Required skills
- Demonstrable experience designing, deploying, and operating big data platforms at scale.
- Hands‑on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies).
- Knowledge of cloud platforms and traditional on‑premise hosting; hands‑on experience with Azure, AWS and/or GCP is beneficial.
- Proven security architecture expertise, including:
- Threat modelling and secure‑by‑design practices
- IAM (RBAC/ABAC), privileged access concepts, and service‑to‑service authentication
- Encryption in transit/at rest, key management, secrets management
- Secure logging/telemetry, auditability, and evidence‑driven control assurance
- Secure SDLC, vulnerability management, and dependency/supply‑chain risk controls
- Proven experience delivering end‑to‑end solution designs, from requirements through to production operations and support.
Person Spec
- Strong team player, able to collaborate effectively with colleagues across geographies, time zones, and cultures.
- Excellent written and verbal communication skills, with the ability to explain complex technical topics clearly.
- Able to influence without authority; confident facilitating design workshops and governance forums.
- Strong prioritisation and pragmatic decision‑making, balancing target architecture with delivery realities.
- Comfortable designing solutions end‑to‑end, while also able to roll up sleeves and contribute hands‑on (including coding where needed).
