Splunk Lead - SC cleared

We are seeking a Senior Splunk Lead / Observability Engineering Lead to design, implement, and govern enterprise-wide monitoring and observability solutions across the Bank’s technology estate.

This role will act as a technical authority within the Observability team, driving best practices, automation‑first principles, and scalable monitoring architectures while working closely with engineering, DevOps, and product teams.

1. Seniority & Experience

• Minimum 10–12+ years of IT experience, with 5+ years in Splunk engineering/observability
• Proven experience in enterprise‑scale environments (banking/regulated preferred)

2. Splunk Expertise

• Strong hands‑on experience with:
  • Splunk Enterprise
  • Splunk ITSI (design, configuration, administration)
• Demonstrated experience designing large‑scale multi‑tenant Splunk platforms

3. Automation & Infrastructure as Code

• Hands‑on experience with:
  • Terraform for Splunk automation
  • GitHub (or similar CI/CD tooling)
• Proven ability to implement:
  • automated monitoring patterns and onboarding frameworks.

4. Observability & Monitoring Engineering

• Strong expertise in:
  • Monitoring strategy, design, and implementation
  • Observability frameworks (logs, metrics, traces)
• Experience integrating:
  • Multiple data sources into Splunk
  • Enterprise monitoring ecosystems

5. Platform Ownership & Solution Design

• Experience owning:
  • Monitoring platforms across lifecycle
  • Design → Build → Deploy → Operate
• Ability to:
  • Design modular, scalable observability architectures
  • Translate business needs into technical monitoring solutions

6. Data Analysis & Problem Solving

• Strong capability in:
  • Correlating data across systems
  • Identifying patterns, anomalies, and insights
• Proven analytical and troubleshooting skills

7. Integration & Tooling

• Hands‑on experience integrating Splunk with:
  • ITSM tools
  • Automation tools
  • External monitoring systems

8. Stakeholder & Leadership Capability

• Strong experience:
  • Working with engineering teams, SMEs, and business stakeholders
  • Influencing technical direction
• Ability to:
  • Mentor teams and drive best practices

9. Agile & Delivery

• Experience working in:
  • Agile/Scrum environments
• Ability to manage:
  • Multiple parallel workstreams and deadlines

10. Documentation & Communication

• Ability to:
  • Produce high‑quality technical documentation
  • Communicate complex concepts to technical and non‑technical stakeholders

Key Responsibilities

• Lead the design and implementation of monitoring and observability standards
• Define and deliver automation‑first monitoring patterns using Terraform and CI/CD
• Own and evolve Splunk platform architecture and roadmap
• Configure and optimise Splunk ITSI for enterprise use cases
• Integrate Splunk with:
  • Data sources
  • Monitoring tools
  • ITSM platforms
• Drive adoption of observability best practices across engineering teams
• Provide technical leadership and mentoring to the team
• Investigate incidents and improve:
  • Reliability
  • Performance
  • Monitoring coverage
• Collaborate with:
  • DevOps teams
  • Engineering teams
  • Product owners