Cloud Engineer (Python, AWS) in Edinburgh

We are seeking a skilled Splunk Cloud & ITSI Engineer to design, build, and maintain enterprise‑grade monitoring, analytics, and service‑health solutions. This role combines ITSI service‑centric engineering with Splunk Cloud operational monitoring, including dashboards, alerts, reports, and data governance. The engineer will develop ITSI service models, KPIs, correlation logic, and episodes while also supporting Splunk Cloud administration, CIM alignment, RBAC, and search performance hygiene. This position is ideal for someone strong in ITSI with solid Splunk Cloud monitoring and admin capabilities.

Key Responsibilities

• Build and maintain ITSI service models, including service trees, dependencies, entity rules, and health-scoring frameworks.
• Develop KPIs, multi-KPI logic, adaptive/time-based thresholds, and SLO-aligned indicators using golden signals.
• Configure and optimize Notable Event Aggregation Policies (NEAP) to group alerts into meaningful episodes and reduce noise.
• Create Glass Tables, Deep Dives, Service Analyzer views, and Splunk dashboards for executive and operational visibility.
• Implement ITSI-driven alerting with enrichment, routing to ITSM, suppression windows, and maintenance schedules.
• Build and tune correlation searches powering episodes, service degradation alerts, and automated remediation workflows.
• Support Splunk administration including index/RBAC governance, data onboarding (HEC, UF, DS), CIM alignment, and ingest quality checks.
• Apply search-performance best practices including workload rules, scheduling hygiene, DMA, and summary indexing.
• Develop and maintain Splunk Cloud dashboards, alerts, and scheduled reports for service health and reliability monitoring.
• Build operational dashboards using SPL, data models, and accelerated searches for real-time visibility across logs, metrics, and events.
• Create alerting frameworks with severity levels, routing rules, throttling, and alert hygiene standards.
• Develop scheduled reports, summary indexes, and data model accelerations to optimize performance and reduce Cloud compute cost.
• Manage and optimize knowledge objects including macros, lookups, event types, tags, and field extractions.
• Integrate ITSI and Splunk Cloud with CMDB/ITSM systems, webhook automation, and AIOps/ML-based anomaly detection frameworks.
• Troubleshoot slow or high-latency searches, identify bottlenecks, and implement best-practice SPL optimization.
• Develop, optimize, and maintain advanced SPL queries for dashboards, alerts, correlation searches, and analytics.
• Integrate ML model outputs into dashboards, alerts, and service health indicators for predictive insights.

Required Skills & Experience

• 4–8 years of hands-on experience with Splunk Enterprise / Splunk Cloud.
• Strong proficiency in SPL, search optimization, and dashboard development.
• Deep experience with ITSI, including service modelling, KPIs, thresholds, NEAP, and Service Analyzer.
• Experience with MLTK, anomaly detection, forecasting, and operationalizing ML models.
• Strong understanding of observability concepts (logs, metrics, traces, golden signals).
• Hands-on experience with data onboarding, HEC, Universal Forwarders, Deployment Server, and CIM alignment.
• Knowledge of indexing, RBAC, data models, summary indexing, and workload management.
• Ability to troubleshoot search performance, ingestion issues, and data quality problems.
• Preferred Certification (Splunk Admin, ITSI Admin).