At a Glance
- Tasks: Lead the strategy and design of AI telemetry systems for monitoring and analysing AI model performance.
- Company: Join a leading financial services company at the forefront of AI innovation.
- Benefits: Competitive salary, flexible working options, and opportunities for professional growth.
- Other info: Collaborative culture with a strong emphasis on cross-functional teamwork and career development.
- Why this job: Make a significant impact on AI governance and user experience in a dynamic environment.
- Qualifications: Experience in product management with a focus on observability and AI technologies.
The predicted salary is between 70000 - 90000 € per year.
Requirements
- Experience in product management with a strong foundation in observability, telemetry, data platforms, monitoring, or SRE / DevOps-driven products
- Understanding of LLMs, embeddings, vector search, MCP tools, and AI inference workflows
- Deep familiarity with logging, tracing, metrics, and event-based telemetry systems
- Ability to define data schemas, signal taxonomies, aggregation strategies and data contracts
- Strong analytical skills and ability to derive insights from large-scale system telemetry
- Experience working with senior engineering, data science, risk and governance stakeholders (Desirable)
- Exposure to financial data, analytics systems and enterprise-scale data workflows (Desirable)
- Familiarity with BI tools, metrics stores, distributed tracing and monitoring stacks (Desirable)
- Understanding of cloud infrastructure, serverless runtimes, API gateways and model hosting architectures (Desirable)
- Ability to drive cross-functional alignment and establish group-wide standards
What the job involves
As part of the LSEG AI, we are hiring a Senior Product Manager – AI Observability to own the strategy, design and rollout of telemetry systems that monitor, measure and analyse how AI models, MCPs, and AI-enabled features behave across all LSEG products. This role is responsible for defining how we collect, structure, analyse and act on AI-specific telemetry signals—including prompt patterns, model performance, MCP call usage, latency, error conditions, cost metrics, user behaviour signals, and system reliability.
You will develop a telemetry foundation that supports:
- AI governance & risk
- Model performance tracking
- Cost efficiency
- User experience optimisation
- Operational reliability
- Auditability
- And the long-term evolution of our AI platform
Telemetry & Observability Strategy:
- Define the end-to-end telemetry vision and roadmap for LLMs, MCPs, vector stores, embeddings, inference layers and AI-powered user experiences
- Establish a standardised telemetry schema for capturing prompts, tool calls, model responses, durations, errors, confidence signals, and quality indicators across business lines
- Partner with platform engineering to ensure instrumentation is consistent, scalable and compliant
Signal Design & Data Architecture:
- Identify key signals required for:
- Quality & reliability
- Latency & throughput
- Cost tracking & optimisation
- MCP usage patterns
- User workflow insights
- Failure pattern detection
- Guardrail and safety event monitoring
- Define retention rules, PII considerations, anonymisation and usage policies in partnership with AI Governance and Compliance
Platform & Tooling:
- Own requirements for dashboards, monitoring tools, model comparison views, anomaly detection alerts, and performance scorecards
- Design systems that allow product and engineering teams to self-serve insights about AI model behavior and MCP interactions
- Partner with engineering to build pipelines that support real-time and batch analytics
Cross-Functional Collaboration:
- Collaborate with product owners across LSEG to instrument their AI features consistently
- Partner with AI Evaluation PM (previous role), Model Risk, GSSR, Legal and Compliance to align telemetry with governance frameworks
- Work closely with Engineering and SRE teams to drive observability improvements and reliability engineering for AI systems
Optimisation & Insights:
- Identify cost inefficiencies across model and MCP usage, and drive recommendations to improve ROI
- Surface workflow-level insights on how customers use AI features—informing product roadmaps
- Partner with product leaders across divisions to understand how telemetry can improve customer experience, reliability and performance
Governance & Standards:
- Define and maintain AI telemetry standards and best practices across all LSEG divisions
- Contribute to LSEG’s AI governance and Responsible AI initiatives by providing data-driven insights on model behavior and user impact
- Ensure telemetry systems support auditability, compliance and explainability requirements
Measures of Success:
- Platform Reliability & Performance: Reduction in AI system error rates, latency spikes and operational incidents; Stability and transparency of MCP usage across product lines; Quality and coverage of telemetry signals across all AI-enabled products
- Insight Generation & Adoption: Adoption of telemetry dashboards and self-service tools across divisions; Number of identified and resolved model or MCP reliability issues driven by telemetry; Level of insight generated to influence product prioritisation and AI model choices
- Operational Efficiency & Cost: Improved cost transparency and cost-saving impact via telemetry-driven optimisation; Reduction in unnecessary model invocation or inefficient tool usage
- Governance & Compliance: Contribution to Responsible AI frameworks through robust, auditable telemetry; Improved explainability and traceability of AI features for internal/external stakeholders
Senior Product Manager (AI Observability) employer: Deepstreamtech
At LSEG, we pride ourselves on being an exceptional employer that fosters a culture of innovation and collaboration. As a Senior Product Manager in AI Observability, you will have the opportunity to work at the forefront of AI technology, driving impactful projects that enhance our products and services. We offer a supportive environment with ample opportunities for professional growth, competitive benefits, and a commitment to responsible AI practices, all set in a dynamic location that encourages creativity and teamwork.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Product Manager (AI Observability)
✨Tip Number 1
Network like a pro! Reach out to folks in your industry on LinkedIn or at events. A friendly chat can lead to opportunities that aren’t even advertised yet.
✨Tip Number 2
Show off your skills! Create a portfolio or a project that highlights your experience with telemetry and observability. It’s a great way to demonstrate your expertise beyond just words.
✨Tip Number 3
Prepare for interviews by diving deep into the company’s products and their AI features. Knowing how they work will help you discuss how you can improve their telemetry systems.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive!
We think you need these skills to ace Senior Product Manager (AI Observability)
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience in product management, especially in observability and telemetry. We want to see how your skills align with the requirements listed in the job description.
Showcase Relevant Projects:Include specific examples of projects where you've worked with AI models, data platforms, or monitoring systems. This helps us understand your hands-on experience and how you can contribute to our team.
Be Clear and Concise:When writing your application, keep it straightforward. Use clear language to describe your achievements and how they relate to the role. We appreciate brevity but also want to see your personality shine through!
Apply Through Our Website:Don’t forget to submit your application through our website! It’s the best way for us to receive your details and ensures you’re considered for the role. We can’t wait to hear from you!
How to prepare for a job interview at Deepstreamtech
✨Know Your Stuff
Make sure you brush up on your knowledge of observability, telemetry, and AI workflows. Be ready to discuss specific examples from your past experience that demonstrate your understanding of these concepts, especially in relation to LLMs and data platforms.
✨Showcase Your Analytical Skills
Prepare to talk about how you've derived insights from large-scale system telemetry in previous roles. Think of concrete examples where your analytical skills led to improvements in product performance or user experience.
✨Cross-Functional Collaboration is Key
Be ready to share experiences where you've successfully collaborated with engineering, data science, and governance teams. Highlight how you drove alignment and established standards across different functions, as this will be crucial for the role.
✨Understand the Bigger Picture
Familiarise yourself with the financial data and analytics systems relevant to the role. Being able to connect your technical expertise with business outcomes will show that you understand the strategic importance of AI observability in a financial context.
