Principal Data Engineer - GenAI & LLM Pipelines Lead

About the Role

We’re looking for a versatile Principal Data Engineer who can work across the full stack of Anaplan AI applications – from model integration to setting the technical direction for ingesting, transforming, storing, serving, and governing the data that powers our LLM‑based and agentic systems. The role builds AI features that can be used in real‑time, enabling business users to leverage GenAI in their planning workflows. Candidates must have deep knowledge of machine learning and strong data engineering skills.

Your Impact

• Design and build the retrieval layer powering RAG and agentic workloads, including vector and graph databases, hybrid search, and knowledge‑graph architecture.
• Develop end‑to‑end GenAI features, including backend API services, model integration, monitoring, evaluation, and deployments.
• Engineer feature and context pipelines that balance batch and streaming patterns to feed forecasting and anomaly‑detection models, collaborating closely with data scientists to productionize algorithms.
• Build a data plane for evaluation, implementing rigorous frameworks to continuously monitor, measure, and improve GenAI feature quality, accuracy, latency, and user satisfaction.
• Collaborate with data scientists to productionize ML models and forecasting algorithms.

Your Skills

• Extensive background in Data Science Engineering with a clear track record of principal‑level technical leadership.
• Hands‑on experience building and shipping AI/ML products in production.
• Deep practical experience with LLM‑based systems: RAG architectures, embedding pipelines, prompt and response logging, evaluation frameworks.
• Hands‑on expertise with vector databases, graph databases, and knowledge graphs.
• End‑to‑end exposure to model lifecycle development, including training and deploying ML models in production environments.
• Deep knowledge of LLM APIs, prompt engineering, and conversational AI patterns.
• Proficiency in Python and modern software development practices (testing, code review, CI/CD).

Preferred Skills

• Hands‑on experience with cloud‑native ML infrastructure platforms.
• Knowledge of vector databases (Pinecone, Weaviate, Qdrant) and embedding models.
• Experience with model serving frameworks (vLLM, TensorRT, Ray).
• Familiarity with Anaplan or similar enterprise planning platforms.
• Experience with A/B testing and experimentation frameworks for AI features.
• Experience with model observability tools (LangSmith, W&B, MLflow).

Commitment to Diversity, Equity, Inclusion and Belonging (DEIB)

We believe attracting and retaining the best talent and fostering an inclusive culture strengthens our business. DEIB improves our workforce, enhances trust with partners and customers, and drives business success. We foster an environment where all people are respected and valued regardless of gender identity or expression, sexual orientation, religion, ethnicity, age, neurodiversity, disability status, citizenship, or any other aspect that makes people unique. We hire you for who you are, and we want you to bring your authentic self to work every day.

Equal Employment Opportunity

As set forth in Anaplan’s Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive equitable benefits and all privileges of employment. Please contact us to request accommodation.