Realtime AI Inference Engineer for On-Device, Low-Latency

The Mission

At Iconic, our virtual actors don't just generate “text” or “actions”—they perform. They need to speak, move, and perceive in milliseconds, often running locally on a player's machine alongside a rendering engine. You will bridge the gap between massive research models and the constraints of real-time interactive entertainment.

The Role

You will architect and build the inference engine that powers our digital entities. Your main task will be tearing apart the model architecture to make it run as fast as possible on consumer hardware while keeping their abilities intact for the intended usage. As part of a small, focused team, you'll have significant autonomy and end-to-end ownership. You will work at the intersection of System ML and Game Tech. You might spend one day implementing a custom pruning algorithm for our TTS model, and the next day writing a C++ wrapper to expose that model to our game engine. You will work closely with our Character Research team to ensure that optimization never comes at the cost of the character's soul.

Key Responsibilities

• Architect Low-Latency Runtimes: Build and maintain high-performance inference pipelines for Multimodal LLMs, TTS, and Vision models, targeting both server-side (H100/A100) and consumer edge (RTX 5090, Apple Silicon) environments.
• State-of-the-Art Optimization: Implement advanced techniques like Speculative Decoding, KV-Cache quantization, PagedAttention, and Layer Pruning to minimize Time-To-First-Token (TTFT) and Time-Per-Output-Token (TPOT), maximizing throughput.
• Model Compression: Lead our efforts in post-training quantization (AWQ, GPTQ, GGUF) and distillation to fit massive models into consumer VRAM budgets.
• Engine Integration: Collaborate with the game engineering team to ensure thread-safe, non-blocking asynchronous inference within the game loop.
• Custom Kernel Development: Write custom ops in CUDA, Triton, or Metal when off-the-shelf kernels aren't fast enough.

Requirements

• MSc or PhD in Computer Science, Machine Learning, or a related field (or equivalent industry experience).
• Strong experience with model optimization techniques (quantization, pruning, distillation, knowledge transfer).
• Experience with LLM-specific inference optimizations (KV-cache management, speculative decoding, attention mechanisms).
• Proficiency in C/C++.
• Hands-on experience deploying ML models on-device or in latency-sensitive environments.
• Proficiency in Python and deep learning frameworks (PyTorch, JAX, or TensorFlow).
• Experience with inference optimization tools and runtimes (TensorRT, ONNX Runtime, Core ML, or similar).
• Strong systems and engineering skills.
• Excellent collaboration and communication skills.

Nice to Have

• Experience with On-Device AI stacks: ExecuTorch, CoreML, MLX, or ONNX Runtime.
• Experience in CUDA programming.
• Familiarity with non-NVIDIA compute (AMD/ROCm, DirectML, Vulkan Compute).
• Background in real-time systems or game engines (Unreal, Unity) or Real-Time Rendering.
• Publications or demonstrated work in efficient ML or model compression (NeurIPS, ICML, MLSys, etc.) or open-source contributions to projects like vLLM, SGLang, llama.cpp, or bitsandbytes.

Why Join Us

Be a foundational member of a team innovating at the intersection of AI, art, and storytelling. You'll help shape the research direction, culture, and technical foundations of a company building toward something genuinely new.

What we offer

• Competitive salary and equity compensation.
• 25 days annual leave + bank holidays.
• Private healthcare.
• Based in London with hybrid work.
• Inclusive & friendly company culture with socials and game breaks.

About Iconic

Iconic Interactive is a seed-stage startup building AI that breathes life into virtual worlds. The future of entertainment is personal: entire universes shaped around each of us, where you are not watching a story but living at the center of one, shaping it. We're building every layer of intelligence these experiences need: characters that feel and convey meaning, narrators that weave your story, and world directors that act like an ever-present game master: adapting, orchestrating, surprising. We're a growing team tackling some of the most fascinating problems in AI: creating minds that inhabit and shape new worlds.