At a Glance
- Tasks: Join our Inference team to build and maintain systems serving millions of users worldwide.
- Company: Innovative tech company focused on AI and machine learning.
- Benefits: Competitive salary, flexible work options, and opportunities for professional growth.
- Other info: Encouraging diverse applicants; strong candidates may not meet every qualification.
- Why this job: Make a real impact in AI while tackling complex distributed systems challenges.
- Qualifications: Experience in software engineering, particularly with distributed systems and cloud infrastructure.
The predicted salary is between 70000 - 90000 € per year.
Requirements
- Have significant software engineering experience, particularly with distributed systems
- Are results-oriented, with a bias towards flexibility and impact
- Pick up slack, even if it goes outside your job description
- Want to learn more about machine learning systems and infrastructure
- Thrive in environments where technical excellence directly drives both business results and research breakthroughs
- Care about the societal impacts of your work
- (Desirable) High-performance, large-scale distributed systems
- (Desirable) Implementing and deploying machine learning systems at scale
- (Desirable) Load balancing, request routing, or traffic management systems
- (Desirable) LLM inference optimization, batching, and caching strategies
- (Desirable) Kubernetes and cloud infrastructure (AWS, GCP)
- (Desirable) Python or Rust
Education requirements:
We require at least a Bachelor's degree in a related field or equivalent experience. We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed. Research shows that people who identify as being from underrepresented groups are more prone to experiencing imposter syndrome and doubting the strength of their candidacy, so we urge you not to exclude yourself prematurely and to submit an application if you're interested in this work.
What the job involves
Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.
- Designing intelligent routing algorithms that optimize request distribution across thousands of accelerators
- Autoscaling our compute fleet to dynamically match supply with demand across production, research, and experimental workloads
- Building production-grade deployment pipelines for releasing new models to millions of users
- Integrating new AI accelerator platforms to maintain our hardware-agnostic competitive advantage
- Contributing to new inference features (e.g., structured sampling, prompt caching)
- Supporting inference for new model architectures
- Analyzing observability data to tune performance based on real-world production workloads
- Managing multi-region deployments and geographic routing for global customers
Senior Software Engineer (Inference) employer: Deepstreamtech
As a Senior Software Engineer (Inference) at our company, you will join a dynamic team dedicated to pushing the boundaries of AI technology while enjoying a collaborative and inclusive work culture. We offer competitive benefits, opportunities for professional growth, and a commitment to societal impact, all within a vibrant location that fosters innovation and creativity. Here, your contributions will directly influence both business success and groundbreaking research, making it an exciting place for those passionate about technology and its potential to change the world.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Software Engineer (Inference)
✨Tip Number 1
Network like a pro! Reach out to folks in the industry, attend meetups, and connect with people on LinkedIn. You never know who might have the inside scoop on job openings or can put in a good word for you.
✨Tip Number 2
Show off your skills! Create a portfolio or GitHub repository showcasing your projects, especially those related to distributed systems or machine learning. This gives potential employers a taste of what you can do beyond your CV.
✨Tip Number 3
Prepare for technical interviews by brushing up on your coding skills and understanding system design principles. Practice common interview questions and consider mock interviews to build your confidence.
✨Tip Number 4
Don’t hesitate to apply through our website! Even if you don’t tick every box in the job description, we value diverse experiences and perspectives. If you're passionate about the role, go for it!
We think you need these skills to ace Senior Software Engineer (Inference)
Some tips for your application 🫡
Show Off Your Experience:Make sure to highlight your significant software engineering experience, especially with distributed systems. We want to see how your past projects align with the role, so don’t hold back on the details!
Be Results-Oriented:We love candidates who are results-oriented and flexible. In your application, share examples of how you've made an impact in previous roles, even if it meant stepping outside your usual responsibilities.
Emphasise Your Learning Mindset:If you're keen to learn more about machine learning systems and infrastructure, let us know! We appreciate candidates who are eager to grow and adapt, so mention any relevant courses or projects you've undertaken.
Apply Through Our Website:Don’t forget to apply through our website! It’s the best way for us to receive your application and ensures you’re considered for the role. Plus, we encourage everyone to apply, even if you don’t meet every single qualification.
How to prepare for a job interview at Deepstreamtech
✨Know Your Distributed Systems
Make sure you brush up on your knowledge of distributed systems before the interview. Be ready to discuss your past experiences and how you've tackled challenges in this area. Highlight specific projects where you implemented solutions that improved performance or efficiency.
✨Show Your Flexibility
Demonstrate your results-oriented mindset by sharing examples of when you've picked up slack outside your usual responsibilities. Companies love candidates who can adapt and contribute to different areas, so be prepared to talk about how you've done this in previous roles.
✨Get Familiar with Machine Learning
Since the role involves machine learning systems, take some time to learn about the latest trends and technologies in this field. Discuss any relevant experience you have with deploying ML systems at scale, and don't hesitate to mention any personal projects or research you've done.
✨Prepare for Technical Questions
Expect technical questions related to load balancing, request routing, and caching strategies. Brush up on Kubernetes and cloud infrastructure like AWS or GCP. Practising coding problems or system design scenarios can also help you feel more confident during the technical portion of the interview.
