Nebius in London is seeking a Staff Software Engineer to build the content acquisition and crawling infrastructure for a world-scale search platform designed for agentic AI. You will design distributed crawling, scheduling, and ingestion systems capable of handling billions of URLs and terabytes of data.
Collaborate with indexing and ML teams, improve observability, optimize resource use, and drive safe experimentation with crawling strategies.
Staff Software Engineer, Web-Scale Crawling & Ingestion employer: FDE Times
Databricks is an exceptional employer that fosters a dynamic work culture centred around innovation and collaboration. With a strong emphasis on employee growth, you will have access to cutting-edge technology and opportunities to lead impactful projects in the data and AI space. Located in a vibrant tech hub, our team enjoys a supportive environment that encourages creativity and professional development, making it a truly rewarding place to advance your career.
