Staff Software Engineer (Agentic Search) - Crawler
careers.nebius.com
- Job written in
- English
- Location
- Zürich
- Work type
- On-site
- Type
- Full-time
Nebius is a fast‑growing cloud‑infrastructure firm that powers the worldwide AI market. The company delivers an end‑to‑end AI cloud stack, handling everything from data ingestion and model training to production deployment, while sparing customers the expense of building their own massive compute platforms. Headquartered in Amsterdam and listed on Nasdaq, Nebius operates research and development sites across Europe, the United Kingdom, North America and Israel, employing more than 1,500 engineers with deep expertise in hardware, software and artificial‑intelligence research. The opening is for a Staff Software Engineer focused on the acquisition and crawling layer of an innovative search engine built for AI agents. The role involves creating and maintaining systems that locate, retrieve and continuously refresh web‑scale content from public sites and other large data repositories. You will architect distributed crawlers, schedule jobs, set recrawl policies and devise freshness mechanisms, all while handling billions of URLs and massive data streams. The engineer must also build pipelines for ingesting structured feeds and partner data, guarantee that the crawling platform stays reliable under heavy load, and define metrics that track coverage, timeliness, throughput and content quality. Close collaboration with indexing and machine‑learning groups is required to ensure that harvested material satisfies downstream retrieval and ranking needs, and to enable safe experimentation with new crawling strategies. Candidates need a solid background in backend or distributed engineering, with at least eight years of relevant experience. Mastery of Go or C++ is essential, as is proven ability to work on systems that process tens of thousands of requests per second and manage billions of URLs. A thorough understanding of web protocols such as HTTP, DNS and TLS, together with experience in crawling, scraping and extracting content, is required. Applicants must have operated production‑grade services, diagnosed failures in distributed environments, and possess a deep grasp of scalability, fault tolerance and resource‑management principles. Legal eligibility to work in the country of application is also mandatory. Additional expertise that would strengthen an application includes prior work on web‑crawling projects, building streaming and event‑driven pipelines, and familiarity with messaging systems like Kafka, Pulsar, NATS or RabbitMQ. Experience designing distributed schedulers, queues and asynchronous processing frameworks, as well as knowledge of big‑data tools such as Spark, Flink, Beam or MapReduce, is valued. Backgrounds in advertising technology, social‑network platforms or large‑scale content services are also considered advantageous. Nebius offers competitive pay, opportunities for professional growth, a collaborative culture, and the chance to influence the future of AI through high‑impact projects in an international, innovative environment. What the role asks for: - 8+ years building backend or distributed systems - Strong proficiency in Go or C++ - Experience with massive distributed workloads (10k+ RPS, billions URLs) - Knowledge of HTTP, DNS, TLS, crawling and content extraction - Operated production services and debugged distributed failures - Deep grasp of scalability, fault tolerance and resource management - Authorized to work in the country of application - Designed and built web crawlers - Created streaming pipelines and event‑driven architectures - Used Kafka, Pulsar, NATS, RabbitMQ or similar - Developed distributed schedulers, queues and async processing - Familiar with Spark, Flink, Beam or MapReduce - Background in ad tech, social networks or large content platforms
Software Engineer: Lebenslauf-Vorlage
Bewirb dich mit einem Lebenslauf im Schweizer Aufbau — mit Beispieltext und den Anforderungen, die Software Engineer-Inserate am häufigsten nennen.
Lebenslauf-Vorlage Schweiz ansehenSoftware Engineer: was der Markt gerade verlangt
Software Engineer-Stellen gehören zu den regelmässig ausgeschriebenen Berufen auf SwissJobs.app.
Am häufigsten genannt in Software Engineer-Inseraten:
- Berufserfahrung12% der Inserate
- Kundenkontakt9% der Inserate
- Belastbar / flexibel9% der Inserate
Weitersuchen
- Alle Software Engineer Jobs in der Schweiz
- Alle Jobs in Zürich
- Lebenslauf als Software Engineer: Beispiel und Vorlage
- Vorstellungsgespräch als Software Engineer: die häufigsten Fragen
- Lebenslauf mit KI erstellen
- ATS-Lebenslauf prüfen
- Bewerbungsschreiben für dieses Inserat