Senior Math Libraries Engineer - LLM Integration and Developer Experience

We are seeking a Senior Math Libraries Engineer, LLM Integration and Developer Experience to join NVIDIA, the company behind the world‑leading GPU‑accelerated libraries and SDKs that power AI, data analytics, scientific simulation, computer vision, autonomous vehicles and many other cutting‑edge fields. In this position you will help bring the CUDA‑X math library stack into modern large‑language‑model (LLM) and agentic AI workflows, enabling both human engineers and AI agents to build high‑performance applications more easily. Your daily work will revolve around shaping the architecture and implementation of math‑library APIs and their documentation so they are "LLM‑first": introspectable, explainable and readily synthesised by AI agents. You will collaborate with internal and external stakeholders to deliver timely LLM‑enhanced releases, create metrics, tools and processes that gauge the impact and quality of LLM‑generated code, and prototype solutions that translate existing APIs into representations that work smoothly with popular code‑generation tools. The role requires a PhD or MSc in Computational Science, Computer Science, Applied Mathematics or a closely related discipline (or equivalent experience) and at least three years of relevant engineering experience. You must possess deep knowledge of LLMs, fine‑tuning, reinforcement learning, retrieval‑augmented generation, model‑centric programming and agent tooling, as well as a proven track record of designing clear, composable APIs and delivering well‑documented code for complex technical domains. The ability to juggle multiple projects, work independently with minimal direction, and maintain excellent collaboration, communication and documentation habits is essential. Experience in AI‑assisted software engineering, code generation or programming‑language design is a plus, as is familiarity with NVIDIA's CUDA‑X math libraries such as cuBLAS, cuFFT, cuSOLVER and cuSPARSE. The position is based in Poland, where NVIDIA offers a competitive salary that reflects location and experience, and provides the chance to work alongside some of the most talented and forward‑thinking engineers in the industry. What the role asks for: - PhD or MSc in Computational/Computer Science, Applied Math, or equivalent - Minimum 3 years of relevant engineering experience - Strong knowledge of LLMs, fine‑tuning, RL, RAGs, MCP, agent tooling - Proven ability to design clear, composable APIs and write well‑documented code - Ability to prioritize several projects and work independently with minimal direction - Excellent collaboration, communication and documentation habits - Nice‑to‑have: experience in AI‑assisted software engineering or code generation - Nice‑to‑have: familiarity with CUDA‑X math APIs (cuBLAS, cuFFT, cuSOLVER, cuSPARSE)