Senior Deep Learning Engineer, Accuracy Evaluation

NVIDIA, a leader in accelerated computing and AI infrastructure, is looking for a Senior Deep Learning Engineer, Accuracy Evaluation. The role focuses on creating and refining methods to assess the performance of cutting‑edge deep learning models such as large language models, retrieval‑augmented generation, autonomous agents, and vision systems. By joining a fast‑growing company at the heart of the AI revolution, you will help shape the future of artificial intelligence and influence NVIDIA's roadmap. In this position you will design and construct decision‑grade evaluation environments that cover reasoning, multimodal, long‑context, and agentic capabilities. You will research novel evaluation methodologies for emerging model families where existing benchmarks fall short, and you will build and maintain the supporting infrastructure, including benchmark suites, regression CI pipelines, and statistical analysis tools, that serve model, product, and applied‑research teams across NVIDIA. Collaboration with model research, training, and customer groups will be essential to turn evaluation signals into concrete release decisions, training directions, and competitive positioning. The ideal candidate holds a BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related discipline and brings at least six years of hands‑on experience evaluating large language or multimodal models, especially in agentic or multi‑turn contexts. A solid statistical background is required, encompassing experimental design, significance testing, and regression analysis, as well as the ability to separate signal from noise at scale. Proven experience building reproducible evaluation pipelines and benchmark harnesses, together with clear, precise communication of quantitative findings to researchers and senior leadership, is mandatory. Additional strengths that set candidates apart include deep familiarity with open‑source evaluation frameworks and a track record of designing benchmarks for agentic systems such as tool‑use or environment‑based tests. Publishing or contributing to evaluation research, as well as expertise in measuring model accuracy under low‑precision inference (FP8, INT4, quantization‑aware settings), are valued. Comfort operating large‑scale workloads on HPC/Slurm clusters and managing experiments with MLflow or Weights & Biases further enhances effectiveness. NVIDIA promotes a diverse, inclusive environment and offers competitive compensation based on location and experience. What the role asks for: - BS, MS, or PhD in Computer Science, ML, Statistics, or related field - 6+ years hands‑on experience evaluating LLMs or multimodal AI - Strong statistical foundations (experimental design, significance testing, regression) - Proven experience building reproducible evaluation pipelines and CI systems - Clear, precise communication of quantitative results to stakeholders - Deep familiarity with open‑source evaluation frameworks (nice-to-have) - Experience designing evaluations for agentic systems (nice-to-have) - Record of publishing evaluation research or benchmark design (nice-to-have) - Experience measuring accuracy under low‑precision inference (nice-to-have) - Comfort running large‑scale workloads on HPC/Slurm clusters (nice-to-have)