Software Development · Tel Aviv, Israel

Full-time

Senior Software Engineer, LLM Inference

NVIDIA · Tel Aviv, Israel · Posted Jul 21, 2026

This posting comes from NVIDIA. Confirm the details on their site before applying.

Overview

NVIDIA has been at the forefront of the deep learning revolution, pioneering innovations that have transformed the entire field. As the leading provider of GPUs and AI computing platforms, NVIDIA has empowered researchers and engineers worldwide to accelerate breakthroughs in artificial intelligence. We seek a versatile Senior Software Engineer who is passionate about performance optimization and generative AI.

Responsibilities

  • Implement and optimize inference algorithms for LLM and omnimodal architectures, including hybrid Mamba-Transformer and mixture-of-experts models
  • Profile inference pipelines using NVIDIA's profiling and simulation tools. Correlate simulation predictions against real hardware across data center and edge devices
  • Write and tune GPU kernels (CUDA, Triton) for operators like fused MoE layers, SSM state updates, and quantized GEMMs
  • Solve distributed inference problems: expert parallelism, communication-compute overlap, collective tuning, multi-node deployment
  • Build production-grade software inside major open-source libraries - vLLM, SGLang, Dynamo, FlashInfer
  • Own optimization features end-to-end, from scoping through delivery, collaborating with research, product, and engineering teams worldwide
  • B.Sc., M.Sc., or equivalent experience in Computer Science or Computer Engineering
  • 5+ years of hands-on software engineering experience in performance-critical systems
  • Solid understanding of deep learning architectures (Transformers, SSMs, MoE, …)
  • Experience with systems where hardware constraints matter: GPU programming, memory hierarchy, networking, or distributed computing
  • Strong software engineering fundamentals: clean design, extensibility, testability. Good judgment about when complexity is warranted
  • Effective communicator who works well across teams and time zones
  • Experience optimizing deep learning workloads on NVIDIA GPUs using roofline models, Nsight/PyTorch profilers and end-to-end traces

Preferred qualifications

  • Contributions to open-source inference runtimes and libraries - vLLM, SGLang, FlashInfer, Dynamo or similar
  • Hands-on work with LLM quantization (FP8, NVFP4, MXFP8, mixed-precision) and practical understanding of numerical precision tradeoffs
  • Track record with distributed inference at scale: tensor parallelism, pipeline parallelism, expert parallelism, disaggregation, multi-node orchestration
  • Deep knowledge of the latest LLM architectural trends: multi-token predictors, sparse hybrid models, attention and state-space mechanisms
  • Experience with performance modeling and simulation-to-silicon correlation

Great fits start here.

Whether you're looking for your next role or your next team member, give people and possibilities a place to meet.