Software Development · San Jose, CA

On-site Full-time

Inference Software Engineer

Etched · San Jose, CA · Posted Jun 17, 2025

This posting comes from Etched. Confirm the details on their site before applying.

Overview

Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting. Build, enhance, and scale our runtime, including multi-node inference, intra-node execution, state management, and robust error handling. Optimize routing and communication layers using our collectives.

Responsibilities

  • Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting.
  • Build, enhance, and scale our runtime, including multi-node inference, intra-node execution, state management, and robust error handling.
  • Optimize routing and communication layers using our collectives.
  • Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.

Requirements

  • Proficiency in C++ or Rust.
  • Understanding of performance-sensitive or complex distributed software systems like Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, InfiniBand).
  • Familiarity with PyTorch or JAX.
  • Ported applications to non-standard accelerator hardware or hardware platforms.

Preferred qualifications

  • Developed low-latency, high-performance applications using both kernel-level and user-space networking stacks.
  • Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns.
  • Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).
  • Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.

Benefits

  • Medical, dental, and vision packages with generous premium coverage
  • $500 per month credit for waiving medical benefits
  • Housing subsidy of $2.5k per month for those living within walking distance of the office
  • Relocation support for those moving to San Jose (Santana Row)
  • Various wellness benefits covering fitness, mental health, and more
  • Daily lunch + dinner in our office
  • Unlimited compute budget subject to ROI justification

Great fits start here.

Whether you're looking for your next role or your next team member, give people and possibilities a place to meet.