Data Science and Analytics · Remote

Remote Full-time

Embedded AI Engineer, On-Device Models

Deepgram · Remote · Posted Jul 7, 2026

This posting comes from Deepgram. Confirm the details on their site before applying.

Overview

Deepgram is bringing the world’s most natural interface, voice, to the technology we use every day. Our speech models and real-time infrastructure power AI systems that keep up with the way people actually speak. More than 5,000 organizations run conversational voice AI on Deepgram in production, including Twilio, Cloudflare, Sierra, Decagon, Vapi, Cresta, Granola, and Jack in the Box.

Responsibilities

  • Write and optimize custom kernels and operators (C, C++, Rust, and platform assembly or intrinsics) for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs where the vendor's standard operator set is insufficient for Deepgram models.
  • Own target-side optimization: collapse models onto device execution units through quantization, operator fusion, memory layout, and architecture-specific compilation to meet latency, memory, power, and thermal budgets.
  • Integrate with vendor NPU/DSP toolchains and edge inference runtimes, and extend them with custom operators when the graph doesn't map cleanly.
  • Deliver kernels and runtime components as reusable building blocks that Applied ML Engineers can target when adapting models, with clear interfaces and documented constraints.
  • Build performance-critical runtime code for embedded environments, including embedded Linux, bare-metal, and RTOS targets.
  • Establish per-platform benchmarking and validation for latency, accuracy, power, memory footprint, and utilization, and catch regressions before they ship.
  • Partner with silicon and platform vendors on SDK integration and low-level performance tuning for new chipsets and reference platforms.
  • Feed hardware constraints back to Applied ML and Research so model designs are easier to land on constrained targets.

Requirements

  • Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices.
  • Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments.
  • Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation.
  • Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.
  • A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance.
  • Experience working close to the metal: bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development.
  • Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly.

Preferred qualifications

  • Experience with real-time audio processing on embedded platforms — DSP pipelines, audio codec optimization, wake-word or always-on listening, or streaming inference on microcontrollers and edge SoCs.
  • Depth in ML optimization techniques — custom quantization schemes, mixed-precision inference, or neural architecture search for edge targets.
  • Background in hardware evaluation and benchmarking — systematically comparing accelerators, SoCs, or GPUs for specific workload profiles.
  • Experience shipping AI features in consumer products at scale, and the instinct for what "production quality" means on a battery-powered device.
  • Familiarity with model compilation and optimization toolchains and their tradeoffs across hardware targets.
  • Experience with secure, robust on-device deployment practices — code signing, encrypted model storage, and safe update mechanisms.

Great fits start here.

Whether you're looking for your next role or your next team member, give people and possibilities a place to meet.