Software Development · San Francisco, CA

Staff Software Engineer, RL Environments

Scale AI · San Francisco, CA · Posted Sep 2, 2026

This posting comes from Scale AI. Confirm the details on their site before applying.

Overview

As a Staff Software Engineer, RL Environments, you'll own the technical foundation for how Scale builds, runs, verifies, and delivers RL environments at scale. An RL environment is a real piece of software: a containerized world with real dependencies, real state, real tools, and a grader that has to be correct even when the agent is creative about breaking it.

Responsibilities

  • As a Staff Software Engineer, RL Environments, you'll own the technical foundation for how Scale builds, runs, verifies, and delivers RL environments at scale.
  • An RL environment is a real piece of software: a containerized world with real dependencies, real state, real tools, and a grader that has to be correct even when the agent is creative about breaking it.
  • Building one is a full-stack engineering problem.
  • Building thousands of them reproducibly, cheaply, with trustworthy reward signals and throughput measured in millions of rollouts is a systems problem that very few people have solved.
  • You'll work on both.
  • You'll design the platform: sandboxed execution, environment packaging and versioning, rollout orchestration, trajectory capture, verifier frameworks, and the authoring surfaces that let engineers and domain experts produce environments without reinventing infrastructure each time.
  • And you'll go deep on the environments themselves by instrumenting real applications, designing task suites that expose specific capability gaps, and building graders that hold up under adversarial optimization.
  • This is a hands-on engineering role.
  • You'll set technical direction across multiple teams, and you'll still be the person who writes the hard part.

Requirements

  • 8+ years of software engineering experience with strong fundamentals in distributed systems, system design, data structures, and algorithms.
  • Strong Python skills and a track record of shipping production software; comfort in at least one other part of the stack (TypeScript/React, Go, Rust, or similar).
  • Deep experience with containerization and sandboxed execution, including Docker, VMs, gVisor/Firecracker, Kubernetes, or equivalent.
  • Experience building or operating high-throughput backend systems: orchestration, job scheduling, queuing, and large-scale data pipelines.
  • Hands-on experience building with LLMs including agent loops, tool calling, MCP, or eval harnesses, and enough intuition about model behavior to reason about what a training signal actually teaches.
  • Demonstrated ability to own ambiguous, undefined problems end to end and drive them to a shipped system.
  • Excellent written and verbal communication; ability to align engineers, researchers, and non-engineering partners on a technical direction.

Preferred qualifications

  • Direct experience building RL environments, agentic benchmarks, or eval harnesses (SWE-bench-style task suites, terminal or browser environments, tool-use benchmarks, or in-house equivalents).
  • Familiarity with post-training methods: RLHF, RLAIF, RLVR, GRPO/PPO-family algorithms, rejection sampling, reward modeling, and the practical failure modes of each.
  • Experience designing verifiable reward signals, and firsthand experience with reward hacking and how to defend against it.
  • Experience with RL training or serving stacks (verl, TRL, Ray, vLLM, SGLang, or similar).
  • Experience with high-scale sandbox or code-execution infrastructure, remote development environments, or CI systems.
  • Experience with cloud-native infrastructure across AWS/GCP/Azure, Infrastructure as Code, and CI/CD.
  • Strong observability instincts: tracing, structured logging, and metrics for systems whose failure modes are statistical rather than binary.
  • Experience building internal tools that non-engineers rely on daily, especially data-dense review and annotation interfaces.
  • Experience in a research-adjacent engineering role, translating research goals into production systems.
  • Experience working directly with sophisticated external technical customers.

Benefits

  • Dental insurance, Vision insurance, Paid time off, Stock / equity, Commuter benefits

Great fits start here.

Whether you're looking for your next role or your next team member, give people and possibilities a place to meet.