Software Development · Seattle, WA

Senior Software Engineer II- Streaming and Durability

Compass · Seattle, WA · Posted Jul 22, 2026

This posting comes from Compass. Confirm the details on their site before applying.

Overview

Every transaction, every listing view, every agent interaction at Compass touches a streaming pipeline, a database, or a cache owned by the Streaming & Durability team. This team is responsible for the foundational data layer: the systems that have to be fast, correct, and isolated across thousands of tenants, every single time. We don't just keep the lights on.

Responsibilities

  • Own Kafka at Compass end to end, across our MSK and Confluent clusters: platform health, upgrades, capacity, and cost.
  • Set the technical direction for how the rest of engineering uses streaming.
  • Run the Kafka operational program: client authentication and access standards, Schema Registry ownership with PR-driven schema registration, Kafka Connect upgrades and standard management, producer compression defaults, and the runbooks and drills that let application teams recover quickly.
  • Build and scale Kafka/MSK streaming infrastructure, the connective tissue between Compass services, including topic architecture, consumer patterns, partition rebalancing, upgrades, and dead-letter-queue and redrive guidance.
  • Build multi-tenant data isolation into Compass's data layer, shipping SDKs in Go, Java, and Python that enforce tenant-safe access patterns so product engineers can move fast without risking cross-tenant data exposure.
  • Support the database and caching fleet (RDS/Aurora PostgreSQL, ElastiCache Redis/ValKey, DynamoDB) alongside our database platform partners, including replication, failover, and capacity decisions.
  • Drive production database upgrades at fleet scale, moving toward automation, with Terraform state changes across hundreds of clusters and zero customer-facing impact.
  • Develop deep observability into the data layer and streaming platform, including SLI dashboards, tenant-dimensioned telemetry, and real-time performance insights that let the team (and the company) understand system behavior before customers do.
  • Partner directly with product and platform teams to assess database readiness for scaling events, migrate workloads onto golden-path infrastructure, and resolve the hardest production incidents in the stack.

Requirements

  • You've operated Kafka/MSK at scale and dealt with the hard parts: upgrades, consumer lag, partition rebalancing, and authentication (SASL/SCRAM or IAM).
  • You know the surrounding Kafka ecosystem (Schema Registry, Kafka Connect) well enough to run it and to set standards for other teams.
  • 7+ years of experience building and operating distributed systems at production scale in AWS.
  • Working knowledge of a major managed database (PostgreSQL or similar), enough to partner with a dedicated database team and reason about the infrastructure around it.
  • You've built shared tooling (SDKs, client libraries, platform APIs), and you understand what it takes to make infrastructure accessible and safe for engineers who aren't infrastructure specialists.
  • Kafka client libraries are a plus.
  • You think in trade-offs, not absolutes.
  • You can articulate why you'd choose a soft application-enforced boundary over a hard infrastructure lock and when that calculus changes.
  • You write Go (or can ramp quickly), with Java a plus, and you're comfortable in Terraform, Kubernetes, and IAM-heavy AWS environments.
  • You've led cross-team operational programs, including change management, migration coordination, and rollback planning, where the blast radius of a mistake extends well beyond your own team.
  • You can drive a technical decision end to end: frame the options, prove them out with benchmarks or a proof of concept, and lay out cost and risk for senior leaders.

Preferred qualifications

  • You've worked on multi-tenant isolation problems such as key prefixing, row-level security, and RBAC/ACL models and have opinions about where application-layer enforcement breaks down.
  • You've built or contributed to observability platforms, specifically for database and cache monitoring.
  • You've worked on Kafka cost and efficiency, such as compression, retention, and cross-AZ data transfer.

Benefits

  • The base pay range for this position is $176,000-$196,100; however, base pay offered may vary depending on job-related knowledge, skills, and experience.
  • Bonuses and restricted stock units may be provided as part of the compensation package, in addition to a full range of benefits.
  • Base pay is based on market location.
  • Minimum wage for the position will always be met.
  • Participation in our incentive programs (which may include eligible cash, equity, or commissions).
  • Plus paid vacation, holidays, sick time, parental leave, and recharge leave; medical, tele-health, dental and vision benefits; 401(k) plan; flexible spending accounts (FSAs); commuter program; life and disability insurance; Maven (a support system for new parents); Carrot (fertility benefits)…
  • Notice for California Applicants
  • Los Angeles County Fair Chance Notice

Great fits start here.

Whether you're looking for your next role or your next team member, give people and possibilities a place to meet.