Software Development · San Francisco, CA

Full-time

Software Engineer, Collective Communication

OpenAI · San Francisco, CA · Posted Apr 20, 2024

This posting comes from OpenAI. Confirm the details on their site before applying.

Overview

The Workload Networking team is responsible for the collective communication stack used in our largest training jobs. Using a combination of C++ and CUDA we work on novel collective communication techniques that enable efficient training of our flagship models on our largest custom built supercomputers. The models we train are key ingredients to the AI research progress at OpenAI and the field as a whole, and we continually incorporate learnings from our entire research org into our training platform.

Responsibilities

  • Collaborate closely with ML researchers to design and implement efficient collective operations in C++ and CUDA.
  • Ensure that our largest training jobs take full advantage of the different network transports used in our supercomputers.
  • Work on simulations to inform our future supercomputer network designs.

Requirements

  • Have written distributed algorithms using RDMA in the past.
  • Are comfortable writing low level performance sensitive CPU and/or GPU code.
  • Are familiar with network simulation techniques.

Benefits

  • Bonus, Relocation assistance

Great fits start here.

Whether you're looking for your next role or your next team member, give people and possibilities a place to meet.