Member of Technical Staff Data Infrastructure Storage
Posted Updated
Description
You will build and operate a unified, high-performance data layer across parallel file systems, object storage, distributed caches, and key-value stores. You will optimize data placement, replication, caching, and movement across clouds so training and inference workloads can access datasets, checkpoints, model weights, and KV cache at scale. You will develop performance-critical data-path code, deploy storage clusters, and diagnose bottlenecks across kernels, networks, caches, and storage backends.
Responsibilities
- Build a unified high-performance data layer across file systems, object storage, distributed caches, and key-value stores
- Design data placement, replication, caching, and movement across clouds
- Ensure datasets, checkpoints, model weights, and KV cache are available for GPU workloads at scale
- Develop performance-critical storage and data-path code
- Deploy and operate storage clusters across clouds using infrastructure as code
- Benchmark, profile, and resolve bottlenecks across kernels, networks, caches, and storage backends
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience
- 5+ years of experience designing, building, and operating production storage, caching, or data infrastructure at scale
- Strong software development skills in C/C++, Rust, or Go and experience writing performance-critical data-path code
- Experience using Python or Go for automation
- Deep understanding of distributed storage fundamentals, including consistency models, replication, consensus, metadata scaling, caching, and failure recovery
- Hands-on Kubernetes experience, including CSI for mounting high-performance volumes into ephemeral GPU pods
- Experience deploying and operating storage clusters such as WEKA, VAST, Lustre, Ceph, or MinIO across clouds using Terraform or Pulumi
- Proficiency with benchmarking and profiling tools such as fio, iperf3, NCCL tests, perf, and eBPF
Benefits
- Equity
