Senior GPU Cloud East-West Network Expert SRE SME

You will operate, monitor, tune, and troubleshoot InfiniBand and RoCE fabrics for GPU clusters. You will integrate network telemetry into the collection pipeline, define fault-prediction signals, automate routine mitigations, and coordinate escalations with Nvidia and Mellanox support.

Responsibilities

• Operate InfiniBand fabrics for GPU clusters
• Deploy and operate RoCEv2 networks for RDMA workloads
• Monitor and manage fabrics using UFM
• Tune InfiniBand and RoCE performance, routing, congestion control, and traffic isolation
• Tune NCCL communication and GPU network topology configuration
• Manage firmware lifecycles for InfiniBand switches and HCAs
• Diagnose network faults and routing anomalies
• Coordinate escalations, bugs, and RMAs with Nvidia and Mellanox support
• Integrate InfiniBand and RoCE telemetry into the collection pipeline
• Convert incidents and mitigations into fault-prediction examples and automated workflows

Requirements

• 5+ years of data center networking experience
• At least 3 years focused on InfiniBand or RoCE fabrics
• Experience deploying and operating Nvidia or Mellanox InfiniBand switches at scale
• Knowledge of InfiniBand subnet management, partitioning, and QoS
• RoCEv2 deployment experience including PFC, ECN, and DCQCN
• UFM or equivalent fabric-management proficiency
• Knowledge of 400G and 800G optics and cabling standards
• Experience diagnosing InfiniBand and RoCE issues with ibdiagnet, perfquery, and ibstat
• Understanding of NCCL and GPU communication topology
• Experience with telemetry-driven operations, dashboards, or alerts