Llm Pre-Training & Distributed Engineer (Ai Infrastructure)

HYPHEN CONNECT

Remote, Remote

Australia We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes. Responsibilities Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM. Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors. Automate checkpointing and failure recovery during month-long training runs. Required Skills Deep expertise in 3D parallelism (Data, Tensor, Pipeline). Strong systems engineering background (C++, CUDA, Python). #J-18808-Ljbffr

Apply Now

Published 4 days ago

Report job