We are seeking highly skilled engineers with expertise in machine learning, distributed systems, and high-performance computing to join our Research team. In this role, you will collaborate closely with researchers to build and optimize platforms that train next-generation foundation models on massive GPU clusters. Your work will play a critical role in advancing the efficiency and scalability of cutting-edge generative AI technologies.
Key Responsibilities Scale and optimize systems for training large-scale models across multi-thousand GPU clusters. Profile and enhance the performance of training codebases to achieve best-in-class hardware efficiency. Develop systems to distribute workloads efficiently across massive GPU clusters. Design and implement robust solutions to enable model training in the presence of hardware failures. Build tools to diagnose issues, visualize processes, and evaluate datasets at scale. Optimize and deploy inference workloads for throughput and latency across the entire stack, including data processing, model inference, and parallel processing. Implement and improve high-performance CUDA, Triton, and PyTorch code to address efficiency bottlenecks in memory, speed, and utilization. Collaborate with researchers to ensure systems are designed with optimal efficiency from the ground up. Prototype cutting-edge applications using multimodal generative AI. Qualifications Experience:
3+ years of professional experience in ML pipelines, distributed systems, or high-performance computing. Hands-on experience training large models using Python and PyTorch, with familiarity in the full pipeline: data processing, loading, training, and inference. Proven expertise in optimizing and deploying inference workloads, with experience in profiling GPU/CPU code (e.g., Nvidia Nsight). Deep understanding of distributed systems and frameworks, such as DDP, FSDP, and tensor parallelism. Strong experience writing high-performance parallel C++ and custom PyTorch kernels, with knowledge of CUDA and Triton optimization techniques. Bonus: Experience with generative models (e.g., Transformers, Diffusion Models, GANs) and prototype development (e.g., Gradio, Docker). Technical Skills:
Proficiency in Python, with significant experience using PyTorch. Advanced skills in CUDA/Triton programming, including custom kernel development and tensor core optimization. Strong generalist software engineering skills and familiarity with distributed and parallel computing systems. Note: This position is not intended for recent graduates.
Compensation The salary range for this role in California is $175,000–$250,000 per year. Actual compensation will depend on job-related knowledge, skills, experience, and candidate location. We also offer competitive equity packages in the form of stock options and a comprehensive benefits plan.