Plain-AyI
← All projects
Training & fine-tuning

Megatron-LM

NVIDIA's software for training very large AI language models across many graphics cards at once.

Research-backed

What it does

Megatron-LM is NVIDIA's toolkit for training transformer models, the design behind most modern AI language models, at very large scale. Training such models needs many graphics cards working together, and this software splits the work between them efficiently.

The repository has two parts. Megatron Core is a library of optimised building blocks for people making their own training systems. Megatron-LM adds ready-made training scripts for researchers and for learning how large-scale training works.

A companion tool converts models between Megatron and the popular Hugging Face format.

Who it's for

Research teams and engineers training large AI models on powerful GPU clusters.

Can I use it without coding?

Developers only. Needs programming skills and access to NVIDIA graphics cards, usually many of them. New to this? Start here.

Can I run it?

Cost
FreeNothing to pay or sign up for.
Runs on
Server
Computer
Made for servers or data centres
API key
Not needed

Try it

Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?

Before you install

  • Only download from the official GitHub page. Copies with similar names exist for many popular projects.
  • No standard license listed. Check the GitHub page before using it for business.
  • More on staying safe

Where people are talking about it