Russian tech giant Yandex has released a new open source tool to help AI companies save money and resources when training new models.
In its announcement, Yandex says the new YaFSDP tool can provide faster and more efficient large language model (LLM) training, and can help save up to 20% of GPU resources, saving money and time.Â
Yandex also published resources to try out YaFSDP on GitHub for anyone to view and experiment with.
How to train your LLMÂ
Yandex lays out a lot of technical details in its announcement post, saying the new open-source tool is “the most effective publicly available tool for enhancing GPU communication and reducing memory usage in LLM training”, and can offer up to 26% faster speeds than FSDP, the version before YaFSDP.Â
“LLM training is a time-consuming and resource-intensive process,” says Yandex. “Machine learning engineers and companies that develop their own LLMs invest significant time and GPU resources – which equals money – in training these models. The larger the model, the greater the time and expenses associated with its training.”Â
The company estimates that using YaFSDP to train a model with 70 billion parameters can save the resources of approximately 150 GPUs, translating to around $0.5 to $1.5 million (depending on the virtual GPU provider or platform) in monthly savings, if everything goes well.Â
Using Meta’s Llama models, which have been praised for being at the leading-edge while embracing open source, Yandex says the “final speedup” is 21% and 26% on Llama 2 70B and Llama 3 70B, respectively.Â
YaFSDP isn’t Yandex’s first foray into AI tools, having released CatBoost, YTsaurus, AQLM, and Petals, all of which handle a different niche within AI development.
Yandex’s move comes after it had to divest itself after Russia’s invasion of Ukraine, selling off its domestic business to its previous management in the country.Â
Many large tech companies are also making AI foundational to their products, most recently with Apple releasing its Apple Intelligence services as part of the forthcoming iOS 18 update.