OpenAI spent $80M to $100M training GPT-4; Chinese firm claims it trained its rival AI model for $3 million using just 2,000 GPUs

Date:
Mon, 02 Dec 2024 18:32:00 +0000

Description:
Chinese firm claims it trained rival GPT-4 model for $3 million using just 2,000 GPUs.

FULL STORY ======================================================================01.ai trained an AI model for $3 million using 2000 unnamed GPUS Efficient engineering allows 01.ai to compete globally, company claims 01.ai reduced inference costs to 10 cents per million tokens

Tech companies in China face a number of challenges due to the American
export ban, which restricts access to advanced hardware from US
manufacturers.

This includes cutting-edge GPUs from Nvidia, critical for training
large-scale AI models, forcing Chinese firms to rely on older or less efficient alternatives, making it difficult to compete globally in the
rapidly evolving AI industry.

However, as weve seen time and again , these seemingly insurmountable challenges are increasingly being overcome through innovative solutions and Chinese ingenuity. Kai-Fu Lee, founder and CEO of 01.ai, recently revealed that his team successfully trained its high-performing model, Yi-Lightning, with a budget of just $3 million and 2,000 GPUs. In comparison, OpenAI reportedly spent $80-$100 million to train GPT-4 and is rumored to have allocated up to $1 billion for GPT-5. Making inference fast too

The thing that shocks my friends in the Silicon Valley is not just our performance, but that we trained the model with only $3 million," Lee said (via @tsarnick ).

"We believe in scaling law, but when you do excellent detailed engineering,
it is not the case you have to spend a billion dollars to train a great
model. As a company in China, first, we have limited access to GPUs due to
the US regulations, and secondly, Chinese companies are not valued what the American companies are. So when we have less money and difficulty to get
GPUs, I truly believe that necessity is the mother of invention."

Lee explained the companys innovations include reducing computational bottlenecks, developing multi-layer caching, and designing a specialized inference engine. These advancements, he claims, result in more efficient memory usage and optimized training processes.

When we only have 2,000 GPUs, the team has to figure out how to use it,
Kai-Fu Lee said, without disclosing the type of GPUs used. I, as the CEO,
have to figure out how to prioritize it, and then not only do we have to make training fast, we have to make inference fast... The bottom line is our inference cost is 10 cents per million tokens.

For context, thats about 1/30th of the typical rate charged by comparable models, highlighting the efficiency of 01.ai's approach.

Some people may be skeptical about the claims that you can train an AI model with limited resources and "excellent engineering", but according to UC Berkeleys LMSIS, Yi-Lightning is ranked sixth globally in performance, suggesting that however it has done it, 01.ai has indeed found a way to be competitive with a minuscule budget and limited GPU access. You might also like China wants to be the global AI leader in personal computing AMD
launches a new single-slot GPU for data centers This startup may be the
answer to running AI training on any GPU



======================================================================
Link to news story: https://www.techradar.com/pro/openai-spent-usd80m-to-usd100m-training-gpt-4-ch inese-firm-claims-it-trained-its-rival-ai-model-for-usd3-million-using-just-2- 000-gpus


--- Mystic BBS v1.12 A47 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)