Alpaca AI: Stanford researchers clone ChatGPT AI for just $600
They have also released the tools needed for people to train their own AI.
Researchers at Stanford’s Center for Research on Foundation Models (CRFM) have unveiled an artificial intelligence (AI) model that works much like the famous ChatGPT but cost them only $600 to train. The researchers said that they hadn’t optimized their process and future models could be trained for lesser.
Until OpenAI’s ChatGPT was launched to the public in November last year, Large Language Models (LLMs) were largely a topic of discussion among AI researchers. The company has spent millions of dollars training them and making sure that they provided responses to human queries in the way another human would respond.
Microsoft’s investments into OpenAI to the tune of billions suggest that training AI models is hard. But researchers at Stanford seem to have done it at a modest budget that could allow AI companies to be spun out of garages.
How Stanford trained AI for minimal costs
A critical component of this achievement was LLaMA 7B, an open-source language model, which the researchers got access to. Interestingly, this model comes from Meta, Mark Zuckerberg’s company, and is one of the smallest and most low-cost language models available today.
Trained on trillion tokens, the language model has some capabilities that are equipped with but nowhere close to the levels that we have seen with ChatGPT. The researchers then turned to GPT, the AI behind the chatbot, and used an Application Programming Interface (API) to use 175 human-written instruction/output pairs to generate more in the same style and format.
Generating 20 such statements at a time, the researchers amassed 52,000 sample conversations in very little time, which cost them $500. This dataset was then used to post-train the LLaMa model. Turning to eight 80-GB A100 cloud processing computers, the researchers completed this task in just three hours having spent less than $100.
The trained model, dubbed, Alpaca was then tested against ChatGPT itself in various domains and beat GPT in its own game. The researchers go on to state that their process wasn’t really optimized and they could have gotten better results, had they used GPT-4, the latest version of the AI.
The researchers have now released the 52,000 questions that were used in the research alongside the code that was used to generate them, allowing many others to repeat the process and replicate the results. The AI and its responses are not subject to any guardrails that OpenAI has ensured in its chatbot, so one can expect some really nasty replies.
But what if someone does not really care what the chatbot says and about whom and wants it to work without filters? There are Open AI’s user terms that prevent users from building competing AI and LLaMA access available only for researchers. But beyond that, there is hardly anything that could prevent one from developing their pet AI.
Guess, this is where regulation comes in. AI is racing very fast and lawmakers are really to catch up soon, else AI will write it itself.
SHOW COMMENT ()
