Alpaca

Stanford researchers fine-tuned Meta's LLaMA-7B using 52,000 instruction-following examples generated by GPT-3.5 (text-davinci-003), with the API costs totaling approximately $500. This demonstrated that effective instruction tuning could be achieved at a fraction of the cost of training a model from scratch.

LLaMA is Meta's foundational large language model family, while Alpaca is a fine-tuned version of LLaMA-7B specifically adapted for instruction-following tasks. Stanford created Alpaca by taking LLaMA-7B and training it on GPT-3.5-generated instruction data to make it behave more like a chatbot or assistant.

Alpaca weights were released by Stanford, but because they are derived from Meta's LLaMA, usage is subject to LLaMA's licensing restrictions, which were originally non-commercial. Researchers and developers should check the current terms on both the Stanford Alpaca GitHub repository and Meta's LLaMA license before use.

Stanford released Alpaca in March 2023 as part of a research effort to show that instruction-following capabilities could be achieved cheaply. It quickly became one of the most discussed open-weight instruction-tuned models in the early wave of post-LLaMA fine-tuning research.