zoom_in Click to enlarge

MPT-7B

▲ ▼

description MPT-7B Overview

MPT-7B is a 7-billion-parameter transformer model developed by MosaicML and released in May 2023. Trained on 1 trillion tokens of text and code, the model was released under the Apache 2.0 license, allowing for commercial use. It was designed to demonstrate that capable open-source large language models could be trained efficiently. MosaicML also released several variants, including versions fine-tuned for instruction following and others capable of handling extended context windows of up to 84,000 tokens.

help MPT-7B FAQ

What is MPT-7B?

MPT-7B is a transformer language model with 7 billion parameters, developed by MosaicML. It was released in May 2023 as an open model intended to demonstrate capable open-source language technology.

How much training data was used for MPT-7B?

MPT-7B was trained on 1 trillion tokens of text and code. That training scale is notable because the model has 7 billion parameters while still targeting broad language and programming tasks.

Can a company use MPT-7B commercially?

MPT-7B was released under the Apache 2.0 license, which permits commercial use subject to the license terms. The model was released by MosaicML in May 2023.

What are the MPT-7B-Instruct and StoryWriter variants?

MPT-7B was released in variants aimed at different uses, including instruction-following and long-form writing. The StoryWriter line is associated with extended-context generation, while the base model contains 7 billion parameters.

swap_horiz

Looking for MPT-7B alternatives? Compare top competitors ranked & scored

arrow_forward