zoom_in Click to enlarge

DBRX

▲ ▼

language

description DBRX Overview

DBRX is an open-weight mixture-of-experts (MoE) language model released by Databricks in March 2024. It has 132 billion total parameters with 36 billion active per token, utilizing 16 expert groups in a fine-grained routing architecture. Databricks trained the model on its own infrastructure and reported that it surpassed other open MoE models on standard benchmarks at the time of release. The model weights were made available for research and commercial use under an open license.

help DBRX FAQ

Who created DBRX and when was it released?

DBRX was created by Databricks and released in March 2024 as an open-weight mixture-of-experts language model. Databricks trained the model on its own infrastructure, leveraging its expertise in data processing and enterprise machine learning platforms.

What architecture does DBRX use?

DBRX uses a mixture-of-experts (MoE) architecture with 132 billion total parameters and 36 billion active parameters per token. It employs 16 expert groups with a fine-grained routing system, meaning each token is processed by only a subset of the model's experts rather than all parameters simultaneously.

Is DBRX open source or open weights?

DBRX is released under an open-weight license, meaning the model weights are available for download and use. Databricks has made the model accessible through its own platform and through Hugging Face, allowing researchers and developers to build on it for various applications.

How does DBRX compare to Mixtral and other MoE models?

Databricks reported that DBRX outperformed Mixtral and other open-weight MoE models on standard language benchmarks at the time of its release. The model uses a more fine-grained expert routing system than Mixtral's 8-expert design, with 16 expert groups allowing for more specialized computation per token.

swap_horiz

Looking for DBRX alternatives? Compare top competitors ranked & scored

arrow_forward