zoom_in Click to enlarge

Gopher

▲ ▼

description Gopher Overview

Gopher is a 280-billion-parameter language model developed by DeepMind and detailed in a December 2021 research paper. The model was part of a family of language models ranging from 44 million to 280 billion parameters, with Gopher being the largest variant. At the time of release, Gopher achieved state-of-the-art results on many language understanding benchmarks, particularly in specialized domains like medicine and science. DeepMind's research with Gopher provided valuable insights about training efficiency, leading to their subsequent Chinchilla model which demonstrated similar performance with significantly fewer parameters.

help Gopher FAQ

How many parameters does DeepMind's Gopher model have?

Gopher is a 280-billion-parameter language model released by DeepMind in late 2021. It was one of the largest Transformer-based models at the time and was designed to compete with OpenAI's 175-billion-parameter GPT-3.

What did DeepMind's Chinchilla paper reveal about Gopher's training?

DeepMind's 2022 Chinchilla study found that Gopher was significantly undertrained for its parameter count, showing that most models at the time used too few data tokens relative to their size. The 70-billion-parameter Chinchilla, trained on far more data, actually outperformed the 280-billion-parameter Gopher on most benchmarks.

Did DeepMind release smaller Gopher model variants?

Yes, DeepMind published a family of Transformer language models ranging from 44 million to 280 billion parameters alongside Gopher. These smaller variants were used to study how model performance scales with size across different language understanding tasks.

swap_horiz

Looking for Gopher alternatives? Compare top competitors ranked & scored

arrow_forward