zoom_in Click to enlarge

RecurrentGemma

▲ ▼

language

description RecurrentGemma Overview

RecurrentGemma is an open large language model developed by Google DeepMind and released in 2024. It is built upon the Griffin architecture, which integrates recurrent neural network layers with local attention mechanisms instead of relying solely on global attention. This structural shift significantly reduces the model's memory footprint and computational cost during inference, making it highly efficient for generating text. The model is intended for researchers and developers who require capable text generation on standard hardware with constrained memory resources.

help RecurrentGemma FAQ

What is RecurrentGemma?

RecurrentGemma is an open language-model family released by Google DeepMind in 2024. It is based on the Griffin architecture rather than using a conventional transformer with global attention at every layer.

What is the Griffin architecture used in RecurrentGemma?

Griffin combines gated recurrent layers with local sliding-window attention. This hybrid design lets RecurrentGemma process sequences using recurrent state while limiting attention to nearby tokens.

How is RecurrentGemma different from standard Gemma?

Standard Gemma models are transformer-based, while RecurrentGemma uses the Griffin hybrid architecture. Both are Google DeepMind open-weight language-model families, but their sequence-processing mechanisms are different.

What can RecurrentGemma be used for?

RecurrentGemma can be used for text generation and other language-model experiments on compatible hardware. Its recurrent design was intended to improve efficiency for some long-sequence workloads compared with relying entirely on global attention.

swap_horiz

Looking for RecurrentGemma alternatives? Compare top competitors ranked & scored

arrow_forward