zoom_in Click to enlarge

PaLM

▲ ▼

description PaLM Overview

PaLM (Pathways Language Model) is a 540-billion-parameter language model developed by Google and announced in April 2022. The model was trained using Google's Pathways system, a new ML architecture designed to efficiently train models across thousands of accelerator chips. PaLM demonstrated strong few-shot performance across reasoning tasks, code generation, and multilingual understanding, including breakthrough capabilities in chain-of-thought reasoning where models explicitly work through intermediate steps. Google later released variants including PaLM-E for robotics applications and PaLM 2, which powers various Google products including Bard.

help PaLM FAQ

How many parameters does Google's PaLM model have?

PaLM was released by Google in 2022 with 540 billion parameters, making it one of the largest dense language models at the time of its announcement. The model used Google's Pathways training system to achieve strong few-shot performance across reasoning, multilingual, and code-generation tasks.

Can I access Google PaLM through an API?

Google made PaLM and its successor PaLM 2 available through the Vertex AI platform and the Generative Language API. Developers could also experiment with the model through Google's MakerSuite tooling, which was later rebranded as Google AI Studio.

What hardware did Google use to train PaLM?

PaLM was trained using 6,144 TPU v4 chips across two Cloud TPU v4 Pods, demonstrating the scalability of the Pathways system. This large-scale distributed training setup was notable for efficiently handling the 540-billion-parameter model.

How did PaLM perform on chain-of-thought reasoning compared to GPT-3?

PaLM achieved significant improvements over GPT-3 on benchmarks like GSM8K for grade-school math problems and BIG-bench tasks, particularly when using chain-of-thought prompting. At 540 billion parameters, PaLM was roughly three times the size of GPT-3's 175 billion parameters.

swap_horiz

Looking for PaLM alternatives? Compare top competitors ranked & scored

arrow_forward