zoom_in Click to enlarge

XLNet

▲ ▼

description XLNet Overview

XLNet is an autoregressive language model introduced in 2019 by researchers from Google Brain and Carnegie Mellon University. It uses permutation language modeling, predicting tokens under varied factorization orders so that training can capture bidirectional context without masking input tokens in the manner used by BERT. Built with ideas from Transformer-XL, it was evaluated on language understanding, question answering, and document ranking tasks.

help XLNet FAQ

Who created the XLNet language model?

XLNet was introduced in 2019 through a collaboration between researchers from Google Brain and Carnegie Mellon University. It was proposed as an advanced alternative to the highly popular BERT model.

How does XLNet handle bidirectional context?

XLNet uses a permutation language modeling technique, which allows it to predict tokens under varied factorization orders. This means it can capture bidirectional context during training without needing to mask tokens like BERT does.

Is XLNet an autoregressive model?

Yes, XLNet is fundamentally an autoregressive language model, but it uses a generalized approach via permutation language modeling. This allows it to learn the context from all sides of a sequence, maximizing the expected likelihood of the text.

When was XLNet introduced to the AI community?

The XLNet model was introduced in 2019 via a paper that generated significant excitement in the natural language processing community. At the time, it achieved state-of-the-art results on several major NLP benchmarks.

swap_horiz

Looking for XLNet alternatives? Compare top competitors ranked & scored

arrow_forward