language-perceiver

Papers using or citing this model 3

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 122.9K citations · 2018 · North American Chapter of the Association for Computational Linguistics · arXiv

  We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional...

• Perceiver IO: A General Architecture for Structured Inputs & Outputs 945 citations · 2021 · arXiv (Cornell University) · arXiv

  A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible. Current architectures, however, cannot be applied beyond a small set of stereotyped settings, as they...

• Byte Pair Encoding is Suboptimal for Language Model Pretraining 347 citations · 2020 · arXiv

  The success of pretrained transformer language models (LMs) in natural language processing has led to a wide range of pretraining setups. In particular, these models employ a variety of subword tokenization methods, most notably...