Measuring and Improving Consistency in Pretrained Language Models

Measuring and Improving Consistency in Pretrained Language Models

AbstractConsistency of a model—that is, the invariance of its behavior under meaning-preserving alternations in its input—is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect...

Full description

Bibliographic Details
Main Authors:	Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
Format:	Article
Language:	English
Published:	The MIT Press 2021-01-01
Series:	Transactions of the Association for Computational Linguistics
Online Access:	https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00410/107384/Measuring-and-Improving-Consistency-in-Pretrained

Similar Items

Erratum: Measuring and Improving Consistency in Pretrained Language Models
by: Yanai Elazar, et al.
Published: (2022-01-01)

Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals
by: Yanai Elazar, et al.
Published: (2021-01-01)

Geographic Adaptation of Pretrained Language Models
by: Valentin Hofmann, et al.
Published: (2024-04-01)

Explaining pretrained language models' understanding of linguistic structures using construction grammar
by: Leonie Weissweiler, et al.
Published: (2023-10-01)

Geographic adaptation of pretrained language models
by: Hofmann, V, et al.
Published: (2024)

Where’s My Head? Definition, Data Set, and Models for Numeric Fused-Head Identification and Resolution
by: Elazar, Yanai, et al.
Published: (2019-11-01)

oLMpics-On What Language Model Pre-training Captures
by: Alon Talmor, et al.
Published: (2021-03-01)

DagoBERT: generating derivational morphology with a pretrained language model
by: Hofmann, V, et al.
Published: (2020)

Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes
by: Ofer Sabo, et al.
Published: (2021-01-01)

Text-based NP Enrichment
by: Yanai Elazar, et al.
Published: (2022-08-01)

Survey of Applications of Pretrained Language Models
by: SUN Kaili, LUO Xudong , Michael Y.LUO
Published: (2023-01-01)

Keeping Models Consistent between Pretraining and Translation for Low-Resource Neural Machine Translation
by: Wenbo Zhang, et al.
Published: (2020-11-01)

Extracting event knowledge from pretrained language models
by: Ong, Claudia Beth
Published: (2023)

Data-efficient domain adaptation for pretrained language models
by: Guo, Xu
Published: (2023)

Sub-Character Tokenization for Chinese Pretrained Language Models
by: Chenglei Si, et al.
Published: (2023-05-01)

To pretrain or not? A systematic analysis of the benefits of pretraining in diabetic retinopathy.
by: Vignesh Srinivasan, et al.
Published: (2022-01-01)

To pretrain or not? A systematic analysis of the benefits of pretraining in diabetic retinopathy
by: Vignesh Srinivasan, et al.
Published: (2022-01-01)

Half-Context Language Models
by: Hinrich Schütze, et al.
Published: (2021-03-01)

University Student Dropout Prediction Using Pretrained Language Models
by: Hyun-Sik Won, et al.
Published: (2023-06-01)

Sequence-to-sequence pretraining for a less-resourced Slovenian language
by: Matej Ulčar, et al.
Published: (2023-03-01)

Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation
by: Mathieu De Coster, et al.
Published: (2022-04-01)

Assessing the accuracy and consistency of generative pretrained transformers in assigning Eastern Cooperative Oncology Group performance status
by: Chun En Yau, et al.
Published: (2024-09-01)

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval
by: Ohad Rubin, et al.
Published: (2024-10-01)

Leveraging Text-to-Text Pretrained Language Models for Question Answering in Chemistry
by: Dan Tran, et al.
Published: (2024-03-01)

Pretrained Natural Language Processing Model for Intent Recognition (BERT-IR)
by: Vasima Khan, et al.
Published: (2021-11-01)

Improving prediction performance of general protein language model by domain-adaptive pretraining on DNA-binding protein
by: Wenwu Zeng, et al.
Published: (2024-09-01)

Pretraining Enhanced RNN Transducer
by: Junyu Lu, et al.
Published: (2024-12-01)

Model-driven smart contract generation leveraging pretrained large language models
by: Jiang, Qinbo
Published: (2024)

Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images
by: Lu, Ming Yang (Max)
Published: (2023)

An embarrassingly simple method to mitigate undesirable properties of pretrained language model tokenizers
by: Hofmann, V, et al.
Published: (2022)

Constructing Chinese taxonomy trees from understanding and generative pretrained language models
by: Jianyu Guo, et al.
Published: (2024-10-01)

Error Analysis of Pretrained Language Models (PLMs) in English-to-Arabic Machine Translation
by: Hend Al-Khalifa, et al.
Published: (2024-02-01)

Exploring the Data Efficiency of Cross-Lingual Post-Training in Pretrained Language Models
by: Chanhee Lee, et al.
Published: (2021-02-01)

Arabic Grammatical Error Detection Using Transformers-based Pretrained Language Models
by: AlOyaynaa Sarah, et al.
Published: (2023-01-01)

Injecting User Identity Into Pretrained Language Models for Document-Level Sentiment Classification
by: Xinlei Cao, et al.
Published: (2022-01-01)

Incorporating Typological Information into Pretrained Word Representations for Improved Performance
by: Vighnesh, Subramaniam
Published: (2022)

Critical Analysis of Deconfounded Pretraining to Improve Visio-Linguistic Models
by: Nathan Cornille, et al.
Published: (2022-03-01)

Not wacky vs. definitely wacky: a study of scalar adverbs in pretrained language models
by: Lorge, I, et al.
Published: (2023)

Validating pretrained language models for content quality classification with semantic-preserving metamorphic relations
by: Pak Yuen Patrick Chan, et al.
Published: (2024-12-01)

OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue
by: Zhi Chen, et al.
Published: (2023-12-01)