사전 학습 언어모델(BERT)의 자동사에 대한 내적표현 탐색

초록

This study investigates how the BERT language model captures the relationship between the semantic structure of intransitive verbs and the animacy of their subjects through probing techniques. To achieve this, intransitive verbs were categorized into Core, Less Core, and Periphery based on their semantic centrality. The internal representations were then analyzed when each verb type was combined with wuther animate or inanimate subjects. By examining classification performance across verb types using both token-based and sentence-based embeddings from BERT, the results showed the highest accuracy and clearest distinctions in the model’s intermediate layers (layers 6 to 9). Furthermore, the findings indicate that BERT’s internal representations of intransitive verb types exhibit a non-linear structure. These results demonstrate that pre-trained language models (PLMs) like BERT effectively encode the semantic structure of intransitive verbs and subject animacy in a way that aligns with human linguistic intuition. This study provides a theoretical foundation for the future development of language models that incorporate human language processing and cognitive plausibility.

키워드