郑州大学学报（工学版）2024，Vol.45Issue(3)：89-95,7.DOI:10.13705/j.issn.1671-6833.2024.03.002
基于MacBERT和R-Drop的地质命名实体识别
Geological Named Entity Recognition Based on MacBERT and R-Drop
摘要
Abstract
The commonly used deep learning methods based on BERT pre-trained model in geological named entity recognition were character-based approaches,and could not utilize word-level information.Additionally,the drop-out mechanism in neural networks might cause inconsistency between the training and inference stage.To address this issue,a geological named entity recognition model MBCR based on MacBERT and R-Drop was proposed.First-ly,MacBERT was used to learn text feature representations,which could fully utilize character and word informa-tion.Then,BiGRU was employed to encode context features,effectively extracting complete semantic information.Subsequently,CRF was adopted to capture dependencies between labels and generate the optimal label sequence.Moreover,R-Drop was introduced during the training process to further enhance the model's generalization capabili-ties.Compared with BiLSTM-CRF,BERT-BiLSTM-CRF,and other models,the proposed MBCR model improved the F1-score on the NERdata dataset by 2.08-4.62 percentage points and on the Boson dataset by 1.26-17.54 percentage points.关键词
命名实体识别/地质/MacBERT/BiGRU/R-DropKey words
named entity recognition/geology/MacBERT/BiGRU/R-Drop分类
信息技术与安全科学引用本文复制引用
刘昕,徐洪珍,刘爱华,邓德军..基于MacBERT和R-Drop的地质命名实体识别[J].郑州大学学报（工学版）,2024,45(3):89-95,7.基金项目
国家自然科学基金资助项目(62066003) （62066003）
江西省教育厅科技计划项目(GJJ160554) （GJJ160554）
江西省抚州市人才计划项目(2021ED008) （2021ED008）
江西省网络空间安全智能感知重点实验室室开放项目(JKLCIP202202) （JKLCIP202202）
