Named entity recognition in electronic health records using transfer learning bootstrapped neural networks
Neural networks (NNs) have become the state of the art in many machine learning applications, such as image, sound (LeCun et al., 2015) and natural language processing (Young et al., 2017; Linggard et al., 2012). However, the success of NNs remains dependent on the availability of large labelled dat...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Elsevier
2019
|
| _version_ | 1797063179373641728 |
|---|---|
| author | Gligic, L Kormilitzin, A Goldberg, P Nevado-Holgado, AJ |
| author_facet | Gligic, L Kormilitzin, A Goldberg, P Nevado-Holgado, AJ |
| author_sort | Gligic, L |
| collection | OXFORD |
| description | Neural networks (NNs) have become the state of the art in many machine learning applications, such as image, sound (LeCun et al., 2015) and natural language processing (Young et al., 2017; Linggard et al., 2012). However, the success of NNs remains dependent on the availability of large labelled datasets, such as in the case of electronic health records (EHRs). With scarce data, NNs are unlikely to be able to extract this hidden information with practical accuracy. In this study, we develop an approach that solves these problems for named entity recognition, obtaining 94.6 F1 score in I2B2 2009 Medical Extraction Challenge (Uzuner et al., 2010), 4.3 above the architecture that won the competition. To achieve this, we bootstrap our NN models through transfer learning by pretraining word embeddings on a secondary task performed on a large pool of unannotated EHRs and using the output embeddings as a foundation of a range of NN architectures. Beyond the official I2B2 challenge, we further achieve 82.4 F1 on extracting relationships between medical terms using attention-based seq2seq models bootstrapped in the same manner. |
| first_indexed | 2024-03-06T20:56:06Z |
| format | Journal article |
| id | oxford-uuid:394e89cd-7497-452b-86c7-1aa2c666d772 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T20:56:06Z |
| publishDate | 2019 |
| publisher | Elsevier |
| record_format | dspace |
| spelling | oxford-uuid:394e89cd-7497-452b-86c7-1aa2c666d7722022-03-26T13:54:49ZNamed entity recognition in electronic health records using transfer learning bootstrapped neural networksJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:394e89cd-7497-452b-86c7-1aa2c666d772EnglishSymplectic ElementsElsevier2019Gligic, LKormilitzin, AGoldberg, PNevado-Holgado, AJNeural networks (NNs) have become the state of the art in many machine learning applications, such as image, sound (LeCun et al., 2015) and natural language processing (Young et al., 2017; Linggard et al., 2012). However, the success of NNs remains dependent on the availability of large labelled datasets, such as in the case of electronic health records (EHRs). With scarce data, NNs are unlikely to be able to extract this hidden information with practical accuracy. In this study, we develop an approach that solves these problems for named entity recognition, obtaining 94.6 F1 score in I2B2 2009 Medical Extraction Challenge (Uzuner et al., 2010), 4.3 above the architecture that won the competition. To achieve this, we bootstrap our NN models through transfer learning by pretraining word embeddings on a secondary task performed on a large pool of unannotated EHRs and using the output embeddings as a foundation of a range of NN architectures. Beyond the official I2B2 challenge, we further achieve 82.4 F1 on extracting relationships between medical terms using attention-based seq2seq models bootstrapped in the same manner. |
| spellingShingle | Gligic, L Kormilitzin, A Goldberg, P Nevado-Holgado, AJ Named entity recognition in electronic health records using transfer learning bootstrapped neural networks |
| title | Named entity recognition in electronic health records using transfer learning bootstrapped neural networks |
| title_full | Named entity recognition in electronic health records using transfer learning bootstrapped neural networks |
| title_fullStr | Named entity recognition in electronic health records using transfer learning bootstrapped neural networks |
| title_full_unstemmed | Named entity recognition in electronic health records using transfer learning bootstrapped neural networks |
| title_short | Named entity recognition in electronic health records using transfer learning bootstrapped neural networks |
| title_sort | named entity recognition in electronic health records using transfer learning bootstrapped neural networks |
| work_keys_str_mv | AT gligicl namedentityrecognitioninelectronichealthrecordsusingtransferlearningbootstrappedneuralnetworks AT kormilitzina namedentityrecognitioninelectronichealthrecordsusingtransferlearningbootstrappedneuralnetworks AT goldbergp namedentityrecognitioninelectronichealthrecordsusingtransferlearningbootstrappedneuralnetworks AT nevadoholgadoaj namedentityrecognitioninelectronichealthrecordsusingtransferlearningbootstrappedneuralnetworks |