Modelling‚ Visualising and Summarising Documents with a Single Convolutional Neural Network
Capturing the compositional process which maps the meaning of words to that of documents is a central challenge for researchers in Natural Language Processing and Information Retrieval. We introduce a model that is able to represent the meaning of documents by embedding them in a low dimensional vec...
| Main Authors: | , , , , |
|---|---|
| Format: | Report |
| Published: |
University of Oxford
2014
|
| _version_ | 1826284733448847360 |
|---|---|
| author | Denil, M Demiraj, A Kalchbrenner, N Blunsom, P de Freitas, N |
| author_facet | Denil, M Demiraj, A Kalchbrenner, N Blunsom, P de Freitas, N |
| author_sort | Denil, M |
| collection | OXFORD |
| description | Capturing the compositional process which maps the meaning of words to that of documents is a central challenge for researchers in Natural Language Processing and Information Retrieval. We introduce a model that is able to represent the meaning of documents by embedding them in a low dimensional vector space, while preserving distinctions of word and sentence order crucial for capturing nuanced semantics. Our model is based on an extended Dynamic Convolution Neural Network, which learns convolution filters at both the sentence and document level, hierarchically learning to capture and compose low level lexical features into high level semantic concepts. We demonstrate the effectiveness of this model on a range of document modelling tasks, achieving strong results with no feature engineering and with a more compact model. Inspired by recent advances in visualising deep convolution networks for computer vision, we present a novel visualisation technique for our document networks which not only provides insight into their learning process, but also can be interpreted to produce a compelling automatic summarisation system for texts. |
| first_indexed | 2024-03-07T01:18:17Z |
| format | Report |
| id | oxford-uuid:8f765144-c11c-4d9d-b30f-230c40addfa0 |
| institution | University of Oxford |
| last_indexed | 2024-03-07T01:18:17Z |
| publishDate | 2014 |
| publisher | University of Oxford |
| record_format | dspace |
| spelling | oxford-uuid:8f765144-c11c-4d9d-b30f-230c40addfa02022-03-26T23:04:25ZModelling‚ Visualising and Summarising Documents with a Single Convolutional Neural NetworkReporthttp://purl.org/coar/resource_type/c_93fcuuid:8f765144-c11c-4d9d-b30f-230c40addfa0Department of Computer ScienceUniversity of Oxford2014Denil, MDemiraj, AKalchbrenner, NBlunsom, Pde Freitas, NCapturing the compositional process which maps the meaning of words to that of documents is a central challenge for researchers in Natural Language Processing and Information Retrieval. We introduce a model that is able to represent the meaning of documents by embedding them in a low dimensional vector space, while preserving distinctions of word and sentence order crucial for capturing nuanced semantics. Our model is based on an extended Dynamic Convolution Neural Network, which learns convolution filters at both the sentence and document level, hierarchically learning to capture and compose low level lexical features into high level semantic concepts. We demonstrate the effectiveness of this model on a range of document modelling tasks, achieving strong results with no feature engineering and with a more compact model. Inspired by recent advances in visualising deep convolution networks for computer vision, we present a novel visualisation technique for our document networks which not only provides insight into their learning process, but also can be interpreted to produce a compelling automatic summarisation system for texts. |
| spellingShingle | Denil, M Demiraj, A Kalchbrenner, N Blunsom, P de Freitas, N Modelling‚ Visualising and Summarising Documents with a Single Convolutional Neural Network |
| title | Modelling‚ Visualising and Summarising Documents with a Single Convolutional Neural Network |
| title_full | Modelling‚ Visualising and Summarising Documents with a Single Convolutional Neural Network |
| title_fullStr | Modelling‚ Visualising and Summarising Documents with a Single Convolutional Neural Network |
| title_full_unstemmed | Modelling‚ Visualising and Summarising Documents with a Single Convolutional Neural Network |
| title_short | Modelling‚ Visualising and Summarising Documents with a Single Convolutional Neural Network |
| title_sort | modelling visualising and summarising documents with a single convolutional neural network |
| work_keys_str_mv | AT denilm modellingvisualisingandsummarisingdocumentswithasingleconvolutionalneuralnetwork AT demiraja modellingvisualisingandsummarisingdocumentswithasingleconvolutionalneuralnetwork AT kalchbrennern modellingvisualisingandsummarisingdocumentswithasingleconvolutionalneuralnetwork AT blunsomp modellingvisualisingandsummarisingdocumentswithasingleconvolutionalneuralnetwork AT defreitasn modellingvisualisingandsummarisingdocumentswithasingleconvolutionalneuralnetwork |