A General Model for Side Information in Neural Networks
We investigate the utility of side information in the context of machine learning and, in particular, in supervised neural networks. Side information can be viewed as expert knowledge, additional to the input, that may come from a knowledge base. Unlike other approaches, our formalism can be used by...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/16/11/526 |
| _version_ | 1797460477805068288 |
|---|---|
| author | Tameem Adel Mark Levene |
| author_facet | Tameem Adel Mark Levene |
| author_sort | Tameem Adel |
| collection | DOAJ |
| description | We investigate the utility of side information in the context of machine learning and, in particular, in supervised neural networks. Side information can be viewed as expert knowledge, additional to the input, that may come from a knowledge base. Unlike other approaches, our formalism can be used by a machine learning algorithm not only during training but also during testing. Moreover, the proposed approach is flexible as it caters for different formats of side information, and we do not constrain the side information to be fed into the input layer of the network. A formalism is presented based on the difference between the neural network loss without and with side information, stating that it is useful when adding side information reduces the loss during the test phase. As a proof of concept we provide experimental results for two datasets, the MNIST dataset of handwritten digits and the House Price prediction dataset. For the experiments we used feedforward neural networks containing two hidden layers, as well as a softmax output layer. For both datasets, side information is shown to be useful in that it improves the classification accuracy significantly. |
| first_indexed | 2024-03-09T17:05:39Z |
| format | Article |
| id | doaj.art-a1aa789fea194a83bb0fe1e3f1950972 |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-03-09T17:05:39Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-a1aa789fea194a83bb0fe1e3f19509722023-11-24T14:24:27ZengMDPI AGAlgorithms1999-48932023-11-01161152610.3390/a16110526A General Model for Side Information in Neural NetworksTameem Adel0Mark Levene1Department of Data Science, National Physical Laboratory (NPL), Hampton Road, Teddington TW11 0LW, UKDepartment of Data Science, National Physical Laboratory (NPL), Hampton Road, Teddington TW11 0LW, UKWe investigate the utility of side information in the context of machine learning and, in particular, in supervised neural networks. Side information can be viewed as expert knowledge, additional to the input, that may come from a knowledge base. Unlike other approaches, our formalism can be used by a machine learning algorithm not only during training but also during testing. Moreover, the proposed approach is flexible as it caters for different formats of side information, and we do not constrain the side information to be fed into the input layer of the network. A formalism is presented based on the difference between the neural network loss without and with side information, stating that it is useful when adding side information reduces the loss during the test phase. As a proof of concept we provide experimental results for two datasets, the MNIST dataset of handwritten digits and the House Price prediction dataset. For the experiments we used feedforward neural networks containing two hidden layers, as well as a softmax output layer. For both datasets, side information is shown to be useful in that it improves the classification accuracy significantly.https://www.mdpi.com/1999-4893/16/11/526side informationneural networksknowledge baseexplainable machine learning |
| spellingShingle | Tameem Adel Mark Levene A General Model for Side Information in Neural Networks Algorithms side information neural networks knowledge base explainable machine learning |
| title | A General Model for Side Information in Neural Networks |
| title_full | A General Model for Side Information in Neural Networks |
| title_fullStr | A General Model for Side Information in Neural Networks |
| title_full_unstemmed | A General Model for Side Information in Neural Networks |
| title_short | A General Model for Side Information in Neural Networks |
| title_sort | general model for side information in neural networks |
| topic | side information neural networks knowledge base explainable machine learning |
| url | https://www.mdpi.com/1999-4893/16/11/526 |
| work_keys_str_mv | AT tameemadel ageneralmodelforsideinformationinneuralnetworks AT marklevene ageneralmodelforsideinformationinneuralnetworks AT tameemadel generalmodelforsideinformationinneuralnetworks AT marklevene generalmodelforsideinformationinneuralnetworks |