Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network
The main problem for Supervised Multi-layer Neural Network (SMNN) model lies in finding the suitable weights during training in order to improve training time as well as achieve higher accuracy. The important issue in the training process of the existing SMNN model is initialization of the weight...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English English |
| Published: |
2009
|
| Subjects: | |
| Online Access: | http://psasir.upm.edu.my/id/eprint/11929/1/FSKTM_2009_10_A.pdf |
| _version_ | 1796968139559272448 |
|---|---|
| author | Asadi, Roya |
| author_facet | Asadi, Roya |
| author_sort | Asadi, Roya |
| collection | UPM |
| description | The main problem for Supervised Multi-layer Neural Network (SMNN) model lies in
finding the suitable weights during training in order to improve training time as well as
achieve higher accuracy. The important issue in the training process of the existing SMNN
model is initialization of the weights. However, this process is random and creates the
paradox of low accuracy and high training time.
In this study, a Multi-layer Feed Forward Neural Network (MFFNN) model for
classification problem is proposed. It consists of a new preprocessing technique which
combines data preprocessing and pre-training that offer a number of advantages; training
cycle, gradient of mean square error function, and updating weights are not needed in this
model. The proposed technique is Weight Linear Analysis (WLA) based on mathematical,statistical and physical principles for generating real weights by using input values. WLA
applies global mean and vectors torque formula to solve the problem. We perform data
preprocessing for generating normalized input values and then applying them by a pretraining
technique in order to obtain the real weights. The normalized input values and real
weights are applied to the MFFNN model in one epoch without training cycle. In MFFNN
model, thresholds of training set and test set are computed by using input values and real
weights. In training set each instance has one special threshold and class label. In test set
the threshold of each instance will be compared with the range of thresholds of training set
and the class label of each instance will be predicted.
To evaluate the performance of the proposed MFFNN model, a series of experiment on
XOR problem and two datasets, which are SPECT Heart and SPECTF Heart was
implemented. As quoted by literature, these two datasets are difficult for classification and
most of the conventional methods do not process well on these datasets. For experiment
result, Standard Back Propagation Network (BPN) as SMNN model is considered. SBPN
is changed to MFFNN model by using WLA technique. Accuracy of MFFNN model using
WLA is compared with several strong classification models and SBPN using best and
latest pre-training techniques. Our results, however, show that the proposed model has
been given high accuracy in one epoch without training cycle. The accuracies of 94% for
SPECTF Heart and 92% for SPECT Heart which are the best results. |
| first_indexed | 2024-03-06T07:24:03Z |
| format | Thesis |
| id | upm.eprints-11929 |
| institution | Universiti Putra Malaysia |
| language | English English |
| last_indexed | 2024-03-06T07:24:03Z |
| publishDate | 2009 |
| record_format | dspace |
| spelling | upm.eprints-119292013-05-27T07:50:21Z http://psasir.upm.edu.my/id/eprint/11929/ Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network Asadi, Roya The main problem for Supervised Multi-layer Neural Network (SMNN) model lies in finding the suitable weights during training in order to improve training time as well as achieve higher accuracy. The important issue in the training process of the existing SMNN model is initialization of the weights. However, this process is random and creates the paradox of low accuracy and high training time. In this study, a Multi-layer Feed Forward Neural Network (MFFNN) model for classification problem is proposed. It consists of a new preprocessing technique which combines data preprocessing and pre-training that offer a number of advantages; training cycle, gradient of mean square error function, and updating weights are not needed in this model. The proposed technique is Weight Linear Analysis (WLA) based on mathematical,statistical and physical principles for generating real weights by using input values. WLA applies global mean and vectors torque formula to solve the problem. We perform data preprocessing for generating normalized input values and then applying them by a pretraining technique in order to obtain the real weights. The normalized input values and real weights are applied to the MFFNN model in one epoch without training cycle. In MFFNN model, thresholds of training set and test set are computed by using input values and real weights. In training set each instance has one special threshold and class label. In test set the threshold of each instance will be compared with the range of thresholds of training set and the class label of each instance will be predicted. To evaluate the performance of the proposed MFFNN model, a series of experiment on XOR problem and two datasets, which are SPECT Heart and SPECTF Heart was implemented. As quoted by literature, these two datasets are difficult for classification and most of the conventional methods do not process well on these datasets. For experiment result, Standard Back Propagation Network (BPN) as SMNN model is considered. SBPN is changed to MFFNN model by using WLA technique. Accuracy of MFFNN model using WLA is compared with several strong classification models and SBPN using best and latest pre-training techniques. Our results, however, show that the proposed model has been given high accuracy in one epoch without training cycle. The accuracies of 94% for SPECTF Heart and 92% for SPECT Heart which are the best results. 2009-12 Thesis NonPeerReviewed application/pdf en http://psasir.upm.edu.my/id/eprint/11929/1/FSKTM_2009_10_A.pdf Asadi, Roya (2009) Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network. Masters thesis, Universiti Putra Malaysia. Neural networks (Computer science) English |
| spellingShingle | Neural networks (Computer science) Asadi, Roya Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network |
| title | Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network
|
| title_full | Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network
|
| title_fullStr | Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network
|
| title_full_unstemmed | Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network
|
| title_short | Preprocessing And Pretraining Of Multilayer Feed Forward Neural Network
|
| title_sort | preprocessing and pretraining of multilayer feed forward neural network |
| topic | Neural networks (Computer science) |
| url | http://psasir.upm.edu.my/id/eprint/11929/1/FSKTM_2009_10_A.pdf |
| work_keys_str_mv | AT asadiroya preprocessingandpretrainingofmultilayerfeedforwardneuralnetwork |