The Impact of Normalization Techniques on Performance Backpropagation Networks

Neural networks (NN) are computational models with the capacity to learn, generalize and the most used are multi- layer perceptrons (MLP). Building successful NN applications depends on several aspects such as the process of acquiring, modeling and selecting the appropriate model. The data needs t...

Full description

Saved in:

Bibliographic Details
Main Author:	Norlida, Hassan
Format:	Thesis
Language:	eng eng
Published:	2004
Subjects:	QA76 Computer software
Online Access:	https://etd.uum.edu.my/1394/1/NORLIDA_BT._HASSAN.pdf https://etd.uum.edu.my/1394/2/1.NORLIDA_BT._HASSAN.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-uum-etd.1394
record_format	uketd_dc
institution	Universiti Utara Malaysia
collection	UUM ETD
language	eng eng
topic	QA76 Computer software
spellingShingle	QA76 Computer software Norlida, Hassan The Impact of Normalization Techniques on Performance Backpropagation Networks
description	Neural networks (NN) are computational models with the capacity to learn, generalize and the most used are multi- layer perceptrons (MLP). Building successful NN applications depends on several aspects such as the process of acquiring, modeling and selecting the appropriate model. The data needs to be transformed into a form that is acceptable as input to the MLP network. The transform data often determines the efficiency and possibly the accuracy of result from the network. This study explored several normalization techniques using backpropagation learning. The normalization techniques used in the experiments are Min-Max, Z-Score, Decimal Scaling, Sigmoidal, and Softmax or Logistic technique. To explore the impact of normalization technique on the performance on NN, medical datasets with Boolean target were preprocessed, trained, validated and tested using backpropagation learning algorithm. The criterion of choosing the best model is based on the highest percentage of correct prediction. Three preprocessing phase of building the NN models. The results of each normalization techniques are presented and compared with statistical approach. The results reveal that the utilization of different normalization techniques produces different NN performance. The experiments also indicate that all five normalization techniques of logistic regression achieve lower percentage of correct prediction than the results produced using NN. The findings will not only contribute towards enhancing the performance of backpropagation nets but it will also assist in making decision to the choice of normalization techniques to be applied to a particular dataset.
format	Thesis
qualification_name	masters
qualification_level	Master's degree
author	Norlida, Hassan
author_facet	Norlida, Hassan
author_sort	Norlida, Hassan
title	The Impact of Normalization Techniques on Performance Backpropagation Networks
title_short	The Impact of Normalization Techniques on Performance Backpropagation Networks
title_full	The Impact of Normalization Techniques on Performance Backpropagation Networks
title_fullStr	The Impact of Normalization Techniques on Performance Backpropagation Networks
title_full_unstemmed	The Impact of Normalization Techniques on Performance Backpropagation Networks
title_sort	impact of normalization techniques on performance backpropagation networks
granting_institution	Universiti Utara Malaysia
granting_department	Faculty of Information Technology
publishDate	2004
url	https://etd.uum.edu.my/1394/1/NORLIDA_BT._HASSAN.pdf https://etd.uum.edu.my/1394/2/1.NORLIDA_BT._HASSAN.pdf
_version_	1747827137596882944
spelling	my-uum-etd.13942013-07-24T12:11:46Z The Impact of Normalization Techniques on Performance Backpropagation Networks 2004 Norlida, Hassan Faculty of Information Technology Faculty of Information Technology QA76 Computer software Neural networks (NN) are computational models with the capacity to learn, generalize and the most used are multi- layer perceptrons (MLP). Building successful NN applications depends on several aspects such as the process of acquiring, modeling and selecting the appropriate model. The data needs to be transformed into a form that is acceptable as input to the MLP network. The transform data often determines the efficiency and possibly the accuracy of result from the network. This study explored several normalization techniques using backpropagation learning. The normalization techniques used in the experiments are Min-Max, Z-Score, Decimal Scaling, Sigmoidal, and Softmax or Logistic technique. To explore the impact of normalization technique on the performance on NN, medical datasets with Boolean target were preprocessed, trained, validated and tested using backpropagation learning algorithm. The criterion of choosing the best model is based on the highest percentage of correct prediction. Three preprocessing phase of building the NN models. The results of each normalization techniques are presented and compared with statistical approach. The results reveal that the utilization of different normalization techniques produces different NN performance. The experiments also indicate that all five normalization techniques of logistic regression achieve lower percentage of correct prediction than the results produced using NN. The findings will not only contribute towards enhancing the performance of backpropagation nets but it will also assist in making decision to the choice of normalization techniques to be applied to a particular dataset. 2004 Thesis https://etd.uum.edu.my/1394/ https://etd.uum.edu.my/1394/1/NORLIDA_BT._HASSAN.pdf application/pdf eng validuser https://etd.uum.edu.my/1394/2/1.NORLIDA_BT._HASSAN.pdf application/pdf eng public masters masters Universiti Utara Malaysia Abdullah, C.S., Siraj, F. and Abu Bakar, M.D. (2001) Design of Normal Concrete Mixes Using Neural Network Model. In Proceeding of the 2nd Conference on Information Technology in Asia, in Collaboration with Global Information & Telecommunication Institute. October 17-19,2001. Abidi, S. S. R., and Goh, A. (1998). Neural Network Based Forecasting of Bacteria-Antibiotic Interactions for Infectious Disease Control. In 9th World Congress on Medical Informatics (MedInfo798), Seoul August 18-22. Adali, S., Sapino, M. L., V. S., and Subrahmanian, V.S., (1999) A Multimedia Presentation Algebra. SIGMOD Conference 1999: 121-132. Ahmed, M. N., and Farag, A.A. (1998). Two-stage Neural Network for Medical Volume Segmentation. Accepted for Publication in the Journal of Pattern Recognition Letters, 1998. Alkharouf, N., Cervone, G., El-Askary, H., Tang, J. and Nefissi, S. (2002) A Multivariate Approach for the Investigation of Gene Behavior in Space. URL: http://www.scs.gmu.edu/~nalkhar3/CSI801/project.html. Accessed date: 6 December 2003. Armoni, A., (1998) Use of neural networks in medical1 diagnosis, MD Computing, Mac-Apr; 15(2): 100-4 Aronson, A. R., Bodenreider, O., Chang, H. C., Humphrey, S. M., Mork, J. G., Nelson, S. J., Rindflesch, T. C. and Wilbur, W. J. (1999) The Indexing Initiative, A Report to the Board of Scientific Counselors of the Lister Hill National Center for Biomedical Communications. Beale, R. and Edwards, A. D. N. (1992) Recognizing postures and gestures using neural networks. in R. Beale and J. Finlay (ed.) Neural Networks and Pattern Recognition in Human-Computer Interaction. New York: Ellis Horwood. pp.163-169. Bennett, K. P., and Mangasarian, 0. L., (1992) Robust linear programming discrimination of two linearly inseparable sets, Optimization Methods and Software 1, 23-34, Gordon & Breach Science Publishers. Bennett, P. (1996) A neural net-based weather prediction system. Project Report, Knowledge-Based Systems MSc. School of Cognitive and Computing Sciences, University of Sussex. URL: http://freepages.pavilion.net/usersipbsO170.htm Accessed date: 6 December 2003. Bigus, J. P. (1996) Data Mining with Neural Networks: Solving Business Problems from Application Development to Decision Support, McGraw Hill, New York. Bishop, C. (1995). Neural Networks for Pattern Recognition. Oxford: University Press. Bottaci, L., and Drew, P. J. (1997). Artificial Neural Networks Applied to Outcome Prediction for Colorectal Cancer Patients in Separate Institutions. Lancet, Vol. 350, Issue 9076, pp. 469-473. Byun, H. and KO, B. C. (2003) Robust face detection and tracking for real-life applications. International Journal of Pattern Recognition and Artrficial Intelligence. Caruana, R., Baluja, S., and Mitchell, T. (1996) Using the Future to "Sort Out" the Present: Rankrop and Multitask Learning for Medical Risk Evaluation. Advances in Neural Information Processing Systems 8, The MIT Press, Cambridge, pp. 959- 965. Cestnik, G., Konenenko,I, & Bratko,I. ( 1987) Assistant-86: A Knowledge-Elicitation Tool for Sophisticated Users. In I.Bratko & N.Lavrac (Eds.) Progress in Machine Learning, 3 1-45, Sigma Press. Chowdhury, A., Aljlayl, M., Jensen, E., Beitzel, S., Grossman, D. and Frieder, 0. (2002) Linear Combinations Based on Document Shucture and Varied Stemming for Arabic Retrieval. Proceedings of the idh Text Retrieval Conference (TREC- 2002). Clark, P. and Niblett, T. (1987) Induction in Noisy Domains. In Progress in Machne Learning, In Proceedings of the 2N' European Working Session on Learning, 11- 30, Bled, Yugoslavia: Sigma Press. DARPA Neural Network study (1988) AFCEA International Press, p. 60. Deboeck, Guido J. & Cader, Masud. (Ed.) (1994). Pre- and Postprocessing of Financial Data. Canada: John Wiley & Sons. pp. 27-44. DeLurgio, S. A. (2000) Forecasting Principle and Applications. McGraw-Hill International Editions. Demuth, H., and Beale, M., (1998) Neural Network Toolbox User's Guide Ver.3 For Use With Matlab. Massachusetts: The Mathworks Inc. Dybowski, R. (2000). Neural Computation in Medicine: Perspective and Prospects. In Malmgren, H., Borga, M., Niklasson, L. (eds.) Proceedings of the ANNIMAB-1 Conference (Artificial Neural Networks in Medcine and Biology), Goteborg, 13- 16 May 2000. Springer, pp. 26-36. Engels, R., and Theusinger, C., (1998) Using a Data Metric for Preprocessing Advice for Data Mining Applications. 19" European Conference on Artificial Intelligence. John Wiley & Sons, Ltd. Feng, C., Sutherland, A., King, S., Muggleton, S. and Henery, R. (1 993). Comparison of Machine Learning Classifiers to Statistics and Neural Networks. A1 & Stats Conf. 93. Fielden, M. R., Halgren, R. G., Dere, E., and Zackarewski, T. R., (2002) GP3: GenePix Post-processing Program for Automated Analysis of Raw Microarray Data. Bioinformatics Application Notes. Vol. 1 8. No. 5, pg 77 1 -773. Gomez-Skameta, A. F., Jimenez, F., and Ibanez, J., (1997) Data Preprocessing in Knowledge Discovery with Fuzzy-Evolutionary Algorithm. Departamento de Informatics, Universidad de Murcia. Han, J., and Kamber, M., (2001) Data Mining: Concept and Techniques. Simon Fraser University, Canada. Haykin, S. (1999) Neural Networks: A Comprehensive Foundation, 2nd Edition, New Jersey: Prentice Hall. He, D., Park, H. R., Murray, G. C., Subotin, M. and Oard, D. W., (2002) Topic Tracking at the University of Maryland, Institute for Advanced Computer Studies, University of Maryland. Heden, B., Ohlsson, M., Rittner, R., Pahlm, O., Haisty, W. K., Peterson, C., and Denbrandt, L. (1 996) Agreement Between Artificial Neural Networks and Human Expert for the Electrocardiographic Diagnosis of Healed Myocardial Infarction. Journal of the American College of Cardiology, Vol. 28, pp. 10 12- 10s 16. Hoong, N. K. (1988) Medical Information Science - Framework and Potential. International Seminar and Exhibition Computerization for Development-the Research Challenge, Universiti Pertanian Malaysia: Kuala Lumpur, pp. 19 1 - 198. Jankowski, N. (1999) Approximation and Classification in Medicine with IncNet Neural Networks. Machine Learning and Applications: Machine Learning in Medical pplications. Chania, Greece, pp. 53-5 8. Kaastra, I. and Boyd, M. (1996) Designing a Neural Network for Forecasting Financial and Economic Time Series. Neurocomputing 10: 2 15-236. Elsevier Science B. I? Karkanis, S. A., Magoulas, G. D., Grigoriadou, M., and Schurr, M. (1999) Detecting Abnormalities in Colonoscopic Images by Textual Description and Neural Networks. Machine Learning and Applications: Machine Learning in Medical Applications. Chania, Greece, pp. 59-62. Katz, J. 0. (1990) Developing neural network forecasters for trading, Technical Analysis of Stocks and Commodities. April 1990. 58-70. Keall y, R. ( 1 999) A rtlficial Neural Network: An Introductory Course. URL : http:i!www. maths. uwa. edu.aui-rkeallyhnn all! Accessed date: 9 January 2002. Kivinen, J. and Warmuth, M. K. (2001) Relative Loss Bounds For Multidimensional Regression Problems. Kluwer Academic Publj shers. Machine Learning, 45, 30 1- 329. Klaussen, K. L. and Uhng, J. W. (1994) Cash soybean price prediction with neural networks; NCR-134 Conference on Applied Commodity Analysis, Price Forecasting, and Market Risk Management Proceeding, Chicago, 56-65. Salchenberger, L. M., Mine Cinar, E., and Lash, N. A. (1992) Neural Network: A new tool for predicting thnft failures; Decision Sciences 23 (JulyIAug. 1 992) 899-9 16. Klerfors, D. (1998) Artificial Neural Networks, School of Business and Adminstration, Saint Louis University. Lapuerta, P., Azen, S.P., and La, B. (1995) Use of Neural Networks in predicting the risk of coronary artery disease, Comput Biomed Res, Feb;28(!):38-52 Lapuerta, P., Rajan, S., and Bonacini, M. (1997) Neural networks as prehctors of outcomes in alcoholic patients with severe liver disease, Hepatology. Feb;25(2):302-6 Lawrence, J., (1 99 1) Data Preparation for a Neural Network. AI Expert, Vol. 6, No. 1 1, 34-4 1. LiMin Fu. (1 994). Neural Networks in Computer Intelligence. Singapore: Mc-Graw Hill. pp. 18-19,31, 80-82. Lippmann, R. P., Kulkolich, L., Shahian, D. (1995) Predicting the Risk of Complications in Coronary Artery Bypass Operations Using Neural Networks. Advances in Neural Information Processing Systems 7, The MIT Press, Cambridge, pp. 1 053 - 1062. Mangasarian, 0. L., and Wolberg, W. H., (1990) Cancer diagnosis via linear programming, SIAM News, Vol. 23, Number 5, September 1990, pp 1 & 18. Mangasarian, 0. L., Setiono, R., and Wolberg, W.H. (1 990) Pattern recognition via linear programming: Theory and application to medical diagnosis, in: Large-scale numerical optimization, Thomas F. Coleman and Yuying Li, editors, SIAM Publications, Philadelphia 1990, pp 22-30. McCulloch, W., and Pitts, W. (1943) "A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics, vol. 5, pp. 1 1 5-1 33. Mendelsohn, L. (1 993) Technical Analysis of Stocks & Commodities: Preprocessing Data For Neural Networks. URL : http:/inwwr.day-tradingcommodities. com!preprocessing data. asp. Accessed Date: 3 September 2003. ski, R. S., Mozetic, I., Hong, J., and La~ri3~N,. (1986) The Multi-Purpose Incremental Learning System AQ15 and its Testing Application to Three Medical Domains. In Proceedings of the F$h National Conference on Artrficial InteZZigence, 104 1 - 1045, Philadelpha, PA: Morgan Kaufmann. Minsky, M. and Papert, S., (1969) Perceptrons: An introduction to Computational Geometv, MIT Press. Naeher, L. P., Holford, T. R., Beckett, W. S., Belanger, K., Triche, E. W., Bracken, M. B., and Leaderer, B. P. (1993) Healthy Women's PEF Variations with Ambient Summer Concentrations of PMlo, Phd2.5, SO^^, H+, and 0, . American Journal Respiratory Critical Care Medicine, V . 160, Number 1, 1 1 7- 125. Nelson, M. M. and Illingworth, W.T. (1991) A Practical to Guide to Neural Nets, Addison Wesley, MA. Nignn, A., (1993) Neural Network for pattern recognition, Cambridge, MA: The MIT press, p. 1 1. Nikki, M., and Morgan, N., ( 1999) Combining Connectionist Multi-Band And Full-Band Probability Streams For Speech Recowtion Of Natural Numbers. International Computer Science Institute, University of California. Partridge, D., Abidi, S. S. R., and Goh, A. (1996) Neural Network Applications in Medicine. Proceedings of National Confeence on Research and Development in Computer Science and Its Applications (REDECS'96), Universiti Pertanian Malaysia: Kuala Lumpur, pp. 20 - 23. Plett, G.L., Takeshi Doi and Don Torrieri, (1996) Present and Future Methods of Mine Detection Using Scattering Parameters and An Artificial Neural Networks, Proceeding SPIE vol. 2765, Detection and Remediation Technologies for Mines and Minelike Targets, April 1996. Pranckeviciene, E. (1999) Finding Similarities Between An Activity of the Different EEG7s by means of a Single layer Perceptron. Machine Learning and Applications: MachineLearning in Medical Applications. Chania, Greece, pp. 49- 52. Rashid, R., Jarnaludhn, H. and Sailna Amin, A. (2003) Application of Multi-Layer Perceptron in Modeling Tapioca Starch Hydrolysis. In Artificial Intelligence Applications in Industries, June 24-25,2003. Rosenblatt, F., (1958) The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, Cornell Aeronautical Laboratory, Psychological Review, v65, No. 6, pp. 386-408. Rumelhart, D.E., Hinton, G.E., and Williams, 13. J., (1986) Learning Internal Representations by Error Propagation, in Rumelhart, D. E. and McCleZZand, J. L. (editor), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Cambridge, MA, and London, England. Sarle, W. S. (1999) Neural Network FAQ, part 1 of 7 : Introduction. Periodic posting to the Usenet Newsgroup comp. ai. neural-nets, ft!p:;jft tp. sas. cornitpubhe url;FA0. htm 1 Accessed date: 2 1 May 2002. Schalkoff, R. J. (1997) Artificial Neural Network, McGraw Hill, New York. Shanker, M. S., (1996) Using Neural Networks to Predict the Onset of Diabetes Mellitus, Department of Administrative Sciences, Kent State University, Kent, OH 44242, In Journal of Chemical Information and Computer Sciences, 36. Siraj, F., Zakaria, A., Aziz, A. and Abas, Z., (2003) A Web Based Business Insolvency Classifier using Neural Network. Proceeding of AIAI 2003. Stergiou, C., and Siganos, D., (1996) Neural Networks, Volume 4. Stone, J.V. and Thorton, C.J. (1995) Can Artificial Neural Networks Discover Useful Regularities?. In Artificial Neural Networks, Conference Publication No. 409 IEE. Pages 20 1-205.26-28 June 1995. Street, W. N., Mangasarian, 0. L., and Wolberg, W. H. (1996) Individual and Collective Prognostic Prediction. Thirteenth International Conference on Machine Learning. Tan, M., and Eshelman, L. (1988) Using weighted networks to represent classification knowledge in noisy domains. Proceedings of the Frfih International Conference on Machine Learning, 121-134, Ann Arbor, MI. Tsoukalas, L. H., and Uhng R. E., (1997) Fuzzy and Neural Approaches in Engineering, John Wiley & Sons, Inc. Turban, E., (1 993) Decision Support and Expert Systems: Management Support Systems, Macmillan Publishing Company. Varslot, T. (1997) ArtrJicial Neural Network and Breast Cancer. URL: http:i~~nv~~.~arslot.neti'trondkarstenldagliglivipubli kasjoneriprosiektiilproskt. 1 Accessed date: 18 March 2003. Wang, H., Chen, H., Wang, Y. and Sun, W., (2001) Constructive Competitive Neural Network for Associative Memories, Department of Computer Science and Engineering, Fudan University, Shanghai China. Wolberg, W. H., and Mangasarian, 0. L., (1990) Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences, U.S.A., Vol. 87, December 1990, pp 9 193-9 196. Wolfram, S. (2003) Data Pre-Processing: Neural Network Documentation. URL: -l~t tp:~/docun~ents.~~~olfran~.con~/applicationdne~.~ral~~e -tt urorks/Ne~~ralNe y!2.2.0.h tml Accessed date: 24 March 2003. Yoon, Y., & G. Swales. (1991). Prehctina Stock Price Performance: A Neural Network Approach. Proceedings of the 24' Annual Hawaii International Conference on Systems Sciences. EEE Computer Society Press: vol. 4, pp. 156-62. Zhou, X. Data Mining Semester Project Progress Report. (2003) URL: http:li'hoine.olemiss.edd-xzhou/pr~es1 s. Accessed date: 1 November 2003. Zurada, J. M., (1 992) Introduction To Artificial Neural Systems, Boston: P WS Publishing Company , p. XV.

The Impact of Normalization Techniques on Performance Backpropagation Networks

Similar Items