An improved multiple classifier combination scheme for pattern classification

An improved multiple classifier combination scheme for pattern classification

Combining multiple classifiers are considered as a new direction in the pattern recognition to improve classification performance. The main problem of multiple classifier combination is that there is no standard guideline for constructing an accurate and diverse classifier ensemble. This is due to t...

Full description

Saved in:
Bibliographic Details
Main Author: Abdullah,
Format: Thesis
Language:eng
eng
Published: 2015
Subjects:
QA75 Electronic computers
Computer science
Online Access:https://etd.uum.edu.my/5323/1/s92049.pdf
https://etd.uum.edu.my/5323/2/s92049_abstract.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
id my-uum-etd.5323
record_format uketd_dc
institution Universiti Utara Malaysia
collection UUM ETD
language eng
eng
advisor Ku Mahamud, Ku Ruhana
topic QA75 Electronic computers
Computer science
spellingShingle QA75 Electronic computers
Computer science
Abdullah, ,
An improved multiple classifier combination scheme for pattern classification
description Combining multiple classifiers are considered as a new direction in the pattern recognition to improve classification performance. The main problem of multiple classifier combination is that there is no standard guideline for constructing an accurate and diverse classifier ensemble. This is due to the difficulty in identifying the number of homogeneous classifiers and how to combine the classifier outputs. The most commonly used ensemble method is the random strategy while the majority voting technique is used as the combiner. However, the random strategy cannot determine the number of classifiers and the majority voting technique does not consider the strength of each classifier, thus resulting in low classification accuracy. In this study, an improved multiple classifier combination scheme is proposed. The ant system (AS) algorithm is used to partition feature set in developing feature subsets which represent the number of classifiers. A compactness measure is introduced as a parameter in constructing an accurate and diverse classifier ensemble. A weighted voting technique is used to combine the classifier outputs by considering the strength of the classifiers prior to voting. Experiments were performed using four base classifiers, which are Nearest Mean Classifier (NMC), Naive Bayes Classifier (NBC), k-Nearest Neighbour (k-NN) and Linear Discriminant Analysis (LDA) on benchmark datasets, to test the credibility of the proposed multiple classifier combination scheme. The average classification accuracy of the homogeneous NMC, NBC, k-NN and LDA ensembles are 97.91%, 98.06%, 98.09% and 98.12% respectively. The accuracies are higher than those obtained through the use of other approaches in developing multiple classifier combination. The proposed multiple classifier combination scheme will help to develop other multiple classifier combination for pattern recognition and classification.
format Thesis
qualification_name Ph.D.
qualification_level Doctorate
author Abdullah, ,
author_facet Abdullah, ,
author_sort Abdullah, ,
title An improved multiple classifier combination scheme for pattern classification
title_short An improved multiple classifier combination scheme for pattern classification
title_full An improved multiple classifier combination scheme for pattern classification
title_fullStr An improved multiple classifier combination scheme for pattern classification
title_full_unstemmed An improved multiple classifier combination scheme for pattern classification
title_sort improved multiple classifier combination scheme for pattern classification
granting_institution Universiti Utara Malaysia
granting_department Awang Had Salleh Graduate School of Arts & Sciences
publishDate 2015
url https://etd.uum.edu.my/5323/1/s92049.pdf
https://etd.uum.edu.my/5323/2/s92049_abstract.pdf
_version_ 1747827908453335040
spelling my-uum-etd.53232021-03-18T00:17:43Z An improved multiple classifier combination scheme for pattern classification 2015 Abdullah, , Ku Mahamud, Ku Ruhana Awang Had Salleh Graduate School of Arts & Sciences Awang Had Salleh Graduate School of Arts & Sciences QA75 Electronic computers. Computer science Combining multiple classifiers are considered as a new direction in the pattern recognition to improve classification performance. The main problem of multiple classifier combination is that there is no standard guideline for constructing an accurate and diverse classifier ensemble. This is due to the difficulty in identifying the number of homogeneous classifiers and how to combine the classifier outputs. The most commonly used ensemble method is the random strategy while the majority voting technique is used as the combiner. However, the random strategy cannot determine the number of classifiers and the majority voting technique does not consider the strength of each classifier, thus resulting in low classification accuracy. In this study, an improved multiple classifier combination scheme is proposed. The ant system (AS) algorithm is used to partition feature set in developing feature subsets which represent the number of classifiers. A compactness measure is introduced as a parameter in constructing an accurate and diverse classifier ensemble. A weighted voting technique is used to combine the classifier outputs by considering the strength of the classifiers prior to voting. Experiments were performed using four base classifiers, which are Nearest Mean Classifier (NMC), Naive Bayes Classifier (NBC), k-Nearest Neighbour (k-NN) and Linear Discriminant Analysis (LDA) on benchmark datasets, to test the credibility of the proposed multiple classifier combination scheme. The average classification accuracy of the homogeneous NMC, NBC, k-NN and LDA ensembles are 97.91%, 98.06%, 98.09% and 98.12% respectively. The accuracies are higher than those obtained through the use of other approaches in developing multiple classifier combination. The proposed multiple classifier combination scheme will help to develop other multiple classifier combination for pattern recognition and classification. 2015 Thesis https://etd.uum.edu.my/5323/ https://etd.uum.edu.my/5323/1/s92049.pdf text eng public https://etd.uum.edu.my/5323/2/s92049_abstract.pdf text eng public http://sierra.uum.edu.my/record=b1270960~S1 Ph.D. doctoral Universiti Utara Malaysia Ahmadzadeh, M. R., & Petrou, M. (2003). Use of Dempster-Shafer theory to combine classifiers which use different class boundaries. Pattern Analysis & Applications 6(1), 41-46. Ahn, H., Moon, H., Fazzari, M. J., Lim, N., Chen, J. J., & Kodell, R. L. (2007). Classification by ensembles from random partitions of high-dimensional data. Computational Statistics and Data Analysis, 51, 6166-6179. Aljanaby, A., & Ku-Mahamud, K. R. (2011). Interacted multiple ant colonies optimization approach for the single machine total weighted tardiness problem. Proceedings of the 3rd International Conference on Computing and Informatics (ICOCI), (pp. 19-25). Aslam, J. A., & Montague, M. (2001). Models for metasearch. Proceedings of the 24th Annual International ACM SIGIR Conference on Reseach and Development in Information Retrieval, (pp. 276-284). Ayob, M., & Jaradat, G. (2009). Hybrid ant colony systems for course timetabling problems. Proceedings of the 2nd Conference on Data Mining and Optimization, (pp. 120-126). Banfield, R. E., Hall, L. O., Bowyer, K. W., & Kegelmeyer, W. P. (2005). Ensemble diversity measures and their application to thinning. Information Fusion, 6(1), 49-62. Basu, J. K., Bhattacharyya, D., & Kim, T. h. (2010). Use of artificial neural network in pattern recognition. International Journal of Software Engineering and its Application, 4(2), 23-34. Bay, S. D. (1998). Combining nearest neighbour classifiers through multiple feature subsets. Proceedings of the 17th International Conference on Machine Learning, (pp. 37-45). Bell, J. E., & McMullen, P. R. (2004). Ant colony optimization techniques for the vehicle routing problem. Advanced Engineering Informatics, 18, 41-48. Berrichi, A., & Yalaoui, F. (2013). Efficient bi-objective ant colony approach to minimize total tardiness and system unavailability for a parallel machine scheduling problem. The International Journal of Advanced Manufacturing Technology, 68(9-12), 2295-2310. Bi, Y. (2012). The impact of diversity on the accuracy of evidential classifier ensembles. International Journal of Approximate Reasoning, 53, 584-607. Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford: Clarendon Press. Blum, C. (2005). Ant colony optimization: Introduction and recent trends. Physics of life reviews, 2, 353-373. Blum, C., & Dorigo, M. (2004). The hyper-cube framework for ant colony optimization. IEEE Transactions on Systems, Man and Cybernetics-Part B: Cybernetics, 34(2), 1161-1172. Bolon-Canedo, V., Sanchez-Marono, N., & Alonso-Betanzros, A. (2012). An ensemble of liters and classifiers for microarray data classification. Pattern Recognition, 45, 531-539. Breiman, L. (1996). Bagging predictors. Machine Learning, 24 (2), 123-140. Breiman, L. (2001). Random forest. Machine Learning, 45(1), 5-32. Breve, F. A., Ponti-Junior, M. P., & Mascarenhas, N. D. A. (2006). Neural-network combination for noisy data classification. Proceedings of the II Workshop de Visao Comptacional, (pp. 87-92). Brown, G. (2010). Ensemble learning. In C. Sammut, & G. I. Webb (Eds.), Encyclopedia of Machine Learning (pp. 312-320). New York: Springer Science+Business Media. Brown, G., & Kuncheva, L. I. (2010). “Good” and “Bad” diversity in majority vote ensembles. In N. El-Gayar, J. Kittler, & F. Roli (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, Vol. 5997 (pp. 124-133). Berlin Heidelberg: Springer-Verlag. Bryll, R., Osuna, R. G., & Quek, F. (2003). Attribute bagging: Improving accuracy of classifier ensembles by using random feature subsets. Pattern Recognition, 36, 1291-1302. Bui, T. N., & Nguyen, T. H. (2008). An ant-based algorithm for coloring graphs. Discrete Applied Mathematics, 156(2), 190-200. Bullnheimer, B., Hartl, R., & Strauss, C. (1999). A new rank-based version of the Ant System: A computational study. Central European Journal of Operations Research, 7(1), 25-38. Burton, M., Thomassen, M., Tan, Q., & Kruse, T. A. (2012). Prediction of breast cancer metastasis by gene expression profiles: A comparison of metagenes and single genes. Cancer Informatics, 11, 193-217. Canuto, A. M. P., Abreu, M. C. C., Oliveira, L. M., Xavier, C. J., & Santos, A. M. (2007). Investigating the influence of the choice of the ensemble members in accuracy and diversity of selection-based and fusion-based methods for ensembles. Pattern Recognition Letters, 28, 472-486. Carpaneto, E., & Chicco, G. (2008).Distribution system minimum loss reconfiguration in the hyper-cube ant colony optimization framework. Electric Power Systems Research, 78(12), 2037-2045. Cetin, E. I., Kuruuzum, A., & Irmak, S. (2008). The solution of crew scheduling problem with set partitioning model. Aviation and Space Technology, 3(4), 47-54. Chang, C. F. (2008). Reconfiguration and capacitor placement for loss reduction of distribution systems by ant colony search algorithm. IEEE Transactions on power systems, 23(4), 1747-1755. Chen, L. (2007). A new design of multiple classifier system and its application to classification of time series data. Ph.D Thesis, Department of Electrical and Computer Engineering, University of Waterloo, Ontario Canada. Chen, L., & Kamel, M. S. (2007). A new design of multiple classifier system and its application to the classification of time series data. Proceedings of IEEE International Conference on System, Man and Cybernetics, (pp. 385-391). Chen, W.-N., & Zhang, J. (2013). Ant colony optimization for software project scheduling and staffing with an event-based scheduler. IEEE Transactions on Software Engineering, 39(1), 1-17. Chharia, A., & Gupta, R. K. (2013). Email classifier: An ensemble using probability and rules. Proceeding of the 6th International Conference on Contemporary Computing (IC3), (pp. 130-136). Chicco, G. (2011). Ant colony system-based applications to electrical distribution system optimization. In A. Ostfeld (Ed.), Ant Colony Optimization - Methods and Applications (pp. 237-262). Rijeka, Croatia: Intech. Crawford, B., & Castro, C. (2005). ACO with lookahead procedures for solving set partitioning and covering problems. Proceedings of Workshop on Combination of Metaheuristic and Local Search with Constraint Programming Techniques, 2005. Crawford, B., & Castro, C. (2006). Ant colonies using arc consistency techniques for the set partitioning problem. Professional Practice in Artificial Intelligence IFIP International Federation for Information Processing, 218 (pp. 295-301). Boston: Springer. Crawford, B., Castro, C., & Monfroy, E. (2009). A new ACO transition rule for set partitioning and covering problems. Proceedings of International Conference of Soft Computing and Pattern Recognition, (pp. 426-429). Crawford, B., Soto, R., Monfroy, E., Castro, C., Palma, W., & Paredes, F. (2013). A hybrid soft computing approach for subset problems. Mathematical Problems in Engineering, 2013, 1-12. Dara, R. A. (2007). Cooperative training in multiple classifier systems. Ph.D Thesis, Department of Systems Design Engineering, University of Waterloo, Ontario Canada. Dietterich, T. G., & Bakiri, G. (1995). Solving multiclass learning problems via error correcting output codes. Journal Artificial Intelligence 2, 263-286. Dorigo, M. (1992). Optimization, Learning and Natural Algorithm. Ph.D thesis, Dipartimento di Elettronica Politecnico di Milano, Italy. Dorigo, M., & Gambardella, L. M. (1996). A study of some properties of Ant-Q. In H. Voigt, W. Ebeling, I. Rechenberg, & H. Schwefel (Eds.), Parallel Problem Solving from Nature-PPSN IV, International Conference on Evolutionary Computation, Lecture Notes in Computer Science, Vol. 1141 (pp. 656-665). Berlin Heidelberg: Springer. Dorigo, M., & Gambardella, L. M. (1997). Ant colony system: A cooperative learning approach to the travelling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1), 53-66. Dorigo, M., & Stützle, T. (2004). Ant Colony Optimization. USA: MIT Press. Dorigo, M., Maniezzo, V., & Colorni, A. (1996). The ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics-Part B, 26(1), 29-41. Dougherty, G. (2013). Pattern Recognition and Classification : An Introduction. New York: Springer Science+Business Media. Du, P., Sun, H., & Zhang, W. (2009). Target identification from high resolution remote sensing image by combining multiple classifiers. In J. A. Benediktsson, & J. Kittler (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, Vol. 5519 (pp. 408-417). Berlin Heidelberg: Springer-Verlag. Du, P., Xia, J., Zhang, W., Tan, K., Liu, Y., & Liu, S. (2012). Multiple classifier system for remote sensing image classification: A review. Sensors, 4764-4792. Farid, D. M., Zhang, L., Rahman, C. M., & Strachan, R. (2013). Hybrid decision tree and naive bayes classifiers for multi-class classification task. International Journal of Expert System with Application, 41, 1937-1946. Fook, C. Y., Muthusamy, H., Chee, L. S., Yaacob, S., & Adom, A. H. (2013). Comparison of speech parameterization techniques for the classification of speech disfluencies. Turkish Journal of Electrical Engineering & Computer Sciences, 21, 1983-1994. Forrester, E. C. (2006). A process research framework. Pittsburgh Pennsylvania: Software Engineering Institute, Carnegie Mellon University. Frank, A., & Asuncion, A. (2010). UCI machine learning repository Irvine, CA: University of California, School of Information and Computer Science. Retrived from http://archive.ics.uci.edu/ml/. Freund, Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. Proceedings of the 13th International Conference on Machine Learning, (pp. 148-156). Frinken, V., Fischer, A., & Bunke, H. (2010). Combining neural networks to improve performance of handwritten keyword spotting. In N. E. Gayar, J. Kittler, & F. Roli (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, Vol. 5997 (pp. 215-224). Berlin Heidelberg: Springer-Verlag. G´omez, O., & Bar´an, B. (2005). Omicron ACO. A new ant colony optimization algorithm. CLEI Electronic Journal, 8(1), 1-8. Gangardiwala, A., & Polikar, R. (2005). Dynamically weighted majority voting for incremental learning and comparison of three boosting based approaches. Proceedings of International Joint Conference on Neural Networks, (pp. 1131-1136). Giancito, G., & Roli, F. (2000). Dynamic classifier selection. In J. Kittler, & F. Roli (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, Vol. 1857 (pp. 177- 189). Berlin Heidelberg: Springer-Verlag. Giancito, G., & Roli, F. (2001). Dynamic classifier selection based on multiple classifier behaviour. Pattern Recognition, 34, 1879-1881. Guo, Y. H., & Shi, D. Q. (2012, November). A mutation ant colony algorithm for the generalized assignment problem under the special constraints. Applied Mechanics and Materials, pp. 1862-1865. Hajdu, A., Hajdu, L., Jonas, A., Kovacs, L., & Toman, H. (2013). Generalizing the majority voting scheme to spatially constrained voting. IEEE Transaction on Image Processing, 22(11), 4182-4194. Hamzeloo, S., Shahparast, H., & Jahromi, M. Z. (2012). A novel weighted nearest neighbor Ensemble Classifier. Proceedings of the 16th International Symposium on Artificial Intelligence and Signal Processing (AISP), (pp. 413-416). Han, J., Kamber, M., & Pei, J. (2011). Data Mining Concepts and Techniques. Massachusetts: Elsevier. Hansen, L., & Salamon, P. (1990). Neural network ensembles. IEEE Transaction on Pattern Analysis and Machine Intelligence, 12, 993-1001. He, C. L., & Suen, C. Y. (2007). A hybrid multiple classifier system of unconstrained handwritten numeral recognition. Pattern Recognition and Image Analysis, 17(4), 608-611. Hernandez-Lobato, D., & Martinez-Munoz, G. (2013). How large should ensembles of classifiers be? Pattern Recognition, 46(5), 1323-1336. Ho, T. K. (1998). The random subspace method for constructing decision forest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8), 832-844. Ho, T. K., Hull, J. J., & Srihari, S. N. (1992). On multiple classifier systems for pattern recognition. Proceedings of the 11th International Conference on Pattern Recogition, 2, (pp. 84-87). Hong, G. (2013). A hybrid ant colony algorithm for quadratic assignment problem. The Open Electrical & Electronic Engineering Journal, 7(1), 51-54. Hongbo, S., & Yali, L. (2008). An ensemble classifier based on attribute selection and diversity measure. Proceedings of the 5th International Conference on Fuzzy System and Knowledge Discovery, 2, (pp. 106-110). Huang, C., & Wang, J. (2009). Multi-weighted majority voting algorithm on support vector machine and its application. TENCON 2009 IEEE Region 10 Conference, (pp. 1-4). Huang, Y. S., & Suen, C. Y. (1995). A method of combining multiple experts for the recognition of unconstraned handwritten numerals. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(1), 90-94. Huang, Y. S., Liu, K., & Suen, C. Y. (1994). A neural network approach for multi-classifier recognition systems. Proceedings of the 4th International Workshop on Frontiers in Handwritten Recognition, (pp. 235-244). Jacobs, R. A. (1995). Methods for combining experts probability assessments. Neural Computation, 7, 867-888. Jain, A. K., Duin, R. P. W., & Mao, J. (2000). Statistical pattern recognition : A review. IEEE Transaction on Pattern Analysis and Machine Intelligence, 22(1), 4-37. Jevtic, A., Andina, D., James, A., Gomez, J., & Jamshidi, M. (2010). Unmanned aerial vehicle route optimization using ant system algorithm. Proceedings of the 5th International Conference on system of systems engineering, (pp. 1-6). Kaur, N., & Kaur, U. (2013). Survey of pattern recognition methods. International Journal of Advanced Research in Computer Science and Software Engineering, 3(2), 317-319. Khakabimamaghani, S., Barzinpour, F., & Gholamian, M. R. (2010). A high diversity hybrid ensemble of classifiers. Proceedings of the 2nd International Conference on Software Engineering and Data Mining (SEDM), (pp. 461-466). Kim, E., Kim, W., & Lee, Y. (2002). Combination of multiple classifiers for the costumer's purchase behaviour prediction. Decision Suppot System, 34, 167-175. Kim, H. C., Pang, S., Jo, H. M., Kim, D., & Bang, S. Y. (2003). Constructing support vector machine ensemble. Pattern Recognition, 36, 2757-2767. Kim, H., Kim, H., Moon, H., & Ahn, H. (2011). A weight-adjusted voting algorithm for ensembles of classifiers. Journal of the Korean Statistical Society, 40(4), 437-449. Ko, A. H. R., & Sabourin, R. (2013). Single classifier-based multiple classification scheme for weak classifiers: An experimental comparison. Expert Systems with Application, 40, 3606-3622. Ko, A. H. R., Sabourin, R., & Britto, A. (2008). From dinamic classifier selection to dynamic enemble selection. Pattern Recognition, 41(5), 1718-1731. Ko, A. H. R., Sabourin, R., Britto, A., & Oliveira, L. (2007). Pairwise fusion matrix for combining classifiers. Pattern Recognition, 40, 2198-2210. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the 14th International Joint Conference on Artificial Intelligence, 2 (pp. 1137-1145). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. Kong, H., Li, X., Wang, J., & Kambhamettu, C. (2005). Ensemble LDA for face recognition. In D. Zhang, & A. K. Jain (Eds.), Advances in Biometrics, Lecture Notes in Computer Science, Vol. 3832 (pp. 166-172). Berlin, Heidelberg: Springer-Verlag. Koyuncu, H., & Ceylan, R. (2013). Artificial neural network based on rotation forest for biomedical pattern classification. Proceedings of the 36th International Conference on Telecommunications and signal processing (TSP), (pp. 581-585). Kuncheva, L. I. (2001). Combining classifiers: Soft computing solution. In S. K. Pal, & A. Pal (Eds.), Pattern recognition: From classical to modern approaches (pp. 427-452). Singapore: World Scientific Publishing Co. Kuncheva, L. I. (2002a). A theoretical study on six classifier fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2), 281-286. Kuncheva, L. I. (2002b). Switching between selection and fusion in combining classifiers: An experiment. IEEE Transactions on Systems, Man and Cybernetics-Part B, Cybernetisc, 32(2), 146-155. Kuncheva, L. I. (2004). Combining Pattern Classifiers: Methods and Algorithms. Hoboken, New Jersey: John Wiley & Sons, Inc., Publication. Kuncheva, L. I., & Jain, L. C. (2000). Designing classifier fusion systems by genetic algorithms. IEEE Transactions on Evolutionary Computation, 4(4), 327-336. Kuncheva, L. I., & Whitaker, C. J. (2001). Feature subsets for classifier combination: An enumerative experiment. In G. Goos, J. Hartmanis, & J. V. Leeuwen (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, 2096 (pp. 228-237). Berlin Heidelberg: Springer-Verlag. Kuncheva, L. I., & Whitaker, C. J. (2003). Measures of diversity in classifier ensembles and their relationship with ensemble accuracy. Machine Learning, 51(2), 181-207. Kuncheva, L. I., Skurichina, M., & Duin, R. P. W. (2002). An experimental study on diversity for bagging and boosting with linear classifiers. Information Fusion, 3, 245-258. Lee, D. S., & Srihari, S. N. (1995). A theory of classifier combination: The neural network approach. Proceedings of the 3rd International Conference on Document Analysis and Recognition, 1, (pp. 42-45). Lera, G., & Pinzolas, M. (2002). Neighborhood based Levenberg-Marquardt algorithm for network training. IEEE Transactions on Neural Networks, 13(5), 1200-1204. Li, H., & Sun, J. (2009). Majority voting combination of multiple case-based reasoning for financial distree prediction. Expert Systems With Applications, 36, 4363-4373. Li, H., Wen, G., Yu, Z., & Zhou, T. (2013). Random subspaces evidence classifier. Neurocomputing, 110, 62-69. Li, K., & Gao, H. (2010). A novel measure of diversity for support vector machine ensemble. Proceedings of the 3rd International Sysmposium on Intelligence Information Technology and Security Informatics, (pp. 366-370). Liao, Y., & Moody, J. (2000). Constructing heterogeneous committees via input feature grouping. In S. A. Solla, T. K. Leen, & K. R. Muller (Eds.), Advances in Neural Information Processing Systems, Vol. 12 (pp. 921-927). USA: MIT Press. Lincoln, W., & Skrzypek, J. (1989). Synergy of clustering multiple back propagation networks. Advances in Neural Information Processing Systems, 2, 650-659. Littlestone, J. T., & Warmuth, M. K. (1994). The weighted majority algorithm. Information and Computation, 108(2), 212-261. Liu, J., Sun, J., & Wang, S. (2006). Pattern recognition : An overview. International Journal of Computer Science and Network Security, 6(6), 57-61. Lizárraga, E., Castillo, O., & Soria, J. (2013). A method to solve the traveling salesman problem using ant colony optimization variants with ant set partitioning. Recent Andvances on Hybrid Intelligent Systems Tudies in Computational Intelligence, 451, 237-246. Lofstrom, T., Johansson, U., & Niklasson, L. (2007). Empirically investigating the importance of diversity. Proceedings of the 10th International Conference on Information Fusion, (pp. 1-8). Lu, D., & Weng, Q. (2007). A survey of image classification methods and techniques for improving classification performance. International Journal of Remote Sensing 28(5), 823-870. Lu, F., & Yamaoka, F. (1994). Integration of handwritten digit recognition results using evidential rasoning . Proceedings of International Workshop on Frontiers in Handwriting Recognition, (4), (pp. 456-463). Maimon, O., & Rokach, L. (2005). Decomposition methodology for knowledge discovery and datamining. Berlin Heidelberg: Springer-Verlag. Maniezzo, V., & Carbonaro, A. (2000). An ANTS heuristic for the frequency assignment problem. Future Generation Computer Systems 16(8), 927-935. Maniezzo, V., & Colorni, A. (1999). The ant system applied to the quadratic assignment problem. IEEE Transactions on Data and Knowledge Engineering, 11(5), 769-778. Maniezzo, V., & Milandri, M. (2002). An ant-based framework for very strongly constrained problems. In M. Dorigo, G. D. Caro, & M. Sampels (Eds.), Ant Algorithms, Third International Workshop on Ant Algorithms, Lecture Notes in Computer Science, Vol. 2463 (pp. 222-227). Berlin Heidelberg: Springer-Verlag. Margoosian, A., & Abouei, J. (2013). Ensemble-based classifiers for cancer classification using human tumor microarray data. Proceedings of IEEE International Conference on Electrical Engineering (ICEE '13), (pp. 1-6). Medrano-Gracia, P., Crown, B. R., Bluemke, D. A., Finn, J. P., Lee, D. C., Lima, J. A., et al. (2013). Optimal patient classification via statistical decomposition of a 3D leftventricular atlas. Journal of Cardiovascular Magnetic Resonance, 15(1), 89-89. Melville, P., & Mooney, R. J. (2005). Creating diversity in ensembles using artificial data. Information Fusion 6(1), 99-111. Mesquita, M., & Paias, A. (2008). Set partitioning/ covering-based approaches for the integrated vehicle and crew scheduling problem. Computer and Operations Research, 35(5), 1562-1575. Mu, X., Lu, J., Watta, P., & Hassoun, M. H. (2009). Weighted voting-based ensemble classifiers with application to human face recognition and voice recognition. Proceedings of International Joint Conference on Neural Networks, (pp. 2168-2171). Musehane, R., Netshiongolwe, F. V., Nelwamondo, L., Masisi, L., & Marwala, T. (2008). Relationship between diversity and perfomance of multiple classifiers for decision support. Computing Research Repository - CORR, Vol. abs/0810.3. Nabatchian, A., Abdel-Raheem, E., & Ahmadi, M. (2010).A weighted voting scheme for recognition of faces with ilumination variation. Proceedings of the 11th International Conference Control, Automation, Robotics and Vision, (pp. 896-899). Neelamegam, S., & Ramaraj, E. (2013). Classification algorithm in data mining: An Overview. International Journal of P2P Network Trends and Technology (IJPTT), 4(8), 369-374. Neo, T. K. C., & Ventura, D. (2012). A direct boosting algorithm for the k-nearest neighbor classifier via local warping of the distance metric. Pattern Recognition Letters, 92-102. Parshapoor, M., & Bilstrup, U. (2013). Ant colony optimization for channel assignment. Proceedings of the 4th International Conference on Swarm Intelligence, ICSI 2013., (pp. 314-322). Parvin, H., & Beigi, A. (2011). Clustering ensemble framework via ant colony. In I. Batyrshin, & G. Sidorov (Eds.), Lecture Notes in Computer Science, Vol. 7095 (pp. 153-164). Berlin Heidelberg: Springer-Verlag. Parvin, H., & Bidgoli-Minaei, B. (2011). A new space defined by ant colony algorithm to partition data. In A. Bouchachia (ed.), Lecture Notes in Computer Science, Vol. 6943 (pp. 272-284). Berlin Heidelberg: Springer-Verlag. Parvin, H., & Parvin, S. (2012). A robust classifier ensemble for improving the performance of classification. Proceedings of Special Session-Revised Papers, 11th Mexican International Conference on Artificial Intelligence: Advances in Artificial Intelligence Applications, (pp. 62-66). Parvin, H., Alizadeh, H., & Minaei-Bidgoli, B. (2009a). A new method for constructing classifiers ensembles. International Journal of Digital Content Technology and its applications, 3(2), 62-66. Parvin, H., Alizadeh, H., & Minaei-Bidgoli, B. (2009b). Using clustering for generating diversity in classifier ensemble. International Journal of Digital Content Technology and its Application, 3(2), (pp. 51-57). Pasti, R., & de Castro, L. N. (2007). The Influence of Diversity in an Immune-based algorithm to Train MLP Nerworks. Proceedings of the 6th International Conference on Artificial Immune Systems (pp. 71-82). Berlin, Heidelberg: Springer-Verlag. Peker, M., Sen, B., & Bayir, S. (2012). Using artificial intelligence techniques for large scale set partitioning problems. Procedia Technology, 1, 44-49. Polikar, R. (2006). Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, 6(3), 21-44. Ponti, M. P. (2011). Combining classifier: From the creation of ensembles to the decision fusion. Proceedings of the 24th Conference on Graphics, Patterns and Image Tutorials, (pp. 1-10). Ranawana, R., & Palade, V. (2005a). A neural network based multi-classifier system for gene identification in DNA sequences. Neural Computing Application, 14(2), 122-131. Ranawana, R., & Palade, V. (2005b). Multiple classifier systems : A review and roadmap for developers. International Journal of Hybrid Intelligence Systems, 3(1), 35-61. Randall, M., & Lewis, A. (2010). Modifications and additions to ant colony optimisation to solve the set partitioning problem. Proceedings of the 6th IEEE International Conference on e science Workshops, (pp. 110-116). Rebeiro, R., & Enembreck, F. (2013). A sociologically inspired heuristic for optimization algorithm: A case study on ant systems. Expert System with Application, 40, 1814-1826. Rokach, L. (2006). Decomposition methodology for classification task - A meta decomposer framework. Pattern Analysis & Applications, 9, 257-271. Rokach, L. (2008). Genetic algorithm-based feature set partitioning for classification problems. Pattern Recognition, 41(5), 1676-1700. Rokach, L. (2010). Pattern Classification using Ensemble Methods. Singapore: World Scientific Publishing Co. Pte. Ltd. Rokach, L., & Maimon, O. (2005). Feature set decomposition for decision trees. Journal of Intelligent Data Analysis, 9, 131-158. Rokach, L., Chizi, B., & Maimon, O. (2007). A methodology for improving the performance of non-ranker feature selection filters. Pattern Recognition and Artificial Intelligence, 21(5), 809-830. Roli, F. (2009). Multiple classifier system. In S. Z. Li, & A. K. Jain (Eds), Encyclopedia of Biometrics (pp. 981-986). New York: Springer Science+Business Media. Roli, F., & Giancito, G. (2002). Design of multiple classifier systems. In H. Bunke, & A. Kandel (Eds.), Hybrid methods in pattern recognition (pp. 199-226). Singapore: World Scientific Publishing. Santana, L. E. A. (2010). A comparative analysis of genetic algorithm and ant colony optimization to select attributes for an heterogeneous ensemble of classifiers. Proceedings of IEEE Congress on Evolutionary Computation (CEC), (pp. 1-8). Schapire, R. E. (1990). The strength of weak learnability. Machine Learning, 5(2), 197-227. Serpen, G., & Pathical, S. (2009). Classification in high-dimensional feature spaces: Random subsample ensemble. Proceedings of IEEE International Conference on Machine Learning and Application, (pp. 740-745). Shang, F., & Wang, Y. (2010). An ant system optimization QoS routing algorithm for wireless sensor networks. Proceedings of the 3rd International Workshop on Advanced Computational Intelligence, (pp. 25-27). Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27, 379-423, 623-656. Sharkey, A. J. C. (2002). Types of multinet systems. In G. Goos, J. Hartmanis, & J. van Leeuwen (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, Vol. 2364 (pp. 108-117). Berlin Heidelberg: Springer-Verlag. Shi, H., & Lv, Y. (2008). An ensemble classifier based on attribute selection and diversity measure. Proceedings of the 5th International Conference on Fuzzy Systems and Knowledge Discovery, (pp. 106-110). Shilen, S. (1990). Multiple binary tree classifier. Pattern Recognition 23(7), 757-763. Shin, D., & Kim, S. (2009). Nearest mean classifier via-one class SVM. Proceedings of the 1st International Join Conference on Computational Sciences and Optimization, (pp. 593-596). Shipp, C. A., & Kuncheva, L. I. (2002). Relationships between combination methods and measures. Information Fusion, 3, 135-148. Sigari, M. H., Sureshjani, S. A., & Soltanian-Zadeh, H. (2011). Sprot video classification using an ensemble classifier. Proceeding of the 7th Machine Vision and Image Processing, (pp. 1-4). Simpson, E. H. (1949). Measurement of diversity. Nature, 163, 688. Skurichina, M., & Duin, R. P. W. (2002). Bagging, boosting and the random subspace method for linier classifiers. Pattern Analysis and Applications (5), 121-135. Skurichina, M., Kuncheva, L. I., & Duin, R. P. W. (2002). Bagging and boosting for the nearest mean classifier: Effect of sample size on diversity and accuracy. In F. Roli, & J. Kittler (Eds.), Multiple Classifier Systems, Lecture Notes in Computer Science, Vol. 2364 (pp. 62-71). Berlin Heidelberg: Springer-Verlag. Srimani, P. K., & Koti, M. S. (2012). Medical diagnosis using ensemble classifiers - A novel machine-learning approach. Journal of Advanced Computing, 1, 9-27. Stefano, C. D., Cioppa, A. D., & Marcelli, A. (2002). An adaptive weighted majority vote rule for combining multiple classifiers. Proceedings of the 16th International Conference on Pattern Recognition, 2, (pp. 192-195). Strehl, A., & Ghosh, J. (2002). Cluster ensembles - A knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, 3, 583-617. Stützle, T., & Hoos, H. H. (2000). Max-Min ant system. Future Generation Computer Systems 16(8), 889-914. Su, C. T., Chang, C. F., & Chiou, J. P. (2005). Distribution network reconfiguration for loss reduction by ant colony search algorithm. Electric Power Systems Research, 75, 190-199. Suguna, N., & Thanushkodi, K. (2010). An improved k-nearest neighbor classification using genetic algorithm. International Journal of Computer Science, 7(4), 18-21. Sun, J., & Li, H. (2008). Listed companies' financial distress prediction based on weighted majority voting combination of multiple classifiers. Expert Systems with Applications, 35(3), 818-827. Tahir, M. A., & Smith, J. (2010). Creating diverse nearest-neighbour ensembles using simultaneous metahueristic feature selection. Pattern Recognition Letters, 31(11), 1470-1480. Talha, M., & Solung, G. (2013). Reduced transformed features based breast mammograms classification. Life Science Journal, 10(7s), 907-913. Tao, D., & Tang, X. (2004). SVM-based relevance feedback using random subspace method. Proceedings of IEEE International Conference on Multimedia and Expo (ICME), Vol.1, (pp. 269-272). Theodoris, S., & Koutroumbas, K. (2009). Pattern Recognition (4th ed.). New York: Academic. Ting, K. M., Wells, J. R., Tan, S. C., Teng, S. W., & Webb, G. I. (2011). Feature-subspace aggregating: ensembles for stable and unstable learners. Machine Learning 82(3), 375-397. Turhal, U., Babur, S., Avci, C., & Akbas, A. (2013). Performance improvement for diagnosis of colon cancer by using ensemble classification methods. Proceedings of International Conference on Technological Advances in Electrical, Electronics and Computer Engineering (TAEECE) 2013, (pp. 271-275). Ulas, A., Yildiz, O. T., & Alpaydin, E. (2012). Cost-concious comparison of supervised learning algorithms over multipe data sets. Pattern Recognition, 45, 1772-1781. Valdovinos, R. M., & Sánchez, J. S. (2009). Combining multiple classifiers with dynamic weighted voting. Proceedings of the 4th International Conference on Hybrid Artificial Intelligence Systems (HAIS '09) (pp. 510-516). Berlin Heidelberg: Springer-Verlag. Verma, B., & Blumenstein, M. (2008). Pattern Recognition Technologies and Applications: Recent Advances. United States of America: Information Science Reference IGI Global. Wanas, N. M., & Kamel, M. S. (2002). Weighted combining of neural network ensembles. Proceedings of International Joint Conference on Neural Network, (pp. 1748-1752). Wang, L. J., Wang, X. L., & Chen, Q. C. (2005). GA-based feature subset clustering for combination of multiple nearest neighbors classifiers. Proceedings of International Conference on Machine Learning and Cybernetics, Vol.5, (pp. 2982-2987). Wang, P. S. (2011). Pattern Recognition, Machine Intelligence and Biometrics. Beijing, Berlin Heidelberg: Higher Education Press, Springer-Verlag. Wang, Z. Q., Chen, G. H., & Yu, Z. Z. (2012). Linear discrimiminant analysis based on ensemble learning. Proceedings of International Conference on Biological and Biomedical Sciences, Advances in Biomedical Engineering, Vol.9 (pp. 265-271). USA: Information Engineering Research Institute (IERI). Woods, K., Kegelmeyer, W. P., & Bowyer, K. (1997). Combination of multiple classifier using local accuracy estimates. IEEE Transaction on Pattern Analysis and Machine Intelligence, 19(4), 405-410. Wozniak, M. (2008). Classifier fusion based on weighted voting-Analytical and experimental results. Proceedings of the 8th International Conference on Intelligent Systems Design and Application, (pp. 687-692). Wozniak, M. (2009). Evolutionary approach to produce classifier ensemble based on weigted voting. Proceedings of World Congress on Nature & Biologically Inspired Computing, (NaBIC),2009, (pp. 648-653). Wozniak, M., & Crawczyk, B. (2012). Combined classifier based on feature space partitioning. International Journal of Applied Mathematics and Computer Science, 22(4), 855-866. Wozniak, M., & Zmyslony, M. (2010). Chosen problems of designing effective multiple classifier systems. Proceedings of International Conference on Computer Information Systems and Industrial Management Applications, 2010, (pp. 42-47). Wozniak, M., Grana, M., & Corchado, E. (2014). A survey of multiple classifier systems as a hybrid systems. Information Fusion, 16, 3-17. Wu, C. H., & Liang, W. B. (2011). Emotion recognition of affective speech based on multiple classifier using acoustic-prosodic information and semantic labels. IEEE Transaction on Affective Computing 2(1), 10-21. Xing, L.-N., Chen, Y.-W., Wang, P., Zhao, Q.-S., & Xiong, J. (2010). A knowledge-based ant colony optimization for flexible job shop scheduling problems. Applied Soft Computing, 10(3), 888-896. Xu, L., Krzyzak, A., & Suen, C. Y. (1992). Methods of combining multiple classifiers and their application to handwriting recognition. IEEE Transactions on System, Man and Cybernetics, 22(3), 418-435. Yagmahan, B., & Yenisey, M. M. (2010). A multi-objective ant colony system algorithm for flow shop scheduling problem. Expert Systems with Applications, 37, 1361–1368. Yang, L. Y., Qin, Z., & Huang, R. (2004). Design of multiple classifier system. Proceedings of the 3th International Conference on Machine Learning and Cybernetics (pp. 3272- 3276). IEEE Press. Yang, L. Y., Zhang, J. Y., & Wang, W. J. (2009). Selecting and combining classifiers simultaneously with particle swam optimization. Information Technology Journal, 8(2), 241-245. Yang, P., Yang, Y. H., Zhou, B., & Zomaya, Y. (2010). A review of ensemble methods in bioinformatics. Current Bioinformatics, 5(4), 296-308. Yang, S., & Browne, A. (2004). Neural network ensembles: Combining multiple models for enhanced performance using a multistage approach. Expert System, 21(5), 279-288. Zaamout, K., & Zhang, J. Z. (2012). Improving classification through ensemble neural networks. Proceedings of the 8th International Conference on Natural Computation (ICNC), (pp. 256-260). Zanda, M. (2010). A Probabilistic Perspectif on Ensemble Diversity. Ph.D Thesis, Faculty of Engineering and Physical Sciences, School of Computer, University of Manchester, England. Zarafshan, G. R. F., Latif-Shabgahi, C., & Karimi, A. (2010). A novel weighted voting algorithm based on neural networks for fault tolerant system. Proceedings of the 3rd International Conference on Computer Science and Information Technology, (pp. 135-139). Zhang, B. (2012). Reliable face recognition by random subspace support vector machine enemble. Proceedings of International Conference on Machine Learning and Cybernetics (ICMLC), 2012, (pp. 415-420). Zhang, J., Cheng, L., & Ma, J. (2006). A new multiple classifiers combination algorithm. Proceedings of the 1st International Multi-Symposiums on Computer and Computational Sciences (IMSCC), (pp. 287-291). Zhao, C., Deshwar, A. G., & Morris, Q. (2013). Relapsing-remitting multiple sclerosis classification using elastic net logistic regression on gene expression data. Systems Biomedicine, 1(4), 59-65. Zhao, D., Luo, L., & Zhang, K. (2010). An improved ant colony optimization for the communication network routing problem. Mathematical and Computer Modelling, 52(11-12), 1976-1981. Zhao, J., & Yan, C. (2009). A bottleneck assigned binary ant system for multimodal optimization. Proceedings of the 48th IEEE Conference on Decision and Control held jointly with 28th Chinese Control Conference, (pp. 16-18). Zhou, Z.-H. (2012). Ensemble Methods: Foundation and Algorithms. Florida, USA: CRC Press Taylor & Francis Group.

Similar Items