International Journal of Modern Education and Computer Science (IJMECS)
IJMECS Vol. 6, No. 4, 8 Apr. 2014
Cover page and Table of Contents: PDF (size: 1432KB)
Support Vector Machine as Feature Selection Method in Classifier Ensembles
Full Text (PDF, 1432KB), PP.1-8
Views: 0 Downloads: 0
Author(s)
Index Terms
Classification accuracy, feature selection, classifier ensembles, machine learning, Support Vector Machine
Abstract
In this paper, we suggest classifier ensembles that can incorporate Support Vector Machine (SVM) as feature selection method into classifier ensembles models. Consequences of choosing different number of features are monitored. Also, the goal of this research is to present and compare different algorithmic approaches for constructing and evaluating systems that learn from experience to make the decisions and predictions and minimize the expected number or proportion of mistakes. Experimental results demonstrate the effectiveness of selecting features with SVM in various types of classifier ensembles.
Cite This Paper
Jasmina Đ. Novakovic, "Support Vector Machine as Feature Selection Method in Classifier Ensembles", International Journal of Modern Education and Computer Science (IJMECS), vol.6, no.4, pp.1-8, 2014. DOI:10.5815/ijmecs.2014.04.01
Reference
[1]Doak J. An evaluation of feature selection methods and their application to computer security. Technical report, Davis CA: University of California, Department of Computer Science, 1992.
[2]Almuallim H, Dietterich T G. Learning with many irrelevant features. In: Proc. AAAI-91, Anaheim, CA, 1991, 547-552.
[3]Kira K, Rendell L A. The feature selection problem: tradional methods and a new algorithm. In: Proc. AAAI-92, San Jose, CA, 1992, 122-126.
[4]Blum A I, Langley P. Selection of relevant features and examples in machine learning. Artificial Intelligence, vol. 97, 1997, 245-271.
[5]Duch W, Adamczak R, Grabczewski, K. A new methodology of extraction, optimization and application of crisp and fuzzy logical rules. IEEE Transactions on Neural Networks, vol. 12, 2001, 277-306.
[6]Vapnik V N. The Nature of Statistical Learning Theory. Information Science and Statistics, Springer, 1st edition, 1995.
[7]Boser B E, Guyon I M, Vapnik V N. A Training Algorithm for Optimal Margin Classifiers. In: Proceedings of the Fifth Annual Workshop of Computational Learning Theory (COLT), 1992.
[8]Aizerman M, Braverman E, Rozonoer L. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control 25: 821–837, 1964.
[9]Cortes C, Vapnik V. Support-vector network, Machine Learning, 20:273–297, 1995.
[10]Vapnik V. Statistical Learning Theory. Wiley, New York, NY, 1998.
[11]Breiman L. Bagging Predictors, Machine Learning, vol. 24, no. 2, 123-140, 1996.
[12]Dong L, Yuan Y, Cai Y. Using Bagging Classifier to Predict Protein Domain Structural Class. Journal of Biomolecular Structure & Dynamics, Volume 24, Issue Number 3, 2006.
[13]Freund Y, Schapire R E. Experiments with a New Boosting Algorithm. ICML, 1996.
[14]Kuncheva L. Diversity in Multiple Classifier Systems (editorial). Information Fusion, vol. 6, no. 1, 3-4, 2004.
[15]Rodriguez J J, Kuncheva L I, Alonso C J. Rotation Forest: A New Classifier Ensemble Method. In: IEEE Transactions on pattern analysis and machine intelligence, vol. 28, no. 10, October 2006.
[16]Ting K M, Witten I H. Stacking Bagged and Dagged Models. In: Fourteenth international Conference on Machine Learning, San Francisco, CA, 367-375, 1997.
[17]Prem Melville, Raymond J. Mooney. Constructing Diverse Classifier Ensembles using Artificial Training Examples. Proceedings of the IJCAI-2003, pp.505-510, Acapulco, Mexico, August 2003.
[18]Webb G I. MultiBoosting: A Technique for Combining Boosting and Wagging. Machine Learning, 40, 159–39, Kluwer Academic Publishers, Boston, 2000.
[19]Friedman J, Hastie T, Tibshirani R. Additive logistic regression: A statistical view of Boosting. The Annals of Statistics 2000, vol. 28, No. 2, 337–407.
[20]Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Machine Learning, 46:389-422, 2002.
[21]Weka: Data Mining Software in Java, http://www.cs.waikato.ac.nz/ml/weka/
[22]Yugal Kumar, G. Sahoo. Study of Parametric Performance Evaluation of Machine Learning and Statistical Classifiers. International Journal of Information Technology and Computer Science (IJITCS), vol. 5, no. 6, pp. 57-64, May 2013.
[23]Frank A, Asuncion A. UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science, 2010.