International Journal of Intelligent Systems and Applications (IJISA)
IJISA Vol. 12, No. 1, 8 Feb. 2020
Cover page and Table of Contents: PDF (size: 489KB)
Enhancement Processing Time and Accuracy Training via Significant Parameters in the Batch BP Algorithm
Full Text (PDF, 489KB), PP.43-54
Views: 0 Downloads: 0
Author(s)
Mohammed Sarhan AlDuais
1,*
Fatma Susilawati Mohamad
1
Mumtazimah Mohamad
1
Mohd Nizam Husen
2
Index Terms
Enhancement processing time, accuracy Training, Dynamic momentum factor, Dynamic learning rate, Batch Back-propagation algorithm
Abstract
The batch back prorogation algorithm is anew style for weight updating. The drawback of the BBP algorithm is its slow learning rate and easy convergence to the local minimum. The learning rate and momentum factor are the are the most significant parameter for increasing the efficiency of the BBP algorithm. We created the dynamic learning rate and dynamic momentum factor for increasing the efficiency of the algorithm. We used several data set for testing the effects of the dynamic learning rate and dynamic momentum factor that we created in this paper. All the experiments for both algorithms were performed on Matlab 2016 a. The stop training was determined ten power -5. The average accuracy training is 0.9909 and average processing time improved of dynamic algorithm is 430 times faster than the BBP algorithm. From the experimental results, the dynamic algorithm provides superior performance in terms of faster training with highest accuracy training compared to the manual algorithm. The dynamic parameters which created in this paper helped the algorithm to escape the local minimum and eliminate training saturation, thereby reducing training time and the number of epochs. The dynamic algorithm was achieving a superior level of performance compared with existing works (latest studies).
Cite This Paper
Mohammed Sarhan Al _ Duais, Fatma Susilawati Mohamad, Mumtazimah Mohamad, Mohd Nizam Husen, "Enhancement Processing Time and Accuracy Training via Significant Parameters in the Batch BP Algorithm", International Journal of Intelligent Systems and Applications(IJISA), Vol.12, No.1, pp.43-54, 2020. DOI:10.5815/ijisa.2020.01.05
Reference
[1]M. S. Al_Duais , F. S. Mohamad,” Improved Time Training with Accuracy of Batch Back Propagation Algorithm Via Dynamic Learning Rate and Dynamic Momentum Factor,” IAES International Journal of Artificial Intelligence , Vol. 7, No. 4, pp. 170~178, 2018.
[2]F.Ortega-Zamorano, J.M. Jerez, I. Gómez, & L.Franco,”Layer multiplexing FPGA implementation for deep back-propagation learning”. Integrated Computer Aided Engineering , 24(2), PP.171-185,2017
[3]Y.Hou, & H.Zhao, “Handwritten Digit Recognition Based on Improved BP Neural Network,” Proceedings of 2017 Chinese Intelligent Systems Conference.. CISC 2017 ,pp. 63-70, 2018. [4]E. Ibrahim, Hussien, & Z.E.Mohamed, “Improving Error Back Propagation Algorithm by using Cross Entropy Error Function and Adaptive Learning rate,” International Journal of Computer Applications, 161(8), PP.5-9, 2017.
[5]C.B.Khadse, M.A Chaudhari, & V.B.Borghate, “Conjugate gradient backpropagation based artificial neural network for real time power quality assessment, “ Electrical Power and Energy Systems, 82, PP.197–206 ,2016.
[6]M. S. Al_Duais , F. S Mohamad & M.Mohamad,” Improved the Speed Up Time and Accuracy Training in the Batch Back Propagation Algorithm via Significant Parameter”. International Journal of Engineering & Technology, 7 (3.28) , pp. 124-130, 2018.
[7]Z. X Yang, G. S. Zhao, H. J. Rong & J.Yang, “Adaptive backstepping control for magnetic bearing system via feed forward networks with random hidden nodes,” Neurocomputing, 174, PP.109–120, 2016.
[8]Li, Jie, R.Wang , T.Zhang , & X.Zhang, “Predication Photovoltaic Power Generation Using an Improved Hybrid Heuristic Method. Sixth International conference on information Science and Technology ,pp. 383-387, 2016.
[9]M.Sheikhan, M.A Arabi, &D. Gharavian, “Structure and weights optimization of a modified Elman network emotion classifier using hybrid computational intelligence algorithms: a comparative study,” Connection Science, 27(4), PP.340-357, 2015, doi:10.1080/09540091.2015.1080224
[10]R. Karimi, F.Yousefi, M.Ghaedi & K.Dashtian,” Back propagation artificial neural network and central composite design,” Chemometrics and Intelligent Laboratory Systems, 159, 127–137,PP.2016
[11]R.Kalaivani, K.Sudhagar K, Lakshmi P, “Neural Network based Vibration Control for Vehicle Active Suspension System”, Indian Journal of Science and Technology.9(1), 2016.
[12]S. X. Wu, D. L.Luo, Z. W. Zhou, J. H.Cai, & Y. X. Shi, “A kind of BP neural network algorithm based on grey interval,” ,International Journal of Systems Science, 42(3) ,PP.389-96,2011.
[13]H.. Mo, J.Wang ,H. Niu, “Exponent back propagation neural network forecasting for financial cross-correlation relationship,” Expert Systems with Applications , 53 , PP.106-1016 , 2016.
[14]l.v Kamble, ;D.R Pangavhane, & T.P Singh, “ Improving the Performance of Back-Propagation Training Algorithm by Using ANN,”International Journal of Computer, Electrical, Automation, Control and Information Engineering, 9(1), PP.187- 192, 2015.
[15]J.M.Rizwan, PN.Krishnan, R.Karthikeyan, SR.Kumar, “ Multi layer perception type artificial neural network based traffic control,” Indian Journal of Science and Technology, 9(5), 2016.
[16]X.Xue, Y.Pan, R. Jiang, & Y. Liu, “Optimizing Neural Network Classification by Using the Cuckoo Algorithm,” 11 the International Conference on Natural Computation (ICNC) , PP.24-30,2015.
[17]L. He , Y.Bo, & G.Zhao,” Speech-oriented negate-motion recognition,” Proceedings of the 34th Chinese Control Conference ,PP.3553 -3558 , 2015.
[18]Y.Zhang, S.Zhao, & L.Tang, “Energy Consumption Prediction for Steelmaking Production Using PSO-based BP Neural Network,”Congress on Evolutionary Computation (CEC) PP.3207- 3214, 2016.
[19]Q.Abbas, F. Ahmad, & M. Imran, “Variable Learning rate Based Modification in Backpropagation algorithm (MBPA) of Artificial neural Network for Data Classification,” Science International, 28(3), 2016.
[20]V. P. S. Kirar, “Improving the Performance of Back-Propagation Training Algorithm by Using ANN,” World Academy of Science, Engineering, and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, 9(1) ,PP.187-192,2015
[21]A. A. Hameed , B. Karlik, & M. S. Salman, “Back-propagation algorithm with variable adaptive momentum”, Knowle dge-Base d Systems, 114,PP.79–87, 2016.
[22]W. Zhang, Z. Li, W .Xu, H.Zhou , “A classifier of satellite signals based on the back-propagation neural network,” In 8th International Congress on Image and Signal Processing (CISP), PP.1353-1357,2015.
[23]H. Azami , & J.Escudero, “A comparative study of breast cancer diagnosis based on neural network ensemble via improved training algorithms,” Proceedings of 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) ,PP.2836-2839,2015.
[24]Leema, N., Nehemiah, H. K., & Kannan, A. (2016). Neural Network Classifier Optimization using Differential Evolution with Global Information and Back Propagation Algorithm for Clinical Datasets. Applied Soft Computing. doi:http://dx.doi.org/doi:10.1016/j.asoc.2016.08.001
[25]Kamble, L V; Pangavhane, D R; Singh, T P. (2015). Improving the Performance of Back-Propagation Training Algorithm by Using ANN. International Journal of Computer, Electrical, Automation, Control and Information Engineering, 9(1), 187- 192.
[26]Liew, S. S., Hani, M. K., & Bakhteri, R. (2016). An Optimized Second Order Stochastic Learning Algorithm for Neural Network Training.Neurocomputing. Neurocomputing, 186, 74 – 89.
[27]Andrade, A., Costa, M., Paolucci,, L., Braga, A., Pires, F., Ugrinowitsch, H., & Menzel, H.-J. (2015). A new training algorithm using artificial neural networks to classify gender-specific dynamic gait patterns. Computer Methods in Biomechanics and Biomedical Engineering, 4, 382-390. doi:10.1080/10255842.2013.803081
[28]Kirar, V. P. (2015). Improving the Performance of Back-Propagation Training Algorithm by Using. International Journal of, 9(1), 187-192.
[29]Yui, M., & Kojima, I. (2013). A Database-Hadoop Hybrid Approach to Scalable Machine Learning. IEEE International Congress on Big Data (pp. 1-8). IEEE.
[30]Hamid, N. A., Nawi, N. M., & Ghazali, R. (2012). The effect of adaptive gain and adaptive momentum in improving training time of gradient descent back propagation algorithm on classifation problems. International Scientific Conferenc. 15. ISC.
[31]Nawi, N. M., Khan, A., & Rehman, M. Z. (2013). A new levenberg marquardt based back propagation algorithm training with cuckoo search. The 4th International Conference on Electrical Engineering and Informatics (pp. 18-23). ICEEI.
[32]N.M. Nawi, N. A Hamid, R. S , R.Ransing, Ghazali & M. N. Salleh, “Enhancing Back Propagation NeuralNetwork Algorithm with Adaptive Gain on Classification Problems,” networks, 4(2) ,2011.
[33]T.N .Bui & H. Hasegawa, “Training Artificial Neural Network using modification of Differential Evolution Algorithm,” International journal of Machine Learning and computing, 5(1) , PP.1-6, 2015.
[34]M. S. Al_Duais & F. S. Mohamad, A Review on Enhancements to Speed up Training of the Batch Back Propagation Algorithm. Indian Journal of Science and Technology, 9(46), 2016.
[35]M. S. Al Duais,& F.S.Mohamad, “Dynamically-adaptive Weight in Batch Back Propagation Algorithm via Dynamic Training Rate for Speedup and Accuracy Training. Training,”Journal of Telecommunications and Information Technology. 4 , 2017, doi.org/10.26636/jtit.2017: 113017
[36]S. Scanzio , S.Cumani, R.Gemello, R., F.Mana &F.PLaface, “Parallel implementation of Neural Network training for speech recognition,” Pattern Recognition Letters. Vol..1, no.11, PP.1302–1309,2010.