International Journal of Computer Network and Information Security (IJCNIS)
IJCNIS Vol. 14, No. 5, 8 Oct. 2022
Cover page and Table of Contents: PDF (size: 545KB)
Detection of Unknown Insider Attack on Components of Big Data System: A Smart System Application for Big Data Cluster
Full Text (PDF, 545KB), PP.47-59
Views: 0 Downloads: 0
Author(s)
Swagata Paul
1,2,*
Sajal Saha
3
Radha Tamal Goswami
2
Index Terms
Unknown Insider Attack, Big Data Cluster, Security Analysis of a Big Data Cluster, Framework for Insider Attack Detection in a Big Data Cluster
Abstract
Big data applications running on a big data cluster, creates a set of process on different nodes and exchange data via regular network protocols. The nodes of the cluster may receive some new type of attack or unpredictable internal attack from those applications submitted by client. As the applications are allowed to run on the cluster, it may acquire multiple node resources so that the whole cluster becomes slow or unavailable to other clients. Detection of these new types of attacks is not possible using traditional methods. The cumulative network traffic of the nodes must be analyzed to detect such attacks. This work presents an efficient testbed for internal attack generation, data set creation, and attack detection in the cluster. This work also finds the nodes under attack. A new insider attack named BUSY YARN Attack has been identified and analyzed in this work. The framework can be used to recognize similar insider attacks of type DOS where target node(s) in the cluster is unpredictable.
Cite This Paper
Swagata Paul, Sajal Saha, Radha Tamal Goswami, "Detection of Unknown Insider Attack on Components of Big Data System: A Smart System Application for Big Data Cluster", International Journal of Computer Network and Information Security(IJCNIS), Vol.14, No.5, pp.47-59, 2022. DOI:10.5815/ijcnis.2022.05.04
Reference
[1]M. Aamir and S. M. A. Zaidi, “DDoS attack detection with feature engineering and machine learning: the framework and performance evaluation,” Int. J. Inf. Secur., vol. 18, no. 6, pp. 761–785, Dec. 2019, doi: 10.1007/s10207-019-00434-1.
[2]J. Jiao et al., “Detecting TCP-Based DDoS Attacks in Baidu Cloud Computing Data Centers,” in 2017 IEEE 36th Symposium on Reliable Distributed Systems (SRDS), Hong Kong, Hong Kong, Sep. 2017, pp. 256–258. doi: 10.1109/SRDS.2017.37.
[3]G. Shrivastava, P. Kumar, B. B. Gupta, S. Bala, and N. Dey, Eds., Handbook of Research on Network Forensics and Analysis Techniques: IGI Global, 2018. doi: 10.4018/978-1-5225-4100-4.
[4]T. H. Divyasree and K. K. Sherly, “A Network Intrusion Detection System Based On Ensemble CVM Using Efficient Feature Selection Approach,” Procedia Comput. Sci., vol. 143, pp. 442–449, 2018, doi: 10.1016/j.procs.2018.10.416.
[5]J. Jabez and B. Muthukumar, “Intrusion Detection System (IDS): Anomaly Detection Using Outlier Detection Approach,” Procedia Comput. Sci., vol. 48, pp. 338–346, 2015, doi: 10.1016/j.procs.2015.04.191.
[6]D. Cutting, “The ApacheTM Hadoop®.” May 2019. [Online]. Available: https://hadoop.apache.org
[7]Hortonworks, “Apache Hadoop Ecosystem and Open Source Big Data Projects.” May 2019. [Online]. Available: https://hortonworks.com/ecosystems
[8]Ambari, “The Apache Ambari Project.” May 2019. [Online]. Available: https://ambari.apache.org
[9]Wireshark, “dumpcap - Dump network traffic.” Jul. 2019. [Online]. Available: https://www.wireshark.org/docs/man-pages/dumpcap.html
[10]S. K. Aditham and N. Ranganathan, “SYSTEMS AND METHODS FOR DETECTING ATTACKS IN BIG DATA SYSTEMS,” 20190089720, Mar. 2019 [Online]. Available: http://www.freepatentsonline.com/y2019/0089720.html
[11]W. Glenn and W. Yu, “Cyber Attacks on MapReduce Computation Time in a Hadoop Cluster,” in Big Data Concepts, Theories, and Apps, Springer, 2016, pp. 257–279.
[12]J. Huang, D. M. Nicol, and R. H. Campbell, “Denial-of-service threat to Hadoop/YARN clusters with multi-tenancy,” in 2014 IEEE International Congress on Big Data, 2014, pp. 48–55.
[13]S. Aditham and N. Ranganathan, “A system architecture for the detection of insider attacks in big data systems,” IEEE Trans. Dependable Secure Comput., vol. 15, no. 6, pp. 974–987, 2017.
[14]S. Alzahrani and L. Hong, “Generation of DDoS Attack Dataset for Effective IDS Development and Evaluation,” J. Inf. Secur., vol. 9, no. 04, p. 225, 2018.
[15]W. Haider, J. Hu, J. Slay, B. P. Turnbull, and Y. Xie, “Generating realistic intrusion detection system dataset based on fuzzy qualitative modeling,” J. Netw. Comput. Appl., vol. 87, pp. 185–192, 2017.
[16]Metron, “Apache Metron.” May 2019. [Online]. Available: https://hortonworks.com/apache/metron
[17]Metron, “Apache Metron Big Data Security.” [Online]. Available: https://metron.apache.org
[18]MIT, “Kerberos - Network Auth. Protocol.” [Online]. Available: https://web.mit.edu/kerberos/
[19]Hadoop, “Hadoop in Secure Mode.” Jul. 2019. [Online]. Available: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html
[20]MIT, “Kerberos Drawbacks.” [Online]. Available: https://web.mit.edu/rhel-doc/5/RHEL-5-manual/Deployment_Guide-en-US/ch-kerberos.html
[21]S. Khandelwal, “Insecure Hadoop Clusters Expose Over 5,000 Terabytes of Data.” [Online]. Available: https://thehackernews.com/2017/06/secure-hadoop-cluster.html
[22]M. Yoder and S. Acharya, “Protecting Hadoop Clusters From Malware Attacks.” Nov. 2018. [Online]. Available: https://blog.cloudera.com/blog/2018/11/protecting-hadoop-clusters-from-malware-attacks
[23]C. R. Panigrahi, M. Tiwary, B. Pati, and H. Das, “Big data and cyber foraging: future scope and challenges,” in Techniques and Environments for Big Data Analysis, Springer, 2016, pp. 75–100.
[24]J. Wang, T. Wang, Z. Yang, Y. Mao, N. Mi, and B. Sheng, “SEINA: A stealthy and effective internal attack in Hadoop systems,” in 2017 International Conference on Computing, Networking and Communications (ICNC), 2017, pp. 525–530. doi: 10.1109/ICCNC.2017.7876183.
[25]Z. Dou, I. Khalil, A. Khreishah, and A. Al-Fuqaha, “Robust Insider Attacks Countermeasure for Hadoop: Design and Implementation,” IEEE Syst. J., vol. 12, no. 2, pp. 1874–1885, 2018, doi: 10.1109/JSYST.2017.2669908.
[26]Hakan Kekül, Burhan Ergen, Halil Arslan, "A New Vulnerability Reporting Framework for Software Vulnerability Databases", International Journal of Education and Management Engineering, Vol.11, No.3, pp. 11-19, 2021.
[27]Hariharan. M, Abhishek H. K, B. G. Prasad, "DDoS Attack Detection Using C5.0 Machine Learning Algorithm", International Journal of Wireless and Microwave Technologies, Vol.9, No.1, pp. 52-59, 2019.
[28]T. Raja Sree, S. Mary Saira Bhanu, "Investigation of Application Layer DDoS Attacks Using Clustering Techniques", International Journal of Wireless and Microwave Technologies, Vol.8, No.3, pp. 1-13, 2018.
[29]Xin ZHANG, Ying ZHANG, Raees ALTAF, Xin FENG, "A Multi-agent System-based Method of Detecting DDoS Attacks", International Journal of Computer Network and Information Security, Vol.10, No.2, pp.53-64, 2018.
[30]D. Cutting, “Apache Hadoop 3.1.1 Documentation.” Nov. 2021. [Online]. Available: https://hadoop.apache.org/docs/r3.1.1/
[31]A. S. Foundation, “Apache Hadoop 3.3.0 - Apache Hadoop YARN.” Jun. 2020. [Online]. Available: https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html
[32]S. Paul, S. Saha, and R. T. Goswami, “Big Data Cluster Service Discovery: A System Application for Big Data Cluster Security Analysis,” in Data Science and Analytics, Springer Singapore, 2020, pp. 331–341. doi: 10.1007/978-981-15-5830-6_28.