The Internet of Things (IoT) is revolutionizing the technological market with exponential growth year wise. This revolution of IoT applications has also brought hackers and malware to gain remote access to IoT devices. The security of IoT systems has become more critical for consumers and businesses because of their inherent heterogenous design and open interfaces. Since the release of Mirai in 2016, IoT malware has gained an exponential growth rate. As IoT system and their infrastructure have become critical resources that triggers IoT malware injected by various shareholders in different settings. The enormous applications cause flooding of insecure packets and commands that fueled threats for IoT applications. IoT botnet is one of the most critical malwares that keeps evolving with the network traffic and may harm the privacy of IoT devices. In this work, we presented several sets of malware analysis mechanisms to understand the behavior of IoT malware. We devise an intelligent and hybrid model (IHBOT) that integrates the malware analysis and distinct machine learning algorithms for the identification and classification of the different IoT malware family based on network traffic. The clustering mechanism is also integrated with the proposed model for the identification of malware families based on similarity index. We have also applied YARA rules for the mitigation of IoT botnet traffic.