Timely detection of rice diseases can help farmers to take necessary action and thus reducing the yield loss substantially. Automatic recognition of rice diseases from the rice leaf images using computer vision and machine learning can be beneficial over the manual method of disease recognition through visual inspection. During the recent years, deep learning, a very popular and efficient machine learning algorithm, has shown great promise in image classification task. In this paper, a segmentation-based method using deep neural network for classifying rice diseases from leaf images has been proposed. Disease-affected regions of the rice leaves have been segmented using local segmentation method and the Convolutional Neural Network (CNN) has been trained with those images. Proposed method has been applied on three different datasets including the one created by us which consists of the rice leaf images collected from Bangladesh Rice Research Institute (BRRI). Three state-of-the-art CNN architectures VGG, ResNet and DenseNet, used in the proposed method, have been trained with these three datasets for classifying the diseases. Classification performance of the proposed method using the said three CNN architectures for the three datasets have been analyzed and compared. These results show that this model is quite promising in classifying rice leaf diseases. Outcome of this research is an enhancement in the performance of rice disease classification which is quite significant for the viability of this work to be transformed into a real-time application for the farmers.