Potatoes play a vital role as a staple crop worldwide, making a significant contribution to global food security. However, the susceptibility of potato plants to various leaf diseases poses a threat to crop yield and quality. Detecting these diseases accurately and at an early stage is crucial for the effective management and protection of crops. Recent advancements in Convolutional Neural Networks (CNNs) have demonstrated potential in image categorization applications. Therefore, the goal of this work is to investigate the potential of CNNs in detecting potato leaf diseases. As neural networks have become part of agriculture, numerous researchers have worked on improving the early detection of potato blight using different machine and deep learning methods. However, there are persistent problems related to accuracy and the time it takes for these methods to work. In response to these challenges, we tailored a convolutional neural network (CNN) to enhance accuracy while reducing the trainable parameters, computational time and information loss. To conduct this research, we compiled a diverse dataset consisting of images of potato leaves. The dataset encompassed both healthy leaves and leaves infected with common diseases such as late blight and early blight. We took great care in curating and preprocessing the dataset to ensure its quality and consistency. Our focus was to develop a specialized CNN architecture tailored specifically for disease detection. To improve the performance of the network, we employed techniques like data augmentation and transfer learning during the training phase. The experimental outcomes demonstrate the efficacy of our proposed customized CNN model in accurately identifying and classifying potato leaf diseases. Our model's overall accuracy was an astounding 99.22%, surpassing the performance of existing methods by a significant margin. Furthermore, we evaluated precision, recall, and F1-score to evaluate the model's effectiveness on individual disease classes. To give an additional understanding of the model's behavior and its capacity to distinguish between various disease types, we utilized visualization techniques such as confusion matrices and sample output images. The results of this study have implications for managing potato diseases by offering an automated and reliable solution for early detection and diagnosis. Future research directions may include expanding the dataset, exploring different CNN architectures, and investigating the generalizability of the model across different potato varieties and growing conditions.