Authors: Getnet Tigabie Askale, Achenef Behulu Yibel, Belayneh Matebie Taye, Gashaw Desalegn Wubneh
Categories: Comment, Maize leaf disease detection, CNNs for detection, Model comparison, VGG16, Mobile application for maize leaf disease detection
Source: Plant Methods
Authors: Getnet Tigabie Askale, Achenef Behulu Yibel, Belayneh Matebie Taye, Gashaw Desalegn Wubneh
Maize is the most produced crop in the world, exceeding wheat and rice production. However, its yield is often affected by various leaf diseases. Early identification of maize leaf disease through easily accessible tool is required to increase the yield of maize. Recently, researchers have attempted to detect and classify maize leaf diseases using Deep Learning algorithms. However, to the best of the researcher’s knowledge, nearly all the studies are concentrated on developing an offline model that can detect maize diseases. But, those models are not easily accessible to individuals and don’t provide immediate feedback and monitoring. Thus, in this study, we developed a novel real-time, user-friendly maize leaf disease detection and classification mobile application. The VGG16, AlexNet, and ResNet50 models were implemented and compared their performance on maize disease detection and classification. A total of 4188 images of blight, common_rust, grey_leaf_spot, and healthy were used to train each model. Data augmentation techniques were applied to the dataset to increase the size of the dataset, which can also reduce model overfitting. Weighted cross-entropy loss was also employed to mitigate class-imbalance problems. After training, VGG16 achieved 95% of testing accuracy, AlexNet achieved 91%, and ResNet50 achieved 72% of testing accuracy. The VGG16 model outperformed the other models in terms of accuracy. Consequently, we deployed the VGG16 model into a mobile application to provide real-time disease detection and classification tool for farmers, extension officers, agribusiness managers, and policy-makers. The developed application will enhance early disease detection, decision making, and contribute to better crop management and food security.
The number of people worldwide is expected to be almost 10 billion in 2050 driving the farm and food system [1]. However, degradation of land, water resources, climate change, and food losses limit agricultural production [1]. Maize is the most produced crop in the world, exceeding wheat and rice production. It was also used to produce beverages, and cattle fee. In Ethiopia, it was the most widely grown crop from lowland to highland agro-ecologies. Maize is grown more frequently than any other crop by farmers and contributes the most to the nation’s crop production among the cereal crops. From 2007 to 2017 the area used for maize cultivation increased by almost 60% in Sub-Saharan Africa. Ethiopia is the only Sub-Saharan Africa country which has significant progress in maize production and input usage. Maize is one of the main sources of calories in major maize production regions and it is the primary food crop in Ethiopia. Maize is grown on approximately 2.135 million hectares of land across the country.
However, Due to the degradation of agricultural land and change in cultivation systems, the number of maize diseases and the degree of harm they cause have increased. In addition, due to socioeconomic, biotic and abiotic restrictions maize yields have remained low. In Ethiopia, the most prevalent biotic restrictions on maize production are weeds, diseases, insects and other arthropod pests, followed by nutrient deficits (abiotic) and input availability and market price volatility (socioeconomic) restraints [2]. The main biotic factors limiting maize production and productivity are diseases. Turcicum leaf blight, Grey_leaf_spot and common leaf rust are the most prevalent infectious diseases of the maize in the region. Common rust which is a devastating disease that causes yield losses of up to 61% in disease epidemic years in the country’s key maize growing highland areas. Common rust found in subtropical and tropical agricultural zones as well as in temperate. In the world, Grey_leaf_spot is recognized as one of the most severe maize yield limiting diseases. It also became a pandemic throughout Africa. It was originally discovered in Ethiopia in 1997 along the border of the west wellega and western Ethiopia in illibabur zones. Across the world, a serious foliar disease of maize in the majority of producing locations is called Leaf blight [2].
Thus, an early identification and diagnosis of maize diseases is required to ensure high yield, high quality and high efficiency of maize [3, 4]. Traditionally, manual inspection is used as a method of maize disease and it is more time-consuming, and requires expert knowledge. Currently, with the advancement of deep learning technologies, automated disease detection is implemented using convolutional neural networks. High resolution images of maize leaves can be analyzed by CNN based models to identify diseases such as blight, common rust, and leaf spot. CNN models can achieve high accuracy in detection and classification. Compared to classical methods, deep learning based models offer considerable progress in plant disease detection and they robust when trained with large and high quality datasets [5]. Convolutional Neural Networks (CNN), Vision Transformer (ViT), and U-Net Deep learning models have been used in numerous field for accurate segmentation and feature localization [6, 7]. Effective disease diagnosis has been conducted through the application of artificial neural networks, CNNs, and ViTs models [8]. Vision transformer models have been achieved exceptional accuracy and speed in detecting maize leaves [9, 10]. However, most convolutional neural network models are designed to work offline and they are not appropriate for real-time disease detection [11]. Integrating CNNs model into mobile applications can enable farmers to obtain immediate and accurate feedback in the field. Mobile based CNN model solutions can enhance accessibility, promote early disease intervention, reduce dependency on experts and ultimately improve crop yield and food security. However, to the best of the researcher knowledge, fewer attempts have been made to detection and classify maize leaf disease in real-time [11–13]. Faiza et al. [12] developed a model to detect and classify three maize leaf diseases named Blight, Sugarcane Mosaic virus, and Leaf Spot but it neglect common_rust and healthy maize leaf conditions. A small set of images (1820 in total) was used by [13] to classify blight, common_rust, grey_leaf_spot, and healthy maize diseases, but it achieved lower accuracy (93.53%). Thus, in this study we proposed a real-time mobile based CNN model for detection and classification of maize leaf diseases into blight, common_rust, grey_leaf_spot, and healthy with higher accuracy. The primary contributions of this study are outlined We developed a novel mobile based deep CNN model for classifying maize diseasesWe applied data augmentation technique to improve the classification performanceWe compare different CNN model performance on Maize leaf disease classification and select the best performed to develop mobile appWe deployed CNN model on android studio and developed real-time maize detection systemWe evaluate the proposed model showing promising result
Various attempts have been made to detect and classify maize diseases. A maize leaf disease prediction model from the image of affected maize leaf [3] employs a deep learning approach to detect and classify maize leaf diseases. Erik et al. [1] assessed the performance of three state-of-the-art CNN models (AlexNet, ResNet50, and SqueezeNet) to classify maize diseases into Grey_leaf_spot, common rust, leaf bight, and healthy. The researcher employs Bayesian hyperparameter optimization, fine-tuning strategies, data augmentation, and five-fold cross validation procedures methods to evaluate the model performance and achieved 97% of accuracy for all models. A YOLOv5n based model proposed by [14] attempted to classify common maize leaf spot, grey spot, and rust diseases. Coordinate attention is incorporated in the backbone of YOLOv5n to improve the model accuracy and achieved 94.5% of accuracy.
However, a real time maize disease detection and classification model that can be applied to mobile devices is required [15]. Harry et al. [15]. Employs adaptive thresholding to quantify maize leaf diseases. The researchers train deep learning models, compare their results, develop adaptive thresholding techniques and use regions of interest to estimate image leaf disease severity and achieve 99% of accuracy. Fathimathul et al. [16] can effectively classify common rust, blight, and the northern leaf grey spot using EfficientNet. The researcher employed preprocessing such as resizing images, and data augmentation, and classification and achieved 98.5% of accuracy. However, increasing the size of the data to improve the model accuracy, making the model more versatile by introducing more plant species, and development of a mobile application which is more accessible and user-friendly and farmers can use it in the field to recognize and identify disease in the real time is required [16]. Mobile-DANet model proposed by [4] can successfully identify maize diseases with an accuracy of 98.5% and pointed out deploying the model on portable devices for real-time disease identification is required. ResNet-18, VGG16, and EfficientNet models were employed by [17] to classify maize disease and achieved 97.1% of accuracy. A weighted cross-entropy loss method is used to mitigate class imbalance [17].
Seven convolutional neural network AlexNet, VGG16, VGG19, GoogleNet, InceptionV3, ResNet50 and ResNet101 were tested by [18] to classify maize leaf diseases into healthy, cercospora, common rust, and northern leaf blight. K- Support vector machine, and K-nearest neighbor machine learning classification methods also used to compare the classification performance and AlexNet and Support vector machine scored best result [18]. Android-based recognition tool developed from pre-trained convolutional neural network for recognizing corn diseases [13] can classify common rust, grey_leaf_spot, northern leaf spot, and healthy with an accuracy of 93.4%. An intelligent maize leaf disease detection developed from Manta-Ray Foraging Optimization with convolutional deep learning [19] used median filter for image preprocessing, DenseNet for feature extraction, and MRFO for hyperparameter tuning for maize leaf disease classification. Northern leaf blight, Common rust, and Cercospora leaf spot grey maize leaf diseases were detected using a convolutional neural network method [20]. The model was developed from publicly available 4000 images and is implemented on MATLAB environment and achieved 96.53% of accuracy [20]. VGGNET, InceptionV3, ResNet50, and InceptionResNetV2 deep learning models employed by [21] to detect maize leaf disease have achieved a maximum accuracy of 87.51% with the ResNet50 model. Faiza et al. [12] Proposed smartphone based maize leaf disease detection model to detect three maize leaf diseases employed YOLOvn architecture for real-time classification, segmentation, and tracking system. A maize leaf disease identification model [22] employed a convolutional block attention module (CBAM) into the auto-encoder. Discrete wavelet transformation used for dimensionality reduction, CBAM to deduce the attention diagram of feature maps and encoder for prediction. To address image quality, disease similarity, diverse severity, and limited dataset, Liangliang et al. [23] proposed a multi-scale feature fusion neural network (MResNet) for maize leaf disease classification. Two residual subnets of MResNet allowed the model to detect disease at different scale and improve the performance of deep neural networks. To overcome disease-related phenotypes in germplasm resource particularly in actual field environments, Fei et al. [24] Employed RGB-D post-segmentation with Resnet50, MobilenetV2, Vgg16, and Efficientnet-B3 model to classify three maize leaf diseases. A fine-tuned EfficientNet model developed by [25] attempted to classify maize disease with an accuracy of 98.52%. The details of the related work summary are presented in Table 1 below.Table 1Summary of related workAuthorYearMethodologyLimitationDataset sizeDetection network frameworkFaiza et al. [12]20232675Mobile app based on YOLOv5nRelatively dataset size was smallOnly three classes were identifiedNeglect common_rust disease type in their studyLi et al. [14]20234353YOLOv5nOffline modelRequire deployment to mobile applicationOnly three classes were identifiedQiwen et al. [15]20221162CNNRelatively dataset size was smallDoesn’t support real-time detectionFathimathul et al.[16]20233188EfficientNetRelatively dataset size was smallDoesn’t support real-time identificationKrisnanda et al. [17]20244188ResNet-18, VGG16, and EfficientNetDoesn’t support real-time detectionShanmugam and Ramavel [19]20247316Manta-Ray Foraging Optimization with Deep LearningDoesn’t support real-time identificationLimited applicabilityKifayat et al.[20]20214000CNNRelatively dataset size was smallLimited applicabilityDoesn’t support real-time identificationImran et al. [21]20249052VGGNET, Inception V3, ResNet50, and InceptionResNetV2Limited accuracy (87.51%)Doesn’t support real-time identificationShaodong et al. [22]20233852Convolutional Block Attention Module and auto-encoder light weightRelatively dataset size was smallDoesn’t support real-time detectionLiangliang et al. [23]20244188Residual-based multi-scale networkDoesn’t support real-time detectionFei et al. [24]20231200Resnet50, MobilenetV2, Vgg16, and Efficientnet-B3Relatively dataset size was smallDoesn’t support real-time detection
Most of the researchers are concentrated on developing maize disease detection using convolutional neural network models. However, a real time maize disease detection model that can be easily accessible to farmers and can enable them to monitor and get immediate feedback is required.
The proposed methodology encompasses data acquisition, data preprocessing, proposed model design and selection, and mobile app development. The entire methodology that we follow is depicted by the following figure (Fig. 1).Fig. 1Proposed model architecture
A total of 4188 color (RGB) images were collected from Kaggle1 repository and the dataset is a combination of PlantVillage and PlantDoc [17, 23]. PlantVillage contain over 54,000 images collected from plantation and gardens. Information about plant species and type of disease are presented in the image from diseased and healthy leaves, stems, and fruits of plants. PlantDoc, plant image dataset contains over 87,000 images of crops affected by different diseases. Images of maize leaf disease which are relevant for this study is selected. Finally, the dataset comprises four classes which contains 1146 Blight images, 1306 common_rust images, 574 Grey_leaf_spot images, and 1162 healthy images. The details of maize disease dataset utilized in this study are presented in Table 2 below.Table 2Detail of maize disease dataset utilized in this studyNoTypeNumber (size)1Blight11462common_rust13063Grey_leaf_spot5744Healthy1162Total4188
The following diagram depicts the number of images in each class and their proportion (Fig. 2).Fig. 2A Number of images in each class using bar chart and B Proportion of images in each class using pie chart
The bar chart revealed that the common_rust has the highest number of images, while significantly fewer images are observed in Grey_leaf_spot. Similarly, the pie chart presents the proportion of images in each class, highlighting common_rust has the largest share at 31.2%, followed by healthy class (22.7%), and Blight class (27.4%). While the least share observed on Grey_leaf_spot at 13.7% (Fig. 3).Fig. 3Sample images in each class
To enhance the classification performance of the model and to increase the quality of the data we have applied a data preprocessing technique to the data. We have divided the data into training, validation, and testing to ensure a structured approach for model training and evaluation. The images were resized into 224 X 224 pixel and Image preprocessing and data augmentation is applied by utilizing ImageDataGenerator which helps in preventing overfitting and improving generalization. For training dataset, rescaling, width and height shifting, rotation, and horizontal flipping augmentation techniques are applied. We normalized image pixel values to the range [0, 1] to improve convergence during training. Random rotates the image by up to 20 degrees to increase variability. Ensuring the model is robust to positional changes by width and height shifting. We also diversified the training data by mirroring images with horizontal flipping. For validation and testing images, to maintain consistency in pixel value we applied rescaling and to ensure consistent evaluation and to maintain order Shuffling is disabled. Some classes possess more data than others and the maize leaf disease dataset exhibits class imbalance distribution. Thus, to overcome this imbalance, we employed the weighted cross-entropy loss method [26].
In this step, we have trained three deep CNN architectures for developing maize disease detection models and these architectures are AlexNet, VGG16, and ResNet50. Due to their success in disease detection and classification, VGG16, AlexNet, and ResNet50 models were selected for this study. These models have been showing excellent performance in plant disease classification and are extensively used in agriculture sector [27, 28]. These models are suitable to fine-tune to domain specific dataset and provide a strong benchmark for comparison with newer architectures MobileNet and EfficientNet [29]. MobileNet and EfficientNet architectures are designed for low computational efficiency [30]. However, VGG16, AlexNet, and ResNet50 models are well suited for disease detection where computational resources are not limited. Thus, the goal of this study is to achieve highest accuracy over computational efficiency and we choose these models by their established success on plant disease detection [31, 32]. Each model was trained for 50 epochs by applying an Adam optimizer, learning rate of 0.0001, Categorical cross entropy, and early stopping.
Pertained AlexNet Model was proposed by Alex Krizhevsk in 2012 and has 25 layers consisting of input, convolutional, ReLU, Normalization, max-pooling, fully connected, dropout, and output layers [25, 33–35]. It consists of five convolutional layers with filter sizes, seven ReLU layers, two normalization layers, three max-pooling layers, three fully connected layers, two dropout layers, Softmax activation, and an output layer [25, 33]. A size of 227 X 227 image is recommended for this model [36]. Each convolutional layer consists of a set of filters called convolutional kernels. Kernel is a matrix of integers that slide over the input image to produce the output feature map by multiplying its weight with the corresponding value of subset pixels of the input image. Mathematically the convolution process can be expressed as [37]:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {fl}^{k }\left(p,q\right)={\sum }{c} {\sum }{x,y}ic \left(x,y\right).{el}^{k}\left(u,v\right)
\usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ R(z)=\text{max}(0,z) $$\end{document}R(z)=max(0,z)where z is the function input and R (z) is the function output which equal to the input when the input is positive and equal to zero otherwise. The final layer of the AlexNet architecture is the output layer and it uses Softmax activation function and mathematically expressed as [37]:3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \text{S}(\text{yi})=\frac{exp(yi)}{\sum_{j}^{n}=1\text{exp}(yj)} $$\end{document}S(yi)=exp(yi)∑jn=1exp(yj)where y^i^ is the i^th^ element of the input vector, n is the number of classes. Finally, after building the model, the AlexNet has 222.39 MB 58, 297, 732 trainable parameters. #### VGG16 VGG16, known as a convolutional neural network, was developed by Oxford researchers and ranked second in the 2014 ImageNet Large Scale Image Recognition Competition (ILSVRC) [24, 38]. It is known for its depth and it consists of 16 layers including thirteen convolutional layers, and three fully connected layers [17, 35]. To extract features, these layers perform convolutional operations on the input image. Primarily a size of small 3 × 3 kernels (filters) used by the convolutional layers enables the model to intricate patterns in the data and are often followed by ReLU activation function. A max-pooling layer with 2 × 2 filters follow most convolutional layers. Towards the end of the network, VGG16 has three fully connected layers enabling the model to make final predictions based on the extracted features [24, 26, 38]. VGG16 model has a powerful contribution to image recognition tasks [24]. Finally, the VGG16 model has 56.38 MB size with a total of 14,780, 866 total trainable parameters (Fig. 4).Fig. 4VGG16 architecture #### ResNet50 ResNet50 is a deep residual network proposed by the Microsoft Asia Research Institute [24] that presents a significant advancement in the application of plant leaf disease detection [38]. Due to its residual learning approach, ResNet50 is a widely used convolutional neural network for addressing vanishing gradient problems in deep learning networks. It makes the deeper network to be more trainable by incorporating shortcut connections and effectively learns identity mappings. The ResNet50 architecture consists of fifteen convolutional layers, structured into residual blocks, each containing shortcut connections and stacked convolutional layers. Initially, it extracted features using 7 × 7 convolutional layers and pooling layers and for deeper feature extraction followed by 16 residual modules. It uses 3 × 3 filters to capture fine-grained image details and a recommended input standard is a size of 224 × 224 [24, 38] (Fig. 5).Fig. 5ResNet50 pre-trained model The residual block includes a skip connection that bypasses one or more layers, allowing the gradient to flow directly through the network making it to train very deep networks. Mathematically the residual block can be expressed as [39]:4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ y=F\left(x, \left\{Wi\right\}\right)+x $$\end{document}y=Fx,Wi+xwhere \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ F\left(x, \left\{Wi\right\}\right) $$\end{document}Fx,Wi is the residual function, x the input to the residual block, and y is the output. The following figure illustrates residual learning of a feed forward neural network with shortcut connections (Fig. 6).Fig. 6Residual learning building block diagram Multiple layers can compose the residual function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ F\left(x, \left\{Wi\right\}\right) $$\end{document}Fx,Wi. For example, it might consist of two convolutional layers with batch normalization and ReLU activation in a basic residual block.5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ F\left(x, \left\{Wi\right\}\right)=W2.\sigma \left(W1.x+b1\right)+b2 $$\end{document}Fx,Wi=W2.σW1.x+b1+b2where W1 and W2 are convolutional layers weight matrices, b1 and b2 are bias terms and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \sigma $$\end{document}σ is the ReLU activation function. Within the residual blocks, batch normalization is applied after the convolutional layers and before the activation function to accelerate and stabilize training. Mathematically it can be represented as follows [40]:6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \widehat{x}=\frac{x-\mu B}{\sqrt{{a}^{2}B+\varepsilon }} $$\end{document}x^=x-μBa2B+ε7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ y=y\widehat{x}+\beta $$\end{document}y=yx^+βwhere \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \mu B $$\end{document}μB and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {a}^{2}B $$\end{document}a2B are the mean and variance of the mini-batch, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \varepsilon $$\end{document}ε is a small constant for numerical stability, and y and β are learnable parameters that scale and shift the normalized output. Finally, the ResNet50 model has 90.98 MB size with a total of 23,850,868 parameters, where 262,788 trainable parameters and 23,587,712 non-trainable parameters. ## Experiment In this step we have trained the tree pre-trained architectures (AlexNet, VGG16, and ResNet50) models on maize leaf disease images. Then, we build the model and evaluate their performance using testing images. Finally, we developed a mobile application using the model which outperforms than the other. ### Environmental setup The model is implemented on a single 12 GB NAVIDA Tesla k80 Graphics processing unit, GPU provided by Google Colab using python programming language [41–43]. We divide the input data into 75% training, 15% validation, and 10% testing results in 3139 for training, 626 for validation, and 423 for testing images. For training, ImageDataGenerator is employed to process the dataset and the training images are augmented through rotation, horizontal flips, and width and height shifts for better generalization. Images are only rescaled to 0 and 1 to normalize the pixel value in validation. Adam optimizer, Learning rate of 0.0001 and Categorical cross entropy is used to compile the model for making it suitable for classifying multi-class maize leaf disease classification. During training, early stopping is applied to stop the training if the validation loss stops improving for five consecutive epochs and saves best performing model using model check point. Each model is separately trained for 50 epochs with validation data to monitor performance. After training, the model is tested using testing data and saved for future use. ### Evaluation metrics Separate evaluation to each model is applied and to evaluate the proposed model, we have used standard performance measure metrics such as precision, recall, F1-score, accuracy, and confusion matrix. These evaluation metrics are well suited for classification tasks and are relevant for multi class classification problems [44]. Based on studies of [43, 45, 46] the mathematical formula and the description of precision, recall, f1-score, and accuracy are given below. Recall measures the ability of a classification model to correctly identify all relevant instances of a particular class within a dataset. Mathematically recall is given in Eq. (8).8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \text{Recall}=\frac{True positives}{True Positives+False Negatives} $$\end{document}Recall=TruepositivesTruePositives+FalseNegatives Precision measures the accuracy of the positive predictions made by a classification model. Precision is calculated as the ratio of the number of true positive predictions to the sum of true positive and false positive predictions. Mathematically given precision is given in Eq. (9).9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ Precision=\frac{True positives}{True Positives+False Positives} $$\end{document}Precision=TruepositivesTruePositives+FalsePositives The F1-score can be interpreted as a weighted harmonic mean of the precision and recall and is given in Eq. (10).10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ F1-score=\frac{2*(precision*recall)}{precision+recall} $$\end{document}F1-score=2∗(precision∗recall)precision+recall Accuracy is the amount of correct predictions made compared to the total number of predictions made. Mathematically given in Eq. (11)11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ Accuracy=\frac{True positives+True Negatives}{True Positives+False Positives+False Negatives+True Negatives} $$\end{document}Accuracy=Truepositives+TrueNegativesTruePositives+FalsePositives+FalseNegatives+TrueNegatives In addition, the confusion matrix is implemented to assess the proposed model performance. The confusion matrix provides a comprehensive representation of the correctly classified TP values, FP values that are categorized in the wrong class, false negative (FN) values that belong to the incorrect class, and correctly classified TN values in the other class [43]. Hence, we have used a confusion matrix as a performance measure, which is used to describe the performance of a classification model on the set of test data for which the true values are unknown. In addition to commonly used performance metrics, classification reports, and confusion matrices, we have used confidence interval evaluation metrics to evaluate the proposed method. The confidence interval evaluation metrics offer a statistical range within which the true performance of the model is likely to fall by accounting for uncertainty or variability in the evaluation process [47]. ### Experimental result In this step, we evaluate the performance of each model separately to choose the best performed model. As the figure below illustrates the AlexNet performance on maize leaf disease classification task. The training and validation accuracy plot revealed that there is a consistent upward trend with both training and validation curves converging around 90% of accuracy after 20 epochs which indicates the model is learning effectively without overfitting. The training and validation loss decreases steadily confirming that the model is optimizing well. The classification report also shows the precision, recall, F1-score, and support for each class. It revealed the model is well performed with an overall accuracy of 91%. Healthy class has highest precision, recall and F1-score of 1.00 revealed that the model can correctly classify healthy maize leaf with no misclassification. The model also performs well on the Common_rust class with a 98% precision and 92% recall. However, the model struggles to correctly identify all instances with Grey_leaf_spot leaf diseases and has the lowest recall at 69% Fig. 7. The details of the classification report for the AlexNet model are presented in Table 3 below.Fig. 7AlexNet performance on maize disease classificationTable 3Classification report of AlexNetClassesTesting resultPrecisionRecallF1-scoreSupportBlight0.810.930.87116Common_rust0.980.920.95132Grey_leaf_spot0.820.690.7558Healthy1.001.001.00117Accuracy0.91423Macro avg0.900.880.89423Weighted avg0.920.920.91423 The following figure below depicts the performance of VGG16 on classification of Maize leaf diseases. The training and validation accuracy and loss plot figure below revealed that both the training and validation accuracy steadily increase indicating the model is learning effectively on maize disease detection. The validation accuracy aligns and follows closely to the training accuracy, indicating the model faces minimal overfitting. Both the training and validation loss also has sharp decline confirming the model has effectively converged. On the other hand, the classification report indicates the model performs very well on maize leaf disease classification with an overall accuracy of 95%. Based on the class wise performance the model can accurately classify healthy classes without any misclassification with precision (1.00), recall (1.00), and F1-score (1.00). The model also can correctly classify common_rust with a high precision (0.97), recall (1.00), and f1-score (0.99). It can also correctly classify blight class with high precision (0.93), recall (0.91), and f1-score (0.92) with minor misclassification. However, the model struggles to classify Grey_leaf_spot and has some misclassification because of less training dataset Fig. 8. The details of the classification report for the VGG16 model are presented in Table 4 below.Fig. 8VGG16 performance on maize disease classificationTable 4Classification report of VGG16ClassesTesting resultPrecisionRecallF1-scoreSupportBlight0.930.910.92116Common_rust0.971.000.99132Grey_leaf_spot0.840.830.8358Healthy1.001.001.00117Accuracy0.95423Macro avg0.940.930.93423Weighted avg0.950.950.95423 The training accuracy, training loss, validation accuracy, and validation loss plot of ResNet50 depicted by the figure below indicates the performance of ResNet50 on maize disease detection. The plot indicates a steady learning process over 45 epochs. Initially, the training accuracy starts at 30% and gradually increases to 65%, while the validation accuracy shows noticeable fluctuations and reaches 72%. The fluctuation indicates some sensitivity to batch variations but doesn’t indicate severe overfitting. On the other hand, the training and validation loss shows a consistent decrease indicating effective learning. The validation loss is lower than the training loss revealing the model has good generalization on maize disease detection. In addition, the classification report indicates the model performs quite well on maize leaf disease classification with an overall accuracy of 72%. Based on the class wise performance the model can classify Common_rust class with less misclassification than the other classes with precision (0.89), recall (0.93), and F1-score (0.91). The model also can correctly classify healthy class with a moderate precision (0.72), recall (0.79), and f1-score (0.75). However, the model struggles in identifying Blight and Grey_leaf_spot classes. This happens due to the ResNet50 model having a deeper architecture than AlexNet and VGG16 and requires a large amount of data to train the model. But, the number of the training data is small as compared to the depth of the model and the model get challenging in identification of Blight and Grey_leaf_spot classes Fig. 9. The details of the classification report for the ResNet50 model are presented in Table 5 below.Fig. 9ResNet50 performance on maize disease classificationTable 5Classification report of ResNet50ClassesTesting ResultPrecisionRecallF1-scoreSupportBlight0.560.610.59116Common_rust0.890.930.91132Grey_leaf_spot0.550.280.3758Healthy0.720.790.75117Accuracy0.72423Macro avg0.680.650.65423Weighted avg0.710.720.70423 We have also utilized a confidence interval evaluation metric for each model, and we made a quantitative comparison between AlexNet, ResNet50, and VGG16 models to reveal their performance accuracy and reliability. Based on the test result, the VGG16 model outperforms all other models, achieving 91.25% testing accuracy with a confidence interval of 95%, ranging from 88.18% to 93.85%. Thus, VGG16 has a narrow confidence interval and reveals it has strong generalization on the test set and has consistent results across samples. The AlexNet model also has a competitive testing accuracy of 90.78% with a confidence interval of 87.94% to 93.62% but has a slightly lower accuracy than VGG16. On the other hand, the ResNet50 model achieved a much lower confidence interval of 24.82% to 33.34% and underperformed all other models, which reveals ResNet fails to generalize effectively. We also conducted an experiment to demonstrate the effect of data augmentation on each model. Without data augmentation, the models exhibited overfitting, where they performed well on the training data but failed to generalize to unseen data. However, when data augmentation techniques were applied to the dataset, the training and validation accuracies aligned more closely, indicating a significant reduction in overfitting. The detailed comparison of model performance is presented in Table 6 below.Table 6Detail performance evaluation of each model with and without data augmentationModelTechniques appliedResultTraining accuracyValidation accuracyTraining lossValidation lossTesting accuracyAlexNetWithout data augmentation0.970.880.080.370.90With data augmentation0.880.880.250.260.91VGG16Without data augmentation0.990.940.00280.280.91With data augmentation0.960.950.110.14**0.95**ResNet50Without data augmentation0.860.660.320.830.69With data augmentation0.640.690.860.770.72Bold indicates the proposed model achieves superior results compared to others We also conducted an experiment to assess the robustness and priorities of the proposed method using K-fold cross-validation. K-fold cross-validation is a technique used to evaluate whether a model is functioning properly [48, 49]. Accordingly, we performed a fivefold cross-validation assessment for each model. Among them, the VGG16 model outperformed all others in terms of accuracy. The results of the fivefold cross-validation comparison are presented in Table 7 below.Table 7fivefold cross validation performance evaluation resultFoldModelTraining accuracyValidation accuracyTraining lossValidation lossTesting accuracy1VGG160.800.820.880.420.8220.540.700.970.770.7030.800.810.400.500.8140.840.810.330.340.8150.860.840.400.390.841ResNet500.360.480.941.060.4820.570.591.030.980.5930.570.641.021.060.6440.550.531.061.090.5350.550.591.091.020.591AlexNet0.560.530.890.840.5320.920.840.190.380.8430.580.680.800.900.6840.490.520.880.880.5250.550.581.030.810.58 Based on the above performance evaluation tables, the VGG16 model outperforms all other models in classification accuracy on maize leaf disease classification. ResNet50 underperformed compared to others. This may due to it has deep architecture (complex architecture) and the size of the dataset is not enough to train the model, default settings such as batch size, learning rate, optimizer which are good for other models are not good for ResNet50 [25]. Thus, in the best performing model step we have selected the VGG16 model to deploy into a mobile application that can be easily accessible and allow immediate feedback in maize disease identification to the farmer. ### Comparison with other state-of-the-art models To evaluate the effectiveness of the proposed mobile based deep CNN model for maize leaf disease detection and classification, we compared it with several established state-of-the art models in the domain by making a compressive comparative analysis. We compared our model with recent widely adopted deep learning architectures such as VGG16, VGG19, ResNet50, InceptionV3, EfficientNet-B0, MobileNet, YOLOvn, DenseNet, and Vision Transformer (ViTs), which have shown a remarkable performance on maize leaf disease detection and classification tasks. As shown in Table 5 below, our model outperforms these methods in terms of accuracy evaluation metrics. We have made the base model trainable true which allows us to fine-tune the pre-trained VGG16 architecture and enable it to adopt unique characteristics of our dataset. Consequently, this enables the model to learn specific features and to adjust their filter for representation of the features results in improved accuracy. Our finding demonstrated that the proposed method exceeds the performance of the current methods and become a promising tool for accurate maize leaf health monitoring. The details of the comparison with other state-of-the-art models are presented in Table 8 below.Table 8Comparison with other state-of-the-art modelsAuthorsMethods usedAccuracyMaria et al. [50]VGG16 as a classifierLayer-wise Relevance Propagation (LRP) as model’s transparency enhancement94.67%Jie et al. [51]Convolutional block attention module (CBAM) with the ResNet50 backbone network to reduce complex background noise and boost key featuresMulti-scale feature fusion module to aggregate local and global informationCenterNet target algorithm for detection85.13%Mahajan et al. [52]VGG16VGG19ResNet50InceptionV3EfficientNet-B070.92%91.37%89.69%87.77%91.33%Tong et al. [53]YOLO-Leaf93.88%Ang et al. [54]MobileNet V3-small79.2%Pei et al. [55]YOLOv894.6%Ramadan et al. [56]DenseNet121VGG16ResNet152V2InceptionV3Vision Transformer (ViT-B/16)Vision Transformer (ViT-B/32)Vision Transformer (ViT-L/16)Vision Transformer (ViT-L/32)92.72%89.02%92.24%92.48%94.51%91.29%93.68%92.36%Imran et al. [21]ResNet5087.51%Arabinda et al. [57]DenseNet201 for feature extractionSVM for classification94.6%Abu et al. [58]VGG16VGG19InceptionV3ResNet50MobileNetV2VGG16 + SEResNet50 + SE91.77%91.67%91.68%78.44%94.76%93.44%81.77%**Proposed method****VGG16 based mobile application** **95%**Bold indicates the proposed model achieves superior results compared to others ### Model deployment into mobile application In this step, the outperformed model is selected and deployed to a mobile application. Firstly, we converted the VGG16 model saved in the.h5 format into TensorFlow lite format which is optimized for mobile and embedded devices [59]. Then, we load the pre-trained TensorFlow lite model (VGG16) in the android application. The images which are formatted to match the model’s expected input size and data type are given to the model as input in the form of TensorBuffer and images are resized into 224 × 224 pixel size. The input images can be either browsed from saved files or captured using the mobile device. Thus, we have designed a user interface for the mobile which enables the user to capture maize leaf or to browse images of maize leaf from their local repository. After the input image is given and processed, the model presents a set of class probabilities, which are interpreted to determine the maize leaf status; Healthy, Grey_leaf_spot, Common_rust, and blight. The detected result along with their confidence score is displayed to the user. We used android studio to develop and design maize leaf disease detection mobile application interface and Samsung M11(SM-SM115F) mobile device which have 3 GB Ram Size, and 32 GB storage as emulator. To deploy APK (android application package) on smartphones using the SDK (android software development kit) we follow the following activate the phone developer mode, and enable USB debugging. Additionally, the process involved tapping the build number four times, establishing a USB connection between the phone and the computer and gaining access to the application through Android Studio. Once this setup was complete, the application could be accessed directly on the smartphone from Android Studio [59]. #### Mobile app evaluation To evaluate the developed mobile app maize leaf detector app, first the user must install the mobile application to their mobile then after, the user open the application and select image either from the gallery or live from camera. The embedded VGG16 model detect the maize leaf condition and detect the leaf condition into Blight, common _rust, grey_leaf_spot, and healthy conditions. The following figure below depicts the proposed mobile application working flow (Fig. 10, 11).Fig. 10the proposed mobile application working flow diagramFig. 11**A**, **B**, **C**, **D** Sample maize leaf disease detection using the mobile app The application shown in the attached image below, indicating the model is capable of effectively identifying maize leaf diseases. We have tested the app using testing data and the app successfully detects each class with excellent performance. The app can successfully classify the single input image as Common_rust with a high confidence level of 100%, Healthy with a high confidence level of 99.57%, Blight with a high confidence level of 98.69%, and Grey_leaf_spot with a high confidence level of 97.40%. The user interface is simple to use and more user friendly allowing the user to either capture an image using the camera or select an image from the gallery for detection. The following figure depicts sample evaluation of the mobile app on testing images. The time it takes to make prediction on a single image can be measured by inference time. Because of excessive delay can lead the machine being unable to respond in time and inference times are a crucial performance evaluation metrics for real-time application. The inference time was calculated by running the model on asset of maize leaf images and averaging the time over the number of images [60]. Using both live capture and gallery selection, the inference time was measured across testing images. Depending on device background activity, light variation, image quality, and image resolution, the proposed model achieved an inference time approximately 850 ms (ms) per image. Despite the size of the model (56.38 MB) the proposed model is capable of offering responsive performance suitable for real-time maize disease detection on modern mobile device. ## Conclusion and future work Recently, deep learning algorithm has gain significant attention in pattern recognition and image analysis. Deep learning algorithms are used to develop a system that can help identifying and diagnosis plant leaf disease accurately. They are also incorporated into mobile application for detecting and classifying leaf disease in real time. Thus, a time consuming and prone to error manual inspection can be reduce. Therefore, in this study, we proposed a mobile based maize leaf disease detection and classification which can successfully classify maize diseases into healthy, common_rust, blight, and Grey_leaf_spot. The study experiments three deep learning algorithms (AlexNet, VGG16, and ResNet50) to develop a model that can be deployed into mobile applications. To develop the model the dataset is partitioned into training, validation, and testing. The data augmentation is applied during training and these three models are trained separately on training dataset and assessed their performance separately on testing data. Based on the performance of the model on the testing dataset, VGG16 outperforms all other models in terms of classification accuracy and robustness of the model. The VGG16 outperforms AlexNet by 0.04%, and ResNet50 by 0. 23%. Thus, we deployed the VGG16 model into a mobile application and the application has strong capability in detecting healthy, common_rust, Grey_leaf_spot, and blight maize leaf status. The developed application will offer several advantages for farmers and agricultural stakeholders. The application can be easily accessible to the farmers by their smartphone without the need for expensive laboratory tests and expert consultation. It also enables the farmers for real-time monitoring for maize leaf disease identification, to get immediate feedback, early intervention, and reduce further spreading. The proposed approach has the following limitations; small amount of dataset is used for this study and concentrated on detecting and classifying only blight, common_rust, grey_leaf_spot, and healthy maize leaf conditions, and need to include other plant species to validate its effectiveness in identifying other plant leaf conditions. The generalization of the model’s capability may be affected by the small amount of dataset used and it use may be limited in diverse range and plant conditions. Additionally, the proposed mobile app may not detect unknown leaf conditions not included in the dataset. In the term of future direction, a more amount of dataset will be used to improve the performance of the model. A general model that can detect most plant diseases types will be developed. In addition, environmental data and other features will be included in the training process to further boost the performance of the proposed model. Furthermore, ensemble model will boost the performance of the model and we will implement the ensemble model as a mobile application.