ABSTRACT

In this paper, a new approach is suggested to grade apples based on colour and size. Two different feature extraction methods are developed using image processing techniques in a machine vision environment. The extracted feature set is used to train and test a Back propagation neural network. The graded results are compared with manually graded results.

(Note: This Paper was presented in ICSIP Please Download the paper for proper formatting, images and symbols)

KEY WORDS

Image Processing, Size and color, Grading, Feature Extraction, Back Propagation Neural Network

1. Introduction

The Quality of fruits refers to the classical inherent features and their visual appearance. The uniformity in size, shape and colour are important features that decide the visual appearance of a fruit. A visually appealing fruit always fetches highest price in the market. Hence, it is necessary to grade fruits based upon their visual appearance. The high cost, tediousness, subjectivity and inconsistency associated with the usual manual grading method have been forcing the post harvest industry to apply automation in grading operations.

However, applying automation in grading fruits is not as simple as automating an industrial product. In industry, the manufactured parts are well defined and uniform. Techniques like template matching and fixed object modeling can be used in such situations. These methods are unlikely to produce good results in the grading of fruits due to their morphological diversity and irregularity. Therefore, self-learning techniques such as image processing and neural networks is a good approach. These techniques require the essential features to be extracted and processed. The diversity in biological characteristics of fruits generates large feature sets for processing. Various approaches are already investigated by making use of different features of fruits.

Dervim Unay and B Gossalin [1] obtained structural information like energy, entropy, inertia and local homogeneity from the co-occurrence matrix of original image of apple. Fourier transform was used for calculating the textural features of the image. These feature sets were applied to train a Back propagation neural network for classification of apples. However, in this work, size information was not considered.

Image segmentation is the process of extracting interested objects present in different locations in an image. Hence, it is a kind of pattern classification. It can be treated as maximum likelihood estimation for colour images. To improve flexibility, QI Yingjian and Luo Siwei [2] proposed a method of training the colour pattern using Expected maximum algorithm. Mathematical complexity is high in this method as preprocessing is required for segmentation.

A new feature extraction method was suggested by Masateru Nagata and Qixin [3].The shape features were expressed as thickness, length and curvature. Back propagation algorithm was used for grading apples. As only shape features were considered, it is necessary to increase number of feature sets to improve accuracy.

In these methods, the size of the fruit is not considered as a grading parameter and the complexity involved is high. In normal visual inspection, the size and colour of the fruit are the indispensable parameters for grading fruits.Hence, these two parameters are considered in this work, with apple as a specific case.

2. Grading with colour

Use of colour in image processing is motivated by two factors. First, in automated image analysis colour is a powerful descriptor that simplifies object identification and extraction from a scene. The second factor is that in colour image processing we can accomplish the effect of human eye and brain which can distinguish thousands of colour shades and intensities effectively. The process followed by human eye and brain in perceiving colour is a physio- psychological process.

In grading based on colour, the attempt is to imitate the colour perception of the human eye and modelling by brain. For implementing such a system, the colour information needs to be extracted from the object and modelled for further processing. Among many models available to express the physical nature of colour, we have chosen RGB model and subsequently HSI model is derived from it. In RGB model, the red, green and blue components of individual pixels are extracted for processing purpose. In HSI model the hue (H), saturation (S) and intensity (I) values are calculated using equations 1 to 3.

I = ( 1/3) *(R+G+B) (1)

S = 1 - [3/(R+G+B)]*MIN(R,G,B) (2)

H = Cos-1 {0.5*(R-B) / [(R-G) 2 + (R-B)*(G-B) 1/2} (3)

2.1 Pre-processing

A CCD camera is used to capture the images of apples in this work. For each apple, images are taken at six different positions to cover the whole surface of the apple. To extract the region of interest, which is the apple, thebackground should be eliminated. This is done by binary thresholding technique. In binary thresholding technique, the less predominant component is subtracted from the predominant component. For example, in the case of a red apple, red is the predominant component. A threshold value is fixed using this component. This threshold is used to eliminate the background from the image.

2.2 Feature Extraction

From the background eliminated image, RGB values of each pixel is obtained. For colour analysis of an image, the colour content in each pixel is required. This information is available in the hue value of the HSI model of the image. The HSI model can be derived from the RGB model using the following formulae.

From the HSI model, the hue value of each pixel is separated. The number of pixels at each hue value is then calculated. Using this information. the probability of occurrence of each hue level in the image is obtained. Similarly, the probability values are found for images taken at six different positions.

2.3 Training

A Backpropagation neural network with three layers is designed. The first layer consists of 6 neurons and the output layer has 4 neurons to indicate the four classes of grading. The optimum number of neurons in the hidden layer for this application is found to be 30.

The input to the neural network is an entropy functional difference of the probability of hue values. The range of these values is found to be 1 for the desired class and less than 0.5 for other classes. A thresholding is applied to the values less than 0.5 to make it 0. This generates a binary sequence which has a 1 value at the desired class position and 0’s in all other positions. Hence it is concluded that a binary sequence of 0’s and 1’s can be used as input for training the neural network.

In general for n classes and p positions for each class this process is equivalent to training the neural network with a n x p matrix.Each row has a 1 at the desired class position and a zero at other positions. For example, for grading apples into 4 classes with six positions of image for each class, the matrix will be as shown in Figure 1.

1 0 0 0 0 0

0 1 0 0 0 0

0 0 1 0 0 0

0 0 0 1 0 0

0 0 0 0 1 0

0 0 0 0 0 1

Figure 1 Training matrix

2.4 Testing

When a test apple image is obtained the probability hue values are found out. From this value the entropy functional value of the test apple with itself and with the stored reference values are calculated. The difference between these two entropies are found. This would generate a sequence of numbers which is fed to the trained neural network that will classify the apple into the proper class.

3. Grading based on size

The quality of a fruit can also be decided depending upon flavor (sweetness, acidity), appearance (color, size, shape, blemishes, glossiness), and texture (firmness, mouth feel). For grading based on appearance size is also a vital parameter. The size of the fruit is an important factor for fixing its price. An approach for grading apples based on its size is suggested.

3.1 Pre-processing

The image capturing and pre-processing techniques used in colour analysis are followed here also to get a binary image suitable for further processing. The elongation ratio, Eccentricity, Perimeter and area of apple are considered for analysis. However, during the course of work it is found that area and perimeter are the essential parameters, which give the correct and precise values for the effective classification of fruits based on size.

3.2 Feature Extraction

To extract the size information, the perimeter and area of the fruit are considered. The contour length of the fruit and the number of total pixels covered by the fruit are calculated using image processing techniques. The average value of the contour length and the area for each type of apple are calculated. This information is used for training.

3.3 Training

A Backpropagation neural network with the same architecture as designed for colour grading is used here.

The same methodology applied for colour processing is followed, but with perimeter and area as reference values.

3.4 Testing

When a test apple image is obtained the probability area and perimeter values are found out. From this value the entropy functional value of the test apple with itself and with the stored reference values are calculated. The difference between these two entropies are found. This would generate a sequence of numbers which is fed to the trained neural network that will classify the apple into the proper class.

4. Results

A dataset of 40 apples from four different varieties is collected for training the neural network. Various training trials are conducted with different number of neurons in the hidden layer and different training functions, keeping the mean square error as the training parameter. From the outcome of the trials, optimum number of hidden layer neurons and the best training function is fixed with respect to minimum number of epochs and minimum time.

Another dataset of 20 samples is used for testing. The grading results are compared with manual grading results. Recognition rate for both approaches is given in Table 1.

Table 1 Comparison of Grading of Apples

Method	Variety	Manual Grading (In Numbers)	Automatic Grading (In Numbers)
Colour	Red	5	4
	Pink	5	6
	Yellowish Red	5	5
	Green	5	5
Size	Very Large	5	5
	Large	5	4
	Medium	5	6
	Small	5	5

5. Conclusions

In this work, we applied Back propagation neural network for the classification of apples based on colour and size. For colour feature extraction, we worked on the HSI model of the fruit image. Instead of the direct hue value, average probability hue value is selected for processing. This has increased the discriminating ability between apples of similar colour.

For getting size information, binary model of the fruit image is used. Both area and perimeter are considered for size analysis, which increased the classification efficiency of the system. This method could be a better choice if we extend this work for different agro products.

In our approach, the colour and sizeparameters are considered separately. The two parameters can be combined and used as input to neural network for better recognition results.

References

[1] Dervim Unay, Bernard Gosselin, Apple Defect Detection and Quality Classification with MLP_Neural Networks, in Proeedings of ProRISC, Nov 2002, pp. 501-506,.

[2] QI Yingjian, LUO Siwei and LI Jianyu, Using Neural network in Colour Classification, in Proceedings of IEEE TENCON, 2002, pp. 708-711.

[3] Devrim Unay, Bernard Gosselin, A Quality Grading Approach for Jonagold Apples, in Proceedings of SPS 2004 IEEE Benelus Signal Processing Symposium , April 15-16, 2004, pp.93-96.

[4] F.Pla and et al., An integral Automation of fruit and vegetable sorting by machine vision, in Proc. of the 8th IEEE Int. Conf. on Emerg.Tech. Fact. Autom. Antibes, 2001.

[5] M.Nakata , Q.Cao, Study on Grade Judgement of Fruit Vegetables using Machine Vision, J .Society Hightech.Agri, Vol.I, May 1997, pp.49-59.

[6] Laurene Fauset , Fundamentals of Neural Networks (Prentice Hall of India, 1994).

[7] Rafael.C.Gonzalez, Richard E.Woods, Digital Image Processing (Second Edition, Pearson Education, 2003).

(Note: This Paper was presented in ICSIP Please Download the paper for proper formatting, images and symbols)

Attached Files:

• dip0926.doc (61 KB)

Attached Files:

• dip13703.pdf (252 KB)