This work was presented in the I Symposium National of Horticultural Engineering

Automatic segmentation of images of crops: comparative study of models of colour

G. García-Mateos and S. Jaén Clods (Dep. Of IT and Systems. Faculty of IT. University of Murcia)

D. Escarabajal-Henarejos And J.M. Molina-Martínez (Group of Inv. In Engineering Agromótica and of the Mar. Polytechnical University of Cartagena)

J.L. Hernández-Hernández (Academic Unit of Engineering. Autonomous university of Guerrero, Mexico)

15/04/2014

The processing of images and the artificial vision result increasingly useful and necessary in diverse applications agromóticas of management of crops and processes agroalimentarios. The images offer information very valuable on the state of growth of the plants, the illnesses carenciales, the maduration of the frutales, the apparition of plagues and bad grasses, etc. The technicians based in analysis of colour are fundamental in this command, when allowing classify and characterise of precise form, robust and efficient the distinct regions of the images.

In this article tackles the segmentation of images of crops by means of colour. The key appearance is the modelling of the functions of distribution of probability of the colour of the plants and of the floor; said distributions are modeladas of form no parametric with histograms. The effectiveness of the technician depends on the capacity of the space of colour used. Thus, they have compared diverse spaces of colour: RGB, rgb, XYZ, L*to*b*, L*or*v*, HSV, HLS, YCrCb, YUV, TSL and I1I2I3. In each space, to his time, arise different alternative: which channels use and the size of the histograms. Splitting of a very wide group and miscellaneous of images of lettuce (Lactuca sativa), has carried out an exhaustive and systematic study of the optimum form of modelar the colour in the problem of segmentation. The results show the superiority in the models that separate luminance and crominancia, and especially the space of colour L*to*b*. Incidentally, diverse stages of pre- and post-processing can attain improve the results until a tarpaulin of 99,5% taking only 0,3 seconds by image in a conventional computer.

Introduction

The automatic analysis of digital images of crops is a field of investigation very active and appeal in which confluyen the vision by computer and the agromótica (Cubero et al., 2011). The colour is an essential characteristic in a lot of problems of this command, although no all the artificial vision bases in the use of colour; in fact, it exists a wide variety of alternative technicians that have been used in applications agromóticas, like the analysis of edges, forms, textures, etc. However, the handle of colour presents big advantages in the problems that occupy us by his low dimensionalidad, high expressesiveness and ease of processing. The colour can serve to discriminate of precise and efficient form the pixels, or zones of an image, that correspond to the floor or to vegetation. Of this form, the result of the classification in floor or plant can use for the measurement of the coefficient of vegetal coverage (Fernández-Pacheco et al., 2014). This parameter has a key paper in the monitoring of crops and is widely applied to determine the needs hídricas of the crops using the methodology FAO-56 (Allen et al., 1998).

The problem of the automatic segmentation am used to/silver has been widely studied by diverse researchers, as it can check in two recent reviews of the state of the art (Lin et al., 2013; McCarthy et at., 2010). In the majority of these works the classification does using diverse technicians like umbralización, discriminantes linear, models of gausianos, fuzzy logic, neural networks, and others; but the space of colour is fixed in advance, and throw in lacking comparative studies on the effectiveness of diverse models of colour in plants. However it is usual to find this type of comparative works other commands. For example, Shih and Lui (2012) evaluated comparatively 12 spaces of colour in applications of processings of faces, and arrived to the conclusion that the optimum spaces are YUV and YIQ; Luszczkiewicz-Piatek (2014) studies the election of the space of colour for the recovery of images in big databases; the colour represents with models of mix of gaussian and analyse 11 spaces of colour. By his part, Terrillon and Akamatsu (2000) entered the space TSL, and compared 9 spaces in the detection of faces.

In consequence, the aim of this work is to realizar a complete and exhaustive study, with the purpose to select the space of optimum colour and the most effective form to represent the distributions of colour for the problems of classification in the command agromótico, and in concrete for the automatic segmentation of plants in images.

Materials and methods

Images used for the experimentation

The images used for the experiments correspond to crops of lettuce (Lactuca sativa), in the region of the field of Cartagena (37º46'N, 0º58'Or). They realizar two series of photographies. The first series took place between October of 2010 and January of 2011, and the second between October and December of 2012. In each one of these series did a photographic follow-up of 4 different plots, at intervals from among 2 and 4 days. The images were taken with a compact digital camera Nikon model Coolpix S3300 to high resolution, and present a zenithal sight of the terrain, as it shows the figure 1.

It appears 1: Images used for the experimentation. To) An overview of one of the plots with the area of interest signalled. b) Example of a photography of the plot. c) The same photography after the classification with ENVI in floor and plant.

The images were recortadas and normalised with regard to a rectangular pattern (situated physically in the floor); like result, the images of the first series have a resolution of 1500x1500 pixels and 3600x2000 pixels the second. Afterwards they were divided manually by experts using the software ENVI (Environingingment for Visualizing Images) version 4.0, obtaining binary images where each pixel is classified like floor or plant (Fig. 1c). In total there are 169 images of high quality, that present a big variability regarding the lighting, percentage of vegetal coverage, apparition of shadows, humidity of the floor, types of floor, etc. In the experiments, the first series is used for the learning of the models of colour, and the second for his experimental validation.

Classification and representation of the colour

The concrete colour of a pixel of the floor, or of the plants, can understand like a stochastic event inside the space n-dimensional defined by the space of colour used. In the case of the floor, the distribution of probability of the colour is determined by the type and composition of the floor, the lighting in the moment to take the photo and the own parameters of the camera, like the balance of whites; of similar form occurs with the colour of the vegetal coverage, where the chlorophyl gives place to the predominancia of the green tones.

Suppose known the functions of distribution probability of the colour of the floor, p_{am used to}(colour), and of the colour of the plant, p_plants(colour), where colour is a tupla of a space of colour any one. Considering equiprobables the probabilities a priori of floor and of plant, the probability that a pixel, colour, concrete was of plant or of floor, can calculate of simple form by means of the rule of Bayes, given by the formula:

In consequence, the concrete pixel is assigned to the class of greater probability between P(plants|colour) and P(am used to|colour). Of trivial way, the problem reduces to look for the maximum between p_plants(colour) and p_{am used to}(colour) for each pixel of the images.

The classification pixel-to-pixel obtained of this form can improve with the application of operations of local processing, that take into account the vecindad of the pixels. In concrete, have checked two options: reduction of the image by 2 using interpolation of supermuestreo, and operations of mathematical morphology. Whereas the first achieves to reduce the level of noise of the images (when doing a promediado of pixels), the second allows to clean the result of values espurios. Besides, both operations can apply of form combined.

Spaces of colour analysed

The forms that adopt the functions of distribution of probability, p_{am used to} and p_plants, depend, logically, of the space of colour adopted. Each space generates distinct degrees of separation between the distributions, affecting like this the effectiveness of the classification. The models of colour selected for the study, chosen between the most usual in applications of artificial vision (Luszczkiewicz, 2014), are the following:

RGB, rgb. The space RGB is the most extended in devices of capture and generation of colour. It bases in a model of additive mix, where each colour forms by combination of the three primary colours: red (R), green (G), and blue (B). From this model defines the variant normalised rgb, that consists in dividing the values of RGB by (R+G+B).
XYZ. Is a model that simulates the answer of the three types of fotorreceptores of the retina (those with spectrums of absorption in lengths of high waves, half and drops). The concrete form of these spectrums of absorption was defined by the CIE (Commission internationale of l'éclairage) in 1931.
HSV, HLS. In these models the colour decomposes in terms easily interpretable like the intensity (V, L), the saturation or purity (S), and the nuance of colour (H). They are models very usual in analysis and graphic generation.
YCrCb, YUV. Treats of models oriented to compressesion and transmission of images. They base in the separation of a channel And of luminance (or luminous intensity), and two channels of crominancia (or tone of the colour, independent of his luminosity). The channels Cr and Cb correspond, in essence, to the channels R and B normalised in intensity, respectively.
L*to*b*, L*or*v*. These models were defined by the CIE for industrial applications of the colour. Of similar form to the previous, separate a channel of luminosity (in this case L*) and two channels of crominancia (to*-b* and or*-v*). These last define by means of transformations no linear of the model RGB, with the end to achieve representations perceptualmente uniforms of the colour.
TSL, I1I2I3. Some models have been designed specifically for some applications. For example, the model TSL was created to tackle problems of detection of human leather (Terrillon and Akamatsu, 2000). On the other hand, I1I2I3 proposed for the segmentation of colour, like a decorrelación of the components RGB using the transformed Karhunen-Loeve.

Representation of the distributions of probability of colour

The modelling of functions of density of probability from observations is one of the basic problems in recognition of patterns and in artificial intelligence. When it does not know the distribution of the data (or when this adopts forms very complex), apply the designated 'methods no parametric'. Inside them, the based in histograms are the most common when it works with spaces of drop dimensionalidad. The idea consists in calculating the histogram of the data observed, for each class of interest, whose great is afterwards normalised to value 1. Each cell of the histogram considers an approximation of the function of density in the corresponding rank.

To his time, exist distinct alternatives in the representation of the histograms. In the first place, the number of channels to use, since no all the channels are equal of useful for the problem. As we will see, descartar some channels can produce better resulted that use them all. Second, have the size of the histogram. The histograms suppose a discretisation of the rank of the variables studied. The discretisation does in powers of 2, what allows to obtain a uniform distribution of the values of entrance, represented with bytes. In definite, for each one of the spaces of colour, have the option to use histograms of a channel, two channels or three channels, with 16, 32, 64, 128 or 256 cells by dimension.

Results and discussion

For each one of the alternatives of representation the colour (composed by the triple: space of colour, channels to use, size of histogram), has realizar in the first place the learning of the models of colour of floor and of plant, using the images of the first series (a total of 108 images). The figure 2 sample the histograms 3D of the 11 spaces of colour. On this result has obtained like measure the solapamiento between both distributions of probability, calculated with the integral of the minimum area of both histograms.

Like second experiment has realizar an automatic classification of the images of the second sequence (a total of 61 images), comparing the result with the manual classification of the experts. The value obtained in this case is the percentage of final tarpaulin to level of pixel for each option.

It appears 2: Histograms 3D of the classes plants (in green) and floor (in red) for the spaces of colour analysed. In all the cases, the size of the histogram is of 64 cells by dimension.

Solapamiento Between distributions and tarpaulin of classification

In the table 1 show , of way resumida, the results of the measures of solapamiento and the percentage of tarpaulin. The first parameter measures the separabilidad between classes that offers each configuration (that is to say, the capacity to differentiate the distributions of the class plants and floor), whereas the second sample more clearly the effectiveness of each one in the segmentation.

Can stand out some notable facts. As it was to expect, the solapamiento diminishes with the number of channels and with the size of the histograms. However, whereas in the first factor the reductions are very significant, in the second do not reach 30% of reduction in the best cases. Besides, big sizes of histograms increase the risk of 'sobreajuste' to the data of learning.

The channel that by himself only achieve better results is the to* of the L*to*b*, with only 0,95% of solapamiento and 99,2% of tarpaulin. The best result obtains for 32 cells, although the results are also very good for 64 and 128. The following options, to more distance, are channels I3 in I1I2I3, Cr in YCrCb, V in YUV, or* in L*or*v*, H in HSV/HLS, and T in TSL, with precisions always upper to 96%. By his part, any of the channels of RGB is able by himself only to achieve a solapamiento low, as neither the channels of luminosity of the distinct spaces.

Table 1: Results of the proofs of solapamiento and of the classification am used to/plant. For each space of colour, for each combination of 1, 2 or 3 channels shows : the percentage of solapamiento between classes (left) / the percentage of tarpaulin of the classification (right).

Regarding the combinations of two channels, the main result is that the minors solapamientos produce when deleting the luminosity, what occurs with to*-b* of L*to*b*, or*-v* of L*or*v*, and Cr-Cb of YCrCb. This agrees with the idea that the crominancia contributes information of interest to the problem, whereas the luminance is much less notable. In general, these results coincide with what other authors have obtained in other commands (Luszczkiewicz, 2014; Shih, Liu, 2005), where the spaces L*to*b* and YCrCb are between the most effective.

Finally, have selected some of the configurations that produce better results and on them have applied the additional improvements before described: reduction by 2 of the images; application of mathematical morphology; and the combination of both. In the figure 3 can see some examples of automatic segmentation including these improvements. The complete results can see in the table 2, that shows also the times of execution in a half computer.

It appears 3: Results of the automatic segmentation am used to/plant, using the channels L*-to* of the space L*to*b*, with 64 cells by dimension, reduction by 2 and mathematical morphology. Of izqu. To dcha.: The errors obtained with regard to the manual segmentation are: 0,03%, 0,11% and 0,64%.

Table 2: Taxles of error and time of execution of the automatic classification for the seven better spaces of colour and channels; it shows the half error in % (between bracket, the half time by image in second). Original: images of original resolution; Reduction: size reduced to the half; Morphology: application of the operators of morphology open and close; Both: results to combine both improvements.

Conclusions

The election of the space of optimum colour, as well as of the form to represent the distributions probability of colour, is a fundamental requirement in any application that do use of the colour. Through a series of experiments, in which they have contrasted 385 different combinations of spaces, channels and sizes of histogram, has explored of exhaustive and systematic form the best form to resolve the problem of the classification am used to/plant in the command agromótico.

Of this study can conclude several facts. First, that the classification by colour results viable in the practice when producing some results of 99,5% of tarpaulin in the classification, requiring only 0,3 seconds by image in a standard computer. Segundo, that the best models of colour are those that separate a channel of luminance and two of crominancia, where the seconds are those that contribute the notable information for the problem. In concrete, the space L*to*b* has showed to be above the rest in the majority of the cases; the space RGB, in spite of being one of the most widespread, would not have to be used for the automatic classification. Finally, the application of operations of pre- (reduction of the images with supermuestreo) and post-processing (operators of mathematical morphology) is very useful to improve the results and reduce the times of execution. At present, the team of investigation is working in the implementation of applications to level of user to realizar the automatic segmentation in mobile devices, using the results of this study.

Bibliographic references

Allen, R.G., Pereira, L.S., you Scrape, D., Smith, M. 1998. Crop evapotranspiration-Guidelines for it crop water requirements-FAO Irrigation and drainage paper 56. FAO, Rome 300, 6541.
Cubero, S., Aleixos, N., Moltó, And., Gómez-Sanchis, J., Blasco. J. 2011. Advances in machine vision applications for automatic inspection and quality evaluation of fruit and vegetables. Food and Bioprocess Technology, Vol. 4 (4):487-504.
Fernández-Pacheco, D.G., Escarabajal-Henarejos, D., Ruiz-Channels, To., Conesa, J., Molina-Martínez, J.M. 2014. To new digital image processing based methodology for determining Kc in lettuce crops in the Southeast of Spain. Biosystems Engineering 117:23-34.
Kumar, P., Sengupta, K., it Reads, To. 2002. To comparative study of different colour spaces for foreground and shadow detection for traffic monitoring system. Intelligent Transportation Systems:100-105.
Lin, K., Chen, J., If, H., Junhui, W. 2013. To review on computer vision technologies applied in greenhouse plant stress detection. Advances in Image and Graphics Technologies, 363, 192–200.
Luszczkiewicz-Piatek, M. 2014. Which Colour space should be chosen for robust colour image retrieval based on mixture modeling. Advances in Intelligent Systems and IT. Vol. 233:55-64.
McCarthy, C.L., Cheryl, N.H., Hancock, S.R. 2010 Applied machine vision of plants - to review with implications for field deployment in automated farming operations. Intelligent Service Robotics, 3 (4), 209–217.
Shih, P., Liu, C. 2005. Comparative assessment of content-based face image retrieval in different Colour spaces. Int. J. Patt. Recogn. Artif. Intell. 19(7), 873-893.
Terrillon, J.C., Akamatsu, S. 2000. Comparative performance of different chrominance spaces for Colour segmentation and detection of human faces in complex scene images. International Conf. on Face and Gesture Recognition, 54-61.