Segmentation algorithm for Hangzhou white chrysanthemums based on least squares support vector machine

: In order to realize the visual positioning for Hangzhou white chrysanthemums harvesting robot in natural environment, a color image segmentation method for Hangzhou white chrysanthemum based on least squares support vector machine (LS-SVM) was proposed. Firstly, bilateral filter was used to filter the RGB channels image respectively to eliminate noise. Then the pixel-level color feature and texture feature of the image, which was used as input of LS-SVM model (classifier) and SVM model (classifier), were extracted via RGB value of image and gray level co-occurrence matrix. Finally, the color image was segmented with the trained LS-SVM model (classifier) and SVM model (classifier) separately. The experimental results showed that the trained LS-SVM model and SVM model could effectively segment the images of the Hangzhou white chrysanthemums from complicated background taken under three illumination conditions such as front-lighting, back-lighting and overshadow, with the accuracy of above 90%. When segmenting an image, the SVM algorithm required 1.3 s, while the LS-SVM algorithm proposed in this paper just needed 0.7 s, which was better than the SVM algorithm obviously. The picking experiment was carried out and the results showed that the implementation of the proposed segmentation algorithm on the picking robot could achieve 81% picking success rate.


Introduction
As a traditional natural drink, tea has been favored by many people, and Hangzhou white chrysanthemum as a kind of tea has become a treasure one because of its unique characteristic.However, due to the short flowering period, large amount of harvesting and high quality requirements, the harvesting operation of Hangzhou white chrysanthemum is still mainly manually.Although there are some researches on agricultural picking robots [1] , there is no research on the identification and positioning of the Hangzhou white chrysanthemum picking robot.Therefore, the research work in this paper is necessary and meaningful.
Machine vision is a key component of picking robot and plays a decisive role in the success of target picking.And image segmentation is the key technology of machine vision, which is the basis of target recognition and positioning [3] .Some universities and research institutions have researched on agricultural machine vision, and obtained a series of research results [6] .Wei et al. [9] used the improved OTSU threshold algorithm to automatically extract fruit objects from complex agricultural background.Luo et al. [10] proposed a new method of picking point location based on improved clustering image segmentation and dotted line minimum distance constraint for grapes.Ji et al. [11] used image processing combined with machine learning to segment apples in complex environment.
With the development of machine learning, combining machine learning with image recognition has grown into a hotspot for research and application in recent years [12] .So far, many scholars have successfully applied machine learning to the field of image segmentation and achieved good experimental results [13] .Guerrero et al. [16] proposed a new method based on Support Vector Machines for identifying plants with green spectral components masked and unmasked, the method was also valid for post-treatment evaluation, where loss of greenness in weeds was identified with the effectiveness of the treatment and in crops with damage or masking.Ahmad et al. [17] used the reinforcement learning of threshold segmentation method to segment sweet pepper in the greenhouse.Ma et al. [18] constructed a greenhouse cucumber disease recognition system based on convolutional neural network, which could quickly and accurately segment the image of greenhouse cucumber lesions.However, there are few researches on image segmentation of Hangzhou white chrysanthemum.Fan et al. [19] used S-component fuzzy clustering method to segment the images of Hangzhou white chrysanthemum, but the influence of illumination intensity on image segmentation have not been eliminated in natural environment.Actually, agricultural picking robot needs to ensure effective segmentation of Hangzhou white chrysanthemum under any illumination conditions.
Since the SVM model (classifier) has strong anti-interference ability, but SVM model (classifier) process data at relatively slow speed.Therefore, this paper proposed a color image segmentation method based on least squares support vector machine (LS-SVM) for the image of Hangzhou white chrysanthemum collected in natural environment, and the proposed segmentation algorithm was used to perform the picking experiment on Cartesian coordinate system robot.

Image acquisition and preprocessing
Bilateral filter is a filter that can preserve edge and remove noise.The reason why the filter can achieve reducing image noise and preserving object edges is that it consists of two functions.One affects the filter coefficients determined by the geometric spatial distance, and the other affects the filter coefficients determined by the pixel difference [20] .
The expression of the bilateral filtering method can be stated as follows: And the normalization parameter can be stated as follows: where, ξ and x are the positions of the pixel points of the image; c(ξ, x) represents the distance similarity between the adjacent point ξ and the center point x; s(f(ξ), f(x)) represents the brightness similarity between the adjacent point ξ and the center point x; f(x) indicates the brightness value of the input image at point x; h(x) indicates the brightness value of the output image at point x; k(x) is the normalized parameter whose value is independent of the image content and is constant at the same geometric position.
Extend bilateral filtering to Gaussian kernel, as is stated as follows: ( , where, d(ξ, x)=||ξ-x|| represents the Euclidean distance between point ξ and x.The contrast diagram of the Hangzhou white chrysanthemum images before and after the bilateral filter processing is shown in Figure 1.The noise has a great influence on the quality of the picture.When the pixel was selected in the training sample, the noise point may be selected as the sample data of the training classifier, which would cause the decrease of accuracy of the support vector classifier, then resulting in fault segmentation.Therefore, it is necessary to filter the picture before extracting the sample data from the training images.And the bilateral filter can not only reduce the influence of noise on the segmentation, but also preserve the edge information without being destroyed, which will be more contribute to image segmentation.

Extracting features of color image
In this paper, each pixel of an image was identified as an object or a region corresponding to a part of an object.Image segmentation is considered as a classification task, whose goal is to assign a label to each pixel or region.So extracting effective features from the images is very important for image segmentation.Therefore, this paper would extract the feature of Hangzhou white chrysanthemum from three different illumination intensities.
In the natural environment, it is very difficult to completely separate the Hangzhou white chrysanthemum from the acquired images.The success rate of segmentation is not only affected by the leaves, weeds and soil in the background, but also affected by the illumination conditions.Under natural illumination conditions, the obtained images of Hangzhou white chrysanthemum can be divided into three situations: front-lighting, back-lighting and overshadow, as shown in Figure 1.And the images were analyzed according to these three situations.
(1) Under the condition of front-lighting, there will be a strong reflected light or shadow area on the surface of the Hangzhou white chrysanthemum when the sun is very strong, which will generate a concave edge or hole in the images.
(2) Under the condition of back-lighting, there will be a large shadow area in the images of Hangzhou white chrysanthemum.
(3) Under the condition of overshadow, there will be some shaded areas in the images of Hangzhou white chrysanthemum.

Color feature
According to the above factors affecting the segmentation of Hangzhou white chrysanthemum in the natural environment, the color feature extraction of the image of Hangzhou white chrysanthemum is as follows: (1) Randomly selecting 50 images of each of the three conditions of front-lighting, back-lighting and overshadow from the image library, and then performing bilateral filtering on the selected pictures.
(2) Taking out the parts of flowers, leaves, weeds and soil in each picture under each condition, as shown in Figure 2.
(3) Sample data was evaluated by averaging each of the extracted small images, wherein the size of each image was 150×150.The specific operation steps are: extracting images of flowers, weeds, leaves and soil from 50 images in each condition; then average processing the R, G and B component image of the target objects in each image; after that, extracting 50 target sample points each.The mean formula is as shown in Equation ( 5). 1 where, m and n is the position of the image pixels; K represents the space of R, G and B component; P K mn is the pixel value of position m, n in R, G, B component image respectively; ρ K ij is the pixel value of position i, j in R, G, B component image; d is the size of the average processing window (d =3 in this article).
According to the above method, 7500 training sample data of flowers, weeds, leaves and soil could be obtained by extracting color feature of Hangzhou white chrysanthemum, and a total of 30 000 training sample data sets were obtained.In order to display the difference in color feature of each part of the image more intuitively, 80 samples of flowers, weeds, leaves and soil were selected according to the conditions of front-lighting, back-lighting and overshadow, and 240 sample data of flowers, weeds, leaves and soil could be obtained.According to these sample data, a line chart of the three components R, G and B could be obtained, as shown in Figure 3.In the above figures, the red, green, and blue dots represented the R, G and B pixel values respectively.It could be seen that the pixel distributions of the fetal and ripe Hangzhou white chrysanthemums were very similar, the value of the pixels was arranged as R>G>B, and the pixel distribution of the leaves and weeds was arranged according to G>R>B.In addition, the pixel distribution of soil and Hangzhou white chrysanthemum was also very similar.Although they have large difference in pixel values, they do not fully reflect their characteristic differences.

Texture feature
Texture is a commonly used feature in image segmentation.In order to achieve better segmentation effect, it is usually used together with color information, instead of segmenting the image with color information only.This paper used the method of gray level co-occurrence matrix to extract texture features from images [22] .
The gray level co-occurrence matrix is a method of performing statistics on the basis of the second-order probability density, which uses the spatial information of the image pixels to reflect the texture information of the image.Usually, the texture feature is not directly analyzed by the gray level co-occurrence matrix, but is reflected by the feature quantity evaluated by the gray level co-occurrence matrix.In this paper, the correlation feature quantity was used to analyze the soil, fetal and ripe Hangzhou white chrysanthemum.The mathematical model is stated as follows: ( ) ( ) among them, where, k is the size of the window; p(i, j) is the probability of occurrence of a two-pixel gray level separated by a certain distance.
The specific steps are as follows (1) Selecting 50 pictures of soil, fetal and ripe Hangzhou white chrysanthemum under the three illumination conditions in Figure 2.
(2) Using the mathematical model to analyze the texture feature of soil and Hangzhou white chrysanthemum.In this paper, the gray level was set as L=32, the distance was set as d=1, and the size of the window was set as k=9×9.
According to the three illumination conditions, 50 correlation features were randomly selected from the partial images of the soil, fetal and ripe Hangzhou white chrysanthemum, as shown in Figure 4.

Figure 4 Graph of correlation
It could be seen from Figure 4 that the soil and Hangzhou white chrysanthemum have significant difference in the value of the correlation feature quantity.According to correlation feature quantity, the Hangzhou white chrysanthemum could be distinguished from the soil.Texture feature could make up for the deficiency of using only the color feature as the image segmentation, and improve the correct rate of image segmentation.

Feature selection and analysis
According to the color feature, the Hangzhou white chrysanthemum could be distinguished from the leaves and weeds, but couldn't be distinguished from the soil.Therefore the difference in feature between Hangzhou white chrysanthemum and soil require further research.In order to obtain a higher segmentation rate, the texture feature of Hangzhou white chrysanthemum and soil were analyzed.The results showed that the correlation feature between the Hangzhou white chrysanthemum and the soil based on the gray level co-occurrence matrix has obvious difference.Therefore, the RGB three-channel pixel value and the correlation texture feature quantity of the color image were selected as the feature vector of the Hangzhou white chrysanthemum image.

Support vector machine algorithm 4.1 Standard support vector machine algorithm
Support Vector Machine (SVM) shows many unique advantages in small sample, nonlinear and high-dimensional pattern recognition.Standard SVM can use n-1 dimensional hyperplane to divide n-dimensional training data into two parts.Each part of the data belongs to the same category [24]-.
According to the color feature and texture feature of Hangzhou white chrysanthemum, the training sample vectors are extracted as x i ={R, G, B, CORR}, where, x i ∈R n represents training sample, n represents the dimension of the training sample data (n=4 this article).At the same time, mark a label value for each sample data y i , y i ∈{-1, +1}, where y i =+1 stands for Hangzhou white chrysanthemum label, y i = -1 stands for background label.Therefore, the training sample data provided to the classifier is {x i , y i } m i=1 .where, m is the number of training sample data (m=30000 in this article).The model of the feature space support vector machine is stated as follows: y(x) = w T x+b (7) where, x is linear mapping; b is bias; w is weight vector with the same dimension as the feature space.
Suppose training samples can be correctly classified by the hyperplane, which means for (x i , y i )∈D, if y i =+1, then w T φ(x)+b>0; if y i = -1, then w T φ(x)+b<0. Let As shown in Figure 5, several training sample vectors closest to the hyperplane make the equation satisfied, which are called "support vectors".And the sum of the distances of the two heterogeneous support vectors to the hyperplane is 2 || || = γ w (9)   which is called "margin".
where, C is penalty factor ( the influence level of the sum of slack variable); ε i is the slack variable.
Figure 5 Basic theory of SVM Due to the possible intersection of a small portion of the Hangzhou white chrysanthemum feature vectors and the background feature vectors, probably lead to the correct hyperplane cannot be found.Therefore, the slack variable was introduced to solve this problem.The quadratic programming problem can be solved by the Lagrangian multiplier method when the KKT (Karush-Kuhn-Tucker) condition is satisfied.And the Lagrangian function can be written as: where, α and μ are the parameter vectors whose dimension is the same as samples.After α is evaluated, w and b can be obtained as well.

Improved support vector machine algorithm
Since the traditional support vector machine algorithm needs to solve the quadratic programming problem after being transformed into a dual problem, the complexity of the algorithm is increased and the computational cost is amplified.Therefore this paper used the improved SVM algorithm to segment image of Hangzhou white chrysanthemum.

Least squares SVM algorithm (LS-SVM)
The basic idea of the least squares support vector machine is to estimate the estimation function in pieces and then combine them together to reduce the complexity of the algorithm.The least squares support vector machine is a kind form of support vector machine under the quadratic loss function, which will change the quadratic optimization of the original support vector machine algorithm into the linear equation by constructing the loss function.The optimal result is: where, e i is the deviation of discrete data; γ is the weights of discrete data.
After transforming the dual problem, just needs to solve the linear equation, there is unnecessary to solve the quadratic programming.Therefore, this method simplifies the calculation, reduces the computational cost, and reduces the possibility of falling into local minimum.With the Lagrange multiplier method, the least squares support vector optimization problem can ultimately be transformed into solving a linear equation, as shown in Equation (13).
where, I is the unit matrix; Ω k,l = y k y l x k x l is the core matrix.a and b can be evaluated by solving this linear equation, and the classification function can be finally obtained as follows: 1 ( ) sign( )

LS-SVM sparse processing
It can be found from the value of |a i | in the LS-SVM that the larger |a i | is, the greater the influence on the classification function y(x) will be.Therefore, the feature vectors corresponding to the minimum value of |a i | can be gradually deleted, and the classification function will be generated by training the reduced feature vectors.

Experiment and analysis 5.1 Algorithm segmentation experiment
The features extracted in Section 3 were trained in the established support vector machine model, the flowers were labeled as positive samples, and the others were labeled as negative samples.The basic process is shown in Figure 6.An image with size of 560×480 was randomly selected from the image library as a testing image for experiment.Firstly, the classifier was used to predict each pixel of the selected image.If the predicted value is y i = +1, the pixel belongs to region of Hangzhou white chrysanthemum.If the predicted value is y i = -1, the pixel belongs to region of background.
The image segmentation results are shown in Figure 7 under three illumination conditions.
According to the analysis of the above experimental results, the classifier could segment the Hangzhou white chrysanthemum in any environment.Due to the contingency of the selected image, 60 images were randomly extracted from the image library according to the three kinds of illumination conditions as testing images.And the testing images were used to verify the accuracy and applicability of the standard SVM algorithm and the algorithm proposed in this article for segmenting the Hangzhou white chrysanthemum image.And the testing platform used Intel(R) Core(TM) i7 and Windows10 notebook computer with clock speed of 2.80GHz and memory of 8GB.At the same time, OpenCV3.2 was used as an experimental tool for image segmentation experiments.The indicators for quantitative evaluation of the algorithm are as follows.And some experimental results are shown in Table 1.
where, SC is the rate of successful classification; N c is the number of correctly classified pixels; N t is the total number of pixels in the image.
It could be seen from the above experimental results that the Hangzhou white chrysanthemum could be segmented from the complex background under three different illumination conditions.The average calculation time of the standard SVM algorithm for processing each image was 1.3 s, while the average calculation time of the improved SVM algorithm for processing each image was only 0.7 s.Compared with the SVM algorithm, the LS-SVM algorithm can not only spend less time to segment image, but also has a relatively high image segmentation success rate.Thus, it could be seen that the algorithm proposed in this article was greatly improved compared to the standard SVM algorithm.The pixel classification success rate is mainly affected by the penalty factor C or γ in the SVM model.If C or γ is infinite, it may not be possible to find a hyperplane to separate the two types of data.If C or γ is too small, the classification accuracy of the SVM model may be reduced after sample training.The experimental platform mainly includes a robot with Cartesian coordinate system, end-effector, binocular camera, computer and control system.The whole experimental platform is shown in Figure 8.

Experimental procedure
Firstly, the image of Hangzhou white chrysanthemum was obtained by the binocular camera, and the relative position information of the Hangzhou white chrysanthemum was acquired by the above algorithm.Then the position information was sent to the controller, which would control the end-effector of the Cartesian coordinate system robot to reach the designated position.Finally, the Hangzhou white chrysanthemum would be picked by the end-effector.The specific control procedure is shown in Figure 9.In order to prevent the Hangzhou white chrysanthemum from being damaged during the picking process, an end-effector with a special structure was designed.An airbag device was attached at the bottom of the end-effector, and the Hangzhou white chrysanthemum would be picked by the inflation of the inside of the airbag, as shown in Figure 10.Firstly, move the end-effector to the specified target position, then move the robotic arm in the Z direction while using a small suction to draw the flower into the end-effector.After that, inflate the airbag to wrap the Hangzhou white chrysanthemum.And finally pick the Hangzhou white chrysanthemum by moving the end-effector.

Experimental results and analysis
The improved SVM segmentation algorithm proposed in this article was verified by the experimental platform shown in Figure 8.The experiments were carried out under three different illumination conditions: front-lighting, back-lighting and overshadow.There were three groups of experiments performed in each illumination condition.The quantitative evaluation indexes of each group of picking experiments are stated as follows, and the experimental results are shown in Table 2.
where, PC is the rate of success picking; M c is the number of the Hangzhou white chrysanthemum that were correctly picked; M t is the total number of the Hangzhou white chrysanthemum.According to the experimental results, we knew that the average picking success rate under front-light and overshadow condition was 80%, and the average picking success rate under back-light condition was 83%, picking success rates under three illumination conditions were basically the same.It can be inferred that the algorithm proposed in this paper can basically avoid the influence of illumination intensity on image recognition.The average time to pick a flower from the start of moving the robotic arm to the picking was 12.5 s, and the average success rate was 81%.Meanwhile, the picking time is mainly affected by the control system, therefore, the picking time can be reduced by the optimizing the control system.And the picking success rate is mainly affected by two aspects: (1) some flowers cannot be completely recognized by the algorithm because of their tilted growing posture or the shading by leaves.(2) The different between the motion trajectory calculated by the proposed algorithm and the actual position.Both of these reasons may lead to the inability to pick the Hangzhou white chrysanthemum.

Conclusions
(1) The segmentation of Hangzhou white chrysanthemum in natural environment was researched.Firstly, the bilateral filter was used to filter the image of Hangzhou white chrysanthemum.Then, the color feature and texture feature extracted from the image were trained as inputs to the LS-SVM training model.Finally, a classifier was generated that can segment the Hangzhou white chrysanthemum from a complex background.
(2) The generated classifier was used to segment the images taken under the three conditions of front-lighting, back-lighting and overshadow.The experimental results showed that the average calculation time of the standard SVM algorithm for processing each image was 1.3 s, while the improved SVM algorithm for processing each image was only 0.7 s.Compared with the SVM algorithm, the LS-SVM algorithm can not only spend less time to segment image, but also has a relatively high image segmentation success rate.Thus, it can be seen that the algorithm proposed in this article is greatly improved compared to the standard SVM algorithm.
(3) According to Hangzhou white chrysanthemum picking experimental results under the conditions of front-lighting, back-lighting and overshadow, it can be seen that the average picking time was 12.5 s, and the successful picking rate was 81%.
between the brightness values φ and f.

Figure 1
Figure 1 Result of bilateral filter

Figure 2 Figure 3
Figure 2 Sample images

Figure 6
Figure 6 Structure chart of color image segmetation algorithm

Figure 7
Figure 7 Result of different light condition

Figure 8 Figure 9
Figure 8 Experimental platform of Hangzhou white chrysanthemum picking

Figure 10
Figure 10 The graph of end-effector bottom