Soft Computing Based CBIR System
6.4 The STIRF System
The description of the STIRF(Shape, Texture, Intensity-distribution features with Relevance Feedback) system is presented in Table 6.1.
Problem Definition: Given a large image database consisting of n images, the ob-jective of the system is to retrieve the most relevant images from the database, based on Shape, Texture and Intensity-distribution features and Relevance Feedback.
The STIRF System: The query image is accepted as an input in RGB format and normalized to 256x256 pixels by scaling the image. The image is represented as three 256x256 RGB matrices. Histogram equalization is then performed on the im-age to distribute intensities uniformly across the imim-age. Four shape features, an 8x8 co-occurrence of edge direction matrix, a vector of length eight as edge direction histogram, a 16x16 matrix as the DCT of edge map and a 180x362 matrix as Hough
Table 6.1 Algorithm for STIRF System
1. Input the query image
2. Normalize the image to size 256x256 3. Perform histogram equalization on the image
4. Extract the shape features Hough transform, co-occurrence matrix of edge direction, edge direction histogram and DCT of the edge map.
5. Extract the texture feature by first calculating co-occurrence matrices for different displacement vectors and calculating energy, entropy, inertia and homogeneity for the different co-occurrence matrices.
6. Extract the intensity distribution feature by calculating histogram of intensities.
7. Use Kohonen SOM to cluster each feature individually.
8. Create signature for the image by taking a vector made up of winner neuron index normalized by dividing them by the number of neurons in the neural network used to cluster the feature.
9. For each of the n images in the database, do
a. Use the signature of the query image and images in the database as input to classifier and if output is greater than selected threshold, retrieve the image.
10. Rearrange the images in the order of output from classifier.
11. If relevance feedback is given by user for any of the retrieved images, use relevance feedback to train the classifier.
6.5 Performance Analysis 129 transform are extracted from the processed image. In the next step, the texture fea-tures are computed as Energy, Entropy, Homogeneity and Inertia vectors of size 24, one each for displacement vectors. Finally, the histogram of intensities, a vector of size 256 is obtained as intensity-distribution feature.
Once the feature extraction is completed, it is fed as input to a Kohonen neural network that clusters each feature independently. The number of inputs to the Ko-honen neural network is equal to the feature size. Once clustering is performed, we get eight winner indices, one for each feature as output of clustering. Next, the sig-nature for the image is created, using the clustered features. For each image in the database, the signature of the image along with signature of the query image is fed as input to the classifier.
The output of the classifier is a value between zero and one. All images in the database, that give output of more than a selected threshold for the query image are retrieved. The threshold value decides the number of images retrieved. The retrieved images are presented to user after they are rearranged according to the output from the classifier. The user relevance feedback can be used to train the classifier.
6.5 Performance Analysis
Retrieval tests are conducted with different combinations of texture, intensity and shape features. Figure 6.6 shows the first ten retrieval results based on Hough trans-form alone. The first image to the top left corner of the figure is the query image.
Here a total of three misclassifications (images balloon(6),cloud(7) and cloud(9)) can be seen. The misclassifications are due to the fact that, certain edges in the mis-classified images are responsible for producing Hough transform for these images that are similar to Hough transform of query image. Figure 6.7 shows retrieval re-sult for another query of a plane image based on the remaining three shape features
Fig. 6.6 Hough transform based retrieval
Fig. 6.7 Based on remaining three shape features
(Edge direction histogram, edge direction co-occurrence matrix and DCT of edge map). Here four (images cloud(5), shark(8), shark(9) and flower(10)) out of the top ten are misclassified due to similarity in certain edges present in the images and due to similar edge histogram being produced for the images, even though the objects in the image are different.
Figure 6.8 shows top ten query result of a cloud image based on texture fea-ture alone. There is only one misclassification here (image 6(airplane)). The mis-classification is due to the cloud background of the misclassified image which gives it similar texture feature as query image. Finally, Figure 6.9 shows top ten query results of a bird image based on intensity histogram. Here, we see seven
Fig. 6.8 Retrieval based on Texture feature
6.5 Performance Analysis 131
Fig. 6.9 Retrieval based on Intensity distribution
misclassifications (images polar bear(3), sharks(4), airplane(5), building(6), air-plane(8), tiger(9) and clouds(10)) due to similarity in intensity histograms of dis-similar images. In all these queries, the query image chosen was that image which gave best results with the chosen feature. Similarity matching of each feature is based on some factor independent of the other.
When all the heterogeneous features are combined together and similarity match-ing is done then distance between images is compared in an n dimensional space where n is the number of features. Hence the result obtained is much better and complete. Figure 6.10 shows the top 16 query result when all the features are con-sidered together. Figure 6.11 shows improvement in the performance when rele-vance feedback is used in retrieval. It can be seen that relerele-vance feedback has a significant impact on the rank of the retrieved images. Further, in the top 16, one misclassification (image 16) is resolved by the use of relevance feedback.
Fig. 6.10 Retrieval before Relevance feedback
Fig. 6.11 Retrieval after Relevance feedback
Tests were conducted on the effect of distortions in the image on the retrieval of the image to show the robustness of the system. Figure 6.12 shows the effect of contrast variations on the rank of the retrieved image. The x axis shows the variation in contrast of the image (each step by a factor of 5 units) and the y axis shows the change in the rank of the retrieved image. An interesting fact that can be observed is that lowering of contrast does not affect retrieval as much as increasing contrast of the image beyond a certain point (x = 2). This phenomenon, is due to the fact that, increasing the contrast by a large extent induces edges that previously were not detected in the low contrast image. For instance, when we consider an image of a cloud, when contrast is further increased, the sky becomes distinctly bright blue and clouds distinctly white. Hence the edge detection gives clear outline of the cloud while edge map of the low contrast image does not have such a distinct outline.
From the figure, it can be observed that our system is more robust to lowering of contrast of image than the CBIR system that uses local histogram matching. Both methods however deliver similar performance when contrast is increased.
Figure 6.13 shows effect of sharpness variation of an image on retrieval. The steps used to increase sharpness or blur the image is Gaussian filter of size 5. Positive value for variation indicates that the image is sharpened and negative value indicates that it is blurred. It can be observed that there is a symmetry in the graph about y axis.
That is, sharpening and blurring of the image have similar effects. The rank initially increases and then returns back to rank of original image. Initially, when the image is sharpened or blurred, the distortion in the image, plays a role in increasing the rank of the image. As image is further sharpened or blurred, the system considers a larger area of the image as feature, and hence maps general shape of object with query image and hence performance improves. One more interesting observation in the graph is the sharp rise in the rank when the image is sharpened beyond a particular point (x = 15). This is due to the roughly pixelating effect that occurs due to increasing sharpness of image excessively. Overall, the effect of sharpness of image has little effect on retrieval compared to other distortions as the change in rank of image is in the range of about zero to ten as can be seen in Figure 6.13.
6.5 Performance Analysis 133
0 5 10 15 20 25 30 35 40 45 50
-4 -3 -2 -1 0 1 2 3 4
Rank of image
Variation in contrast Our System Localized histogram matching
Fig. 6.12 Effect of contrast variations on retrieval
0 5 10 15 20 25
-20 -15 -10 -5 0 5 10 15 20
Rank of image
Sharpness.
Our System Localized histogram matching
Fig. 6.13 Effect of sharpness variations on retrieval
0 5 10 15 20 25
0 5 10 15 20
Rank of image
Number of pixels used to shift Our System
Localized histogram matching
Fig. 6.14 Effects of image shift on retrieval
0 10 20 30 40 50 60
0 1 2 3 4 5 6 7 8
Rank of image
Noise Fig. 6.15 Effects of noise on retrieval
6.5 Performance Analysis 135 Here, better results are obtained by our system compared to CBIR system using local histogram matching.
Often a shift effect is seen, especially in the images captured by a moving camera.
Figure 6.14 shows the effect of shift in image on the rank of retrieved image. In the experiment, image was shifted in steps of 5 pixels. Upto a critical point(x = 10) shift effect does not affect the rank of the image retrieved. Beyond the critical point, as shift effect distorts the shape of the image and hence there is a variation in the rank.
It can be observed that our system is more robust to shift in image than the local histogram matching method for image retrieval.
Figure 6.15 shows the effect of noise in image on the rank of retrieved image.
The x axis represents the amount of intentional noise added to the image. The noise added is in steps of 0.2 Gaussian noise pixels being added per pixel of image in all red green and blue channels. It can be observed from Figure 6.15 that low noise has insignificant effect on the rank of the image. As the noise increases, there is a linear increase in the rank of the image. Beyond a particular point (x = 5), addition of noise to the image does not affect the rank of the image significantly. Thus, the rank of the image stabilizes as adding noise to an already distorted image does not affect the image. In all the above experiments, the step of variation is large, so that the effect on image is clearly distinguishable to the human eye.
Description of the Database: The dataset consisting of the classes mentioned in Figure 6.16 plus some assorted images. This is considered as our test environment.
The training was on less than the half the testing data. The table in Figure 6.16 shows the general performance of the system on image database where images are classified by humans roughly based on objects present in them. It can be observed that the performance of the system depends widely on the kind and variety of objects in the image.
The recall is defined as the proportion of relevant documents retrieved and pre-cision is defined as the proportion of retrieved documents that is relevant. The Fig. 6.16 Performance of
the STIRF system Sl.No.
1 Birds 0.468 0.734 0.93 0.468
2 Sea Birds 0.88 0.94 1.00 0.88
3 Buildings 0.36 0.68 0.86 0.36
4 Clouds 0.94 0.97 1.00 0.94
5 Flowers 0.716 0.858 0.97 0.716
Hot air ballons 0.63
8 Sharks 0.342 0.671 0.71 0.342
Category Average
Precision Precision Recall Accuracy
recall, precision and the accuracy for the database is shown in Figure 6.16.
The average precision is calculated as Average Precision = Average (No. of rel-evant pictures retrieved - No. of irrelrel-evant pictures) / No. of relrel-evant pictures in the database, the average is calculated over query of all the images in the class.
6.6 Summary
We have developed a CBIR system based on the combination of heterogeneous STI features. The system is self-adaptive and improves image retrieval based on the user feedback. Unlike traditional systems, which combine features in a linear fashion, this system learns to appropriately combine features in a nonlinear fashion based on relevance feedback. Since features are clustered independently, the system re-trieves images based on various similarity factors like shape, texture and intensity variation. Since the dimension of the signature obtained from the cluster is small, the retrieval process is efficient. The key to the good performance of the system, is that the slow and expensive process of clustering is made static and the classifi-cation which is a relatively less intensive process is made dynamic which helps in increasing effectiveness of the system without compromise in efficiency. Although the proposed model is for CBIR, it can be easily extended to other type of Content Based Multimedia retrieval like audio or video by choosing appropriate features.
References
1. Chen, Y., Wang, J.Z.: A Region-Based Fuzzy Feature Matching Approach to Content-Based Image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelli-gence 24(9) (2002)
2. Sheikholeslami, G., Chang, W., Zhang, A.: SemQuery: Semantic Clustering and Query-ing on Heterogeneous Features for Visual Data. In: ProceedQuery-ings of ACM Multimedia, pp. 3–12. ACM Press, Bristol UK (1998)
3. Li, J., Wang, J.Z., Weiderhold, G.: IRM:Integrated Region Matching Approach for Image retrieval, ACM Multimedia, pp. 147–156 (2000)
4. Yu, L., Gimel’farb, G.: Image Retrieval Using Color Co-occurrence Histograms. Image and Vision Computing, NZ, pp. 41–46 (2003)
5. Srisuk, S., Fooprateepsirr, R., Petrou, M., Waraklang, S., Sunat, K.: A General Frame-work for Image Retrieval Using Reinforcement Learning. Image and Vision Computing, NZ, 36–41 (2003)
6. Faloutsos, C., Barber, R., Flinkner, M., Hafner, J., Niblack, W., Petkovick, D., Equitz, W.: Efficient and Effective Querying bi Image Content. Journal of Intelligent Information Systems: Integrating Artificial Intelligence and Database Technologies 3(3-4), 231–262 (1994)
7. Niblack, W., Barber, R., Equitz, W., Flickner, M., Glasman, E., Petkovick, D., Yanker, P., Faloutsos, C., Taubin, G.: The QBIC Project: Querying Images by Content Using Color, Texture and Shape. In: Proc. of SPIE, in Storage and Retrieval for Image and Video Database, vol. 1908, pp. 173–187 (February 1993)
References 137
8. Picard, R.W., Kabir, T.: Finding Similar Patterns in Large Image Databases. IEEE, Min-neapolis V, 161–164 (1993)
9. Kadyrov, A., Petrou, M.: The Trace Transform and Its Applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(8), 811–828 (2001)
10. Wolf, C., Jolion, J.M., Kropatsch, W., Bischof, H.: Content based Image Retrieval using Interest Points and Texture Features. In: Proc. of International Conference on Pattern Recognition, pp. 4234–4237 (2000)
11. Distasi, R., Nappi, M., Tucci, M., Vitulano, S.: CONTEXT: A Technique for Image Re-trieval Integrating CONtour and TEXTure Information. In: Proc. of International Con-ference on Image Analysis and Processing, pp. 224–229 (2001)
12. Manjunath, B.S., Ma, W.Y.: Texture Features for Browsing and Retrieval of Large Image Data. IEEE Transactions on Pattern Analysis and Machine Intelligence (Special issue on digital libraries) 18(8), 837–842 (1996)
13. Gupta, A., Jain, R.: Visual Information Retrieval. Communications of ACM 40(5), 70–79 (1997)
14. Malhotra, S., Rui, Y., Ortega-Binderberger, M., Huang, T.S.: Supporting Content-based Over Images in MARS. In: Proc. IEEE Int. Conf. on Multimedia Computing and Sys-tems, pp. 632–638 (1997)
15. Ma, W.Y., Manjunath, B.: NeTra: A Toolbox for Navigating Large Image Databases. In:
Proc. IEEE Int. Conf. Image Processing, pp. 568–571 (1997)
16. Carson, C., Thomas, M., Belongie, S., Hellerstien, J.M., Malik, J.: Blobworld: A System for Region-Based Image Indexing and Retrieval. In: Proc. Visual Information Systems, pp. 509–516 (1999)
17. Jain, A.K., Vailaya, A.: Image Retrieval Using Color and Shape. Pattern Recogni-tion 29(8), 1233–1244 (1996)
18. Haralick, R., Shanmugam, K., Dinstein, I.: Textual Features for Image Classification.
IEEE Transactions on Systems, Man, and Cybernetics (2000)
Abstract. Stock market prediction is a complex and tedious task that involves the processing of large amounts of data, that are stored in ever growing databases. The vacillating nature of the stock market requires the use of data mining techniques like clustering for stock market analysis and prediction. Genetic algorithms and neural networks have the ability to handle complex data. In this chapter, we propose a fuzzy based neuro-genetic algorithm - Fuzzy based Evolutionary Approach to Self Organizing Map(FEASOM) to cluster stock market data. Genetic algorithms are used to train the Kohonen network for better and effective prediction. The algorithm is tested on real stock market data of companies like Intel, General Motors, Infosys, Wipro, Microsoft, IBM, etc. The algorithm consistently outperformed regression model, backpropagation algorithm and Kohonen network in predicting the stock market values.
7.1 Introduction
Financial forecasting involves the gathering and processing of enormous amount of data. The future stock market values depend upon various market variables. The market variables like past returns, dividend yields, default spreads, term spreads, level of short term interest rates, and value line predictions, are usually considered for prediction. Even parameters like gross exposure, net exposure, concentration, and volatility in real-time affect the future stock values. Thus the data required for predicting stock market values are high dimensional in nature and reducing its dimensionality results in loss of information. For a human expert, predicting the market using the stock variables is a laborious and error prone task. Stock market prediction involves extraction of information on the correlations between the stock market values from the complex and enormous stock market data, and presents an interesting and challenging data mining task.
Data mining is a process of extracting nontrivial, valid, novel and useful informa-tion from large databases. Clustering is a promising approach to mine stock market data and it attempts to group similar objects based on features in actual data [1].
K.R. Venugopal, K.G. Srinivasa, L.M. Patnaik: Soft Comput. for Data Min. Appl., SCI 190, pp. 139–166.
springerlink.com c Springer-Verlag Berlin Heidelberg 2009
140 7 Fuzzy Based Neuro - Genetic Algorithm for Stock Market Prediction Machine learning techniques like neural network and genetic algorithms have been successfully used to cluster large amount of data.
In backpropagation networks, errors are propagated back during training and us-ing these errors weights are adjusted. Errors in the output determine the errors in hidden layer, which are used as a basis for adjustment of connection weights be-tween the input and hidden layers. Adjusting two sets of weights bebe-tween the pairs of layers and recomputing the outputs is an iterative process that is carried out until the errors fall below a tolerance level. Learning rate parameters scale the adjust-ments to weights and a momentum parameter is used to overcome local minima [2].
The objective of a Kohonen network is to map input vectors (patterns) of arbitrary dimension N onto a discrete map with one or two dimensions. The Kohonen layer works on the idea of neighborhood. Each node has a set of neighbors; when a node wins a competition, the weights of the winning node and its neighbors are changed.
Further the neighbor is from the winner, smaller is the change in its weight. The Kohonen layer is composed of neurons that compete with each other. The weights of the winner and its neighbors are brought closer to input pattern during training and hence the output obtained are clusters, that are topologically ordered [2].
Genetic algorithms are examples of evolutionary computing methods which are highly nonlinear, multifaceted search process. It searches for the best solution in a large set of candidate solutions. Genetic algorithms can be considered as com-putational models consisting of starting set of individuals(P), crossover technique, mutation algorithm, fitness function and an algorithm that iteratively applies the crossover and mutation techniques to P using fitness function to determine the best individuals in P. The fitness function is determined by the goal of the genetic algo-rithm [3]. Fuzzy Inference Systems apply logics on fuzzy sets, and use a defuzzifier to obtain crisp outputs. In cases where the required output is not always crisp and
Genetic algorithms are examples of evolutionary computing methods which are highly nonlinear, multifaceted search process. It searches for the best solution in a large set of candidate solutions. Genetic algorithms can be considered as com-putational models consisting of starting set of individuals(P), crossover technique, mutation algorithm, fitness function and an algorithm that iteratively applies the crossover and mutation techniques to P using fitness function to determine the best individuals in P. The fitness function is determined by the goal of the genetic algo-rithm [3]. Fuzzy Inference Systems apply logics on fuzzy sets, and use a defuzzifier to obtain crisp outputs. In cases where the required output is not always crisp and