• Pattern Classification (2nd Edition)
• Pattern Recognition and Machine Learning (Information Science and Statistics)
• Computer Manual in MATLAB to Accompany Pattern Classification, Second Edition
• The Elements of Statistical Learning
• Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

not intuitive enough
Review written by: N. Lei From
Just a quick browse through, I find that the materials are not intuitive
enough. I tried to look for the explanation for Figure 6.21, but did not
find clear explanation. Some of the deeper stuff probably can be
generated by readers once the basic stuff is discussed in detail and
intuitively. In general, for someone with an excellent math background
tries to go into the pattern recongnition field, this is NOT the book.

Excellent
Review written by: Dr. Lee Carlson From Saint Louis, Missouri USA
Many who work in artificial intelligence have commented that it is the ability of the human brain to engage in pattern recognition that gives it true intelligence. Without a quantitative measure of machine intelligence it is difficult to assess this claim, but there is no doubt that being able to implement pattern recognition and classification in a machine in a manner that enables it to distinguish objects, find profitable patterns in financial time series, teach itself how to play a game by examining the moves, identify subsequences in genome data, identify malicious behavior in networks, and detect fraudulent behavior in mortgage contracts would be a major advance in artificial intelligence and also a profitable one from a financial standpoint. Even if the machine required assistance from a human to do these tasks it would still be very useful. If it were able to do them on its own without any supervision one could justifiably describe it as being more intelligent than one that required such supervision (the counterexample to this imputation of intelligence is simple trial-and-error, which of course is unsupervised).

This book is a formal treatment of pattern recognition that is geared to a readership with a strong mathematical background and which makes as its major theme the difference between `supervised' and `unsupervised' pattern recognition, with this difference sometimes being more qualitative than what one would like. In the introduction to the book the authors make clear the distinction between these approaches, motivate the problem of the classification of features, and outline briefly the stages in the design of a pattern classification system. As is well known, supervised pattern recognition involves the use of training data, whereas unsupervised pattern recognition does not. In the latter case, it is left to the machine to find similarities in the feature vectors, and then cluster the similar feature vectors together. Researchers in the field of pattern recognition have devised an enormous number of algorithms and reasoning patterns to perform both unsupervised and supervised learning, and they have not necessarily developed these approaches in the context of machine intelligence. Thus the book could also be viewed as a mathematical theory of pattern recognition instead of one that is embedded in the field of artificial intelligence. However it is classified it is a useful and important work, and is well worth the time taken to read and study.

One of the most interesting (and esoteric) discussions is found in chapter 15 of the book. One of these concerns algorithms for `competitive learning' wherein representatives are designated and then "compete" with each other after a feature vector X is presented to the algorithm. The "winner" is the representative that is closer to X and the representatives are then updated by moving the winner toward X, with the rest remaining constant or move toward X at a slower rate. The competitive learning algorithm is parametrized by the learning rates of the winner and the losers, and the losers can have different learning rates. The investigator however selects the values of these parameters beforehand, and therefore competitive learning strictly speaking should not be classified as totally unsupervised. To be really unsupervised the competitive learning algorithm would have to make the selection of these parameters and tune them as needed to reach the convergence criterion. The authors do discuss briefly a version of the algorithm where the learning rate is variable, but the rate is still subject to certain constraints. Chapter 15 also contains a brief discussion of the use of genetic algorithms in clustering.

Another topic in the book that is both interesting and important and is still surprisingly unknown by many is that of `independent component analysis'. Independent component analysis (ICA) is a generalization of principal component analysis in that it tries to find a transformation that takes a feature vector into one whose components are mutually independent, instead of merely decorrelated. All of the random variables must be non-Gaussian in order for this technique to work, since the Gaussian case gives back the usual principal component analysis. Independent component analysis is beginning to be applied to many different areas, including finance, risk management, medical imaging, and physics. It remains to see whether it will become a standardized tool in the many mathematical and statistical software packages that exist at the present time. The authors discuss two different ways to perform independent component analysis, one being an approach based on higher order cumulants, and the other, interestingly, on mutual information. In the latter approach, the mutual information between the transformed components is calculated to be the Kullback-Leibler probability distance between the joint probability distribution of the transformed components and the product of the marginal probability densities. This distance is of course zero if the components are statistically independent. The strategy is then to find the transformational matrix that minimizes the mutual information, since this will make the components maximally independent. As the authors point out, the problem with this approach is that the elements of the transformation matrix are hidden in the marginal probability distribution functions of the transformed variables. They then outline an approach that allows them to calculate the mutual information with the assumption that the transformation matrix is unitary.

centred around clustering methods
Review written by: W Boudville From Terra, Sol 3
[A review of the 3rd EDITION 2006.]

The authors give us an indepth survey of pattern recognition methods. All sorts of ideas. Like using a neural network approach with a multilayer perceptron that has back propagation implemented. Or using a Bayesian to classify and infer. Nor do they neglect support vector machines, which is a relatively recent idea that has gained some adherants.

Much of the text centres on clustering algorithms. Sequential and hierarchical, amongst others. Notice that many of the clusters found are rather subjective. Often depending on some initial choice of parameters. Here is one place where you might have to use your expert knowledge, in choosing some clustering method that yields reasonable results for your application.

Pattern Recognition
Review written by: Edward J. Ciaccio From New York, NY
Professor Theodoridis has written an exciting new book on pattern recognition. The topic is sometimes neglected, particularly in the fields of biomedical and electrical engineering, but it is essential to the understanding of signal and image shape on a mathematical basis, including similarities and differences in shape as well as how to extract, recognize, and measure the important components. Professor Theodoridis covers all of the classic steps in pattern recognition in great detail and in a readily understood fashion: sensors and pattern extraction, features extraction and selection, clustering, classification, supervised and unsupervised recognition, and evaluation of the system. Each section is backed up with computer simulation examples so that the reader can gain practical experience while reading the book. The author discusses essential concepts for computer programming of the pattern recognition techniques that are discussed. This work is necessarily mathematical, and therefore will tend to be of greatest interest to advanced students and practicing engineers in a variety of fields. Biomedical engineering is a rapidly expanding field that is key to the improvement of health care quality. There are plenty of biomedical examples including those in the section of the book on computer-aided diagnosis, such as for the detection of cancerous lesions in x-ray mammography. The section on speech recognition will be useful to engineers who are designing turnkey pattern recognition systems that include speech recognition as input and/or for use as a security key. Also included in the work are the most recently developed topics of interest including fuzzy clustering algorithms, and neural networks using genetic and annealing methods. This comprehensive work should prove to be an invaluable tool for the library of design engineers who work with signals and images. I heartily recommend it to all with a basic engineering background.

Edward Ciaccio, PhD
Assoc. Professor of Biomedical Engineering
Columbia University in New York

An interesting pattern recognition book
Review written by: Luisa Mico From Alicante, Spain
I teach and research in pattern recognition, and for both fields I find very interesting this book. In particular, my students use it when a particular topic in the book is studied because concepts are
well described, in particular clustering methods. Also, solution manual is available from the publisher without problems. The main problem of this textbook is that other very important topics in pattern recognition are missing or briefly review
(for example, classification trees,
support vector machines or combining classifiers).