Feature extraction

Hi everyone,

I need some advise regarding a college project that I have. Here is
what I am trying to do:

I have a bunch of automobile crash signals of various collision types.
What I would like to do is do some sort of a feature analysis based on
these signals and classify each kind of crash based on these features.
Ultimately, I want the system to identify a new crash with a certain
probability that it belongs to a certain type.

I was thinking also that some kind of genetic learning algorithm would
be nice for such a system. Someone even mentioned self organizing maps
(SOM) as a good tool for tackling such a problem.

What I would like to know is if someone can recommend some reference
or books that deals with techniques for such kind of feature
extraction. There are a lot of books, but I do not know which one
would be useful for me. They also tend to be a bit expensive and that
is why I need to choose them carefully on my student budget. I do not
have much experience with such kind of adaptive algorithms and I would
like to have some good references with me.

Thanks,

Nish

"Pankaj Daga" <pankajdaga@excite.com> wrote in message 
news:1a4b2aef.0409280835.2555e108@posting.google.com...
> Hi everyone,
>
> I need some advise regarding a college project that I have. Here is
> what I am trying to do:
>
> I have a bunch of automobile crash signals of various collision types.
> What I would like to do is do some sort of a feature analysis based on
> these signals and classify each kind of crash based on these features.
> Ultimately, I want the system to identify a new crash with a certain
> probability that it belongs to a certain type.
>
> I was thinking also that some kind of genetic learning algorithm would
> be nice for such a system. Someone even mentioned self organizing maps
> (SOM) as a good tool for tackling such a problem.
>
> What I would like to know is if someone can recommend some reference
> or books that deals with techniques for such kind of feature
> extraction. There are a lot of books, but I do not know which one
> would be useful for me. They also tend to be a bit expensive and that
> is why I need to choose them carefully on my student budget. I do not
> have much experience with such kind of adaptive algorithms and I would
> like to have some good references with me.

Nish,

I don't know of any approach that doesn't require learning - be it machine 
learning or programmer learning.

If you're going to use feature analysis then you have to identify the 
"features" first and extract them from the signatures.  That's difficult to 
do.

If you use a neural net driven by short-term spectral analysis that might be 
better - then train on collision types.

I don't know how you use genetic learning in this case but maybe that would 
work.  I'm unclear on the system implementation you might have in mind.  I 
guess you'd tweak the classifier coefficients using the genetic algorithm 
until you get the best results?  But, what is the "classifier" and what are 
the "coefficients"?

You may want to rethink the problem.  There is a huge variety of vehicle 
structures.  These structures are the acoustic generators.  So, what hope is 
there of differentiating crash *type* from vehicle types?  Ditto speeds. 
So, I envision a multidimensional space of automobile make, model and year, 
and vehicle speed and direction.  It seems to me naive to attempt to 
classify a crash sound as "rear end collision", "headon collision", "front 
to side collision" because these geometries may not be the driving factor in 
determining the acoustic signatures.  If the situation isn't more limited 
than I've described then I'd be very skeptical.

Fred 

pankajdaga@excite.com (Pankaj Daga) wrote in message news:<1a4b2aef.0409280835.2555e108@posting.google.com>...
> Hi everyone,
> 
> I need some advise regarding a college project that I have. Here is
> what I am trying to do:
> 
> I have a bunch of automobile crash signals of various collision types.
> What I would like to do is do some sort of a feature analysis based on
> these signals and classify each kind of crash based on these features.
> Ultimately, I want the system to identify a new crash with a certain
> probability that it belongs to a certain type.

This would probably amount to use some sort of signal characteristica, 
and feed them into some sort of classifyer, right?

> I was thinking also that some kind of genetic learning algorithm would
> be nice for such a system. Someone even mentioned self organizing maps
> (SOM) as a good tool for tackling such a problem.

Most classifiers I know of, tend to classify signals in terms of what 
they already know. It would be very interesting to learn of a classifier 
that actually gets able to classify a feature as "this is new, I've 
never encounetered this before". Make sure you post here if you find 
a way of doing that.
 
> What I would like to know is if someone can recommend some reference
> or books that deals with techniques for such kind of feature
> extraction. There are a lot of books, but I do not know which one
> would be useful for me. They also tend to be a bit expensive and that
> is why I need to choose them carefully on my student budget. I do not
> have much experience with such kind of adaptive algorithms and I would
> like to have some good references with me.

I like that! Buying non-curricular books from your own money (as opposed
to project money) indicates a certain enthusiasm for the subject. And yes, 
I completely agree with you in getting the good books first. I've been 
buying 'unnecessary' books for the last 15 years, and now I have some 10
to 12 shelf-meters with books, reports and journals. I think there's 
somwhere on the order of 130 - 150 volumes of books now (I have never 
had the guts to actually add up how much money I've spent...), and 
almost everybody who visit me in my office find some book they recognize 
from their own fields.

Rune

In article <f56893ae.0409282153.359a246d@posting.google.com>,
Rune Allnor <allnor@tele.ntnu.no> wrote:
>pankajdaga@excite.com (Pankaj Daga) wrote in message news:<1a4b2aef.0409280835.2555e108@posting.google.com>...
>> I have a bunch of automobile crash signals of various collision types.
>> What I would like to do is do some sort of a feature analysis based on
>> these signals and classify each kind of crash based on these features.
>> Ultimately, I want the system to identify a new crash with a certain
>> probability that it belongs to a certain type.
>
>This would probably amount to use some sort of signal characteristica, 
>and feed them into some sort of classifyer, right?
>
>> I was thinking also that some kind of genetic learning algorithm would
>> be nice for such a system. Someone even mentioned self organizing maps
>> (SOM) as a good tool for tackling such a problem.

Multi-layer perceptrons or SVMs may also be worth checking out.  
You may find the Torch library (http://www.torch.ch/) useful.  It
implements several different classification and clustering techniques.

>Most classifiers I know of, tend to classify signals in terms of what 
>they already know. It would be very interesting to learn of a classifier 
>that actually gets able to classify a feature as "this is new, I've 
>never encounetered this before". Make sure you post here if you find 
>a way of doing that.

When Pankaj Daga wrote "identify a new crash", I think he meant a crash
that was not present in the classifier training data, but belongs to one 
of the types that were present in the training data.

However, the problem Rune described of having a classifier says 
"this is new, I've never encountered this before" (in other words,
an unseen type) is also very interesting.   One term for this
is "novelty detection" (try that on Google -- and judging just from
the title, http://citeseer.ist.psu.edu/658958.html seems like a useful
hit).  I guess the idea is to have the classifier report when a feature 
vector is different from the feature vectors typically seen in training.  
For example, you could estimate a probability distribution (such as a 
mixture of Gaussians) for feature vectors using the training data.  Then, 
given a fresh feature vector, calculate its probability according to that 
distribution, and if the probability is low then say "this is new".  
 

Hi Fred,

I do not have acousic data but acceleration and pressure signals from the sensors.

Nish

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message news:<c-qdnWJcdoMwJcTcRVn-iA@centurytel.net>...
> "Pankaj Daga" <pankajdaga@excite.com> wrote in message 
> news:1a4b2aef.0409280835.2555e108@posting.google.com...
> > Hi everyone,
> >
> > I need some advise regarding a college project that I have. Here is
> > what I am trying to do:
> >
> > I have a bunch of automobile crash signals of various collision types.
> > What I would like to do is do some sort of a feature analysis based on
> > these signals and classify each kind of crash based on these features.
> > Ultimately, I want the system to identify a new crash with a certain
> > probability that it belongs to a certain type.
> >
> > I was thinking also that some kind of genetic learning algorithm would
> > be nice for such a system. Someone even mentioned self organizing maps
> > (SOM) as a good tool for tackling such a problem.
> >
> > What I would like to know is if someone can recommend some reference
> > or books that deals with techniques for such kind of feature
> > extraction. There are a lot of books, but I do not know which one
> > would be useful for me. They also tend to be a bit expensive and that
> > is why I need to choose them carefully on my student budget. I do not
> > have much experience with such kind of adaptive algorithms and I would
> > like to have some good references with me.
> 
> Nish,
> 
> I don't know of any approach that doesn't require learning - be it machine 
> learning or programmer learning.
> 
> If you're going to use feature analysis then you have to identify the 
> "features" first and extract them from the signatures.  That's difficult to 
> do.
> 
> If you use a neural net driven by short-term spectral analysis that might be 
> better - then train on collision types.
> 
> I don't know how you use genetic learning in this case but maybe that would 
> work.  I'm unclear on the system implementation you might have in mind.  I 
> guess you'd tweak the classifier coefficients using the genetic algorithm 
> until you get the best results?  But, what is the "classifier" and what are 
> the "coefficients"?
> 
> You may want to rethink the problem.  There is a huge variety of vehicle 
> structures.  These structures are the acoustic generators.  So, what hope is 
> there of differentiating crash *type* from vehicle types?  Ditto speeds. 
> So, I envision a multidimensional space of automobile make, model and year, 
> and vehicle speed and direction.  It seems to me naive to attempt to 
> classify a crash sound as "rear end collision", "headon collision", "front 
> to side collision" because these geometries may not be the driving factor in 
> determining the acoustic signatures.  If the situation isn't more limited 
> than I've described then I'd be very skeptical.
> 
> Fred

pankajdaga@excite.com (Pankaj Daga) wrote in message news:<1a4b2aef.0409282307.228ec26e@posting.google.com>...
> Hi Fred,
> 
> I do not have acousic data but acceleration and pressure signals from the sensors.
> 
> Nish

OK... this makes me think of crash tests, with crash test dummies 
and the whole package. If so, you would know what cars are involved, 
where the cars are hit and what object hits the car (or vice versa). 
What could the 'features' you are looking for be, then... I speculate 
further, along the lines that you might be interested in identifying 
the 'bumps' and 'thumps' of, say, the bumber hitting the forward panel 
of the car body, the panel hitting the radiator, the radiator hitting 
the engine block, etc. 

Could it be that what you really want, is some black box that can take 
the seismo-acoustic signature as input and tell you the chain of 
events when all those objects smash into each other, at output...?

If I'm right in my speculations about your application, I wouldn't be 
too optimistic. I have yet to see any piece of data that comes with 
'tags' of the sorts I suspect you are after. In the applications 
I know of, it's a task for a human operator (and a difficult one, 
at that) to sort out such questions.

Rune

In response to Rune's request for information regarding classifiers that
detect the presence of data that  has not been encountered previously by the
system; this is type of classifier is referred to as a novelty detector. See
link below.

http://citeseer.ist.psu.edu/658958.html


Luca

"Pankaj Daga" <pankajdaga@excite.com> wrote in message
news:1a4b2aef.0409280835.2555e108@posting.google.com...
> Hi everyone,
>
> I need some advise regarding a college project that I have. Here is
> what I am trying to do:
>
> I have a bunch of automobile crash signals of various collision types.
> What I would like to do is do some sort of a feature analysis based on
> these signals and classify each kind of crash based on these features.
> Ultimately, I want the system to identify a new crash with a certain
> probability that it belongs to a certain type.
>
> I was thinking also that some kind of genetic learning algorithm would
> be nice for such a system. Someone even mentioned self organizing maps
> (SOM) as a good tool for tackling such a problem.
>
> What I would like to know is if someone can recommend some reference
> or books that deals with techniques for such kind of feature
> extraction. There are a lot of books, but I do not know which one
> would be useful for me. They also tend to be a bit expensive and that
> is why I need to choose them carefully on my student budget. I do not
> have much experience with such kind of adaptive algorithms and I would
> like to have some good references with me.
>
> Thanks,
>
> Nish

Hi Rune,

As David pointed in the thread below, I am mostly interested ina  new
signal being classified according to the knowledge existing in the
system. However, classification of totally novel signals would also be
very interesting.

I have a lot of enthusiasm! However, the topic can get so
overwhelming, it is difficult to know where to start.

Best,

Nish

allnor@tele.ntnu.no (Rune Allnor) wrote in message news:<f56893ae.0409282153.359a246d@posting.google.com>...
> pankajdaga@excite.com (Pankaj Daga) wrote in message news:<1a4b2aef.0409280835.2555e108@posting.google.com>...
> > Hi everyone,
> > 
> > I need some advise regarding a college project that I have. Here is
> > what I am trying to do:
> > 
> > I have a bunch of automobile crash signals of various collision types.
> > What I would like to do is do some sort of a feature analysis based on
> > these signals and classify each kind of crash based on these features.
> > Ultimately, I want the system to identify a new crash with a certain
> > probability that it belongs to a certain type.
> 
> This would probably amount to use some sort of signal characteristica, 
> and feed them into some sort of classifyer, right?
> 
> > I was thinking also that some kind of genetic learning algorithm would
> > be nice for such a system. Someone even mentioned self organizing maps
> > (SOM) as a good tool for tackling such a problem.
> 
> Most classifiers I know of, tend to classify signals in terms of what 
> they already know. It would be very interesting to learn of a classifier 
> that actually gets able to classify a feature as "this is new, I've 
> never encounetered this before". Make sure you post here if you find 
> a way of doing that.
>  
> > What I would like to know is if someone can recommend some reference
> > or books that deals with techniques for such kind of feature
> > extraction. There are a lot of books, but I do not know which one
> > would be useful for me. They also tend to be a bit expensive and that
> > is why I need to choose them carefully on my student budget. I do not
> > have much experience with such kind of adaptive algorithms and I would
> > like to have some good references with me.
> 
> I like that! Buying non-curricular books from your own money (as opposed
> to project money) indicates a certain enthusiasm for the subject. And yes, 
> I completely agree with you in getting the good books first. I've been 
> buying 'unnecessary' books for the last 15 years, and now I have some 10
> to 12 shelf-meters with books, reports and journals. I think there's 
> somwhere on the order of 130 - 150 volumes of books now (I have never 
> had the guts to actually add up how much money I've spent...), and 
> almost everybody who visit me in my office find some book they recognize 
> from their own fields.
> 
> Rune

Thanks David!

I will check out the torch library.

Best,

Nish

gelbart@ICSI.Berkeley.EDU (David Gelbart) wrote in message news:<cjdju0$1c56$1@agate.berkeley.edu>...
> In article <f56893ae.0409282153.359a246d@posting.google.com>,
> Rune Allnor <allnor@tele.ntnu.no> wrote:
> >pankajdaga@excite.com (Pankaj Daga) wrote in message news:<1a4b2aef.0409280835.2555e108@posting.google.com>...
> >> I have a bunch of automobile crash signals of various collision types.
> >> What I would like to do is do some sort of a feature analysis based on
> >> these signals and classify each kind of crash based on these features.
> >> Ultimately, I want the system to identify a new crash with a certain
> >> probability that it belongs to a certain type.
> >
> >This would probably amount to use some sort of signal characteristica, 
> >and feed them into some sort of classifyer, right?
> >
> >> I was thinking also that some kind of genetic learning algorithm would
> >> be nice for such a system. Someone even mentioned self organizing maps
> >> (SOM) as a good tool for tackling such a problem.
> 
> Multi-layer perceptrons or SVMs may also be worth checking out.  
> You may find the Torch library (http://www.torch.ch/) useful.  It
> implements several different classification and clustering techniques.
> 
> >Most classifiers I know of, tend to classify signals in terms of what 
> >they already know. It would be very interesting to learn of a classifier 
> >that actually gets able to classify a feature as "this is new, I've 
> >never encounetered this before". Make sure you post here if you find 
> >a way of doing that.
> 
> When Pankaj Daga wrote "identify a new crash", I think he meant a crash
> that was not present in the classifier training data, but belongs to one 
> of the types that were present in the training data.
> 
> However, the problem Rune described of having a classifier says 
> "this is new, I've never encountered this before" (in other words,
> an unseen type) is also very interesting.   One term for this
> is "novelty detection" (try that on Google -- and judging just from
> the title, http://citeseer.ist.psu.edu/658958.html seems like a useful
> hit).  I guess the idea is to have the classifier report when a feature 
> vector is different from the feature vectors typically seen in training.  
> For example, you could estimate a probability distribution (such as a 
> mixture of Gaussians) for feature vectors using the training data.  Then, 
> given a fresh feature vector, calculate its probability according to that 
> distribution, and if the probability is low then say "this is new".

Hi Luca,

This adds another interesting dimension to the problem.

Thanks,

Nish

"Luca Notini" <L.notini@lboro.ac.uk> wrote in message news:<cjefu7$r5m$1@sun-cc204.lut.ac.uk>...
> In response to Rune's request for information regarding classifiers that
> detect the presence of data that  has not been encountered previously by the
> system; this is type of classifier is referred to as a novelty detector. See
> link below.
> 
> http://citeseer.ist.psu.edu/658958.html
> 
> 
> Luca
> 
> "Pankaj Daga" <pankajdaga@excite.com> wrote in message
> news:1a4b2aef.0409280835.2555e108@posting.google.com...
> > Hi everyone,
> >
> > I need some advise regarding a college project that I have. Here is
> > what I am trying to do:
> >
> > I have a bunch of automobile crash signals of various collision types.
> > What I would like to do is do some sort of a feature analysis based on
> > these signals and classify each kind of crash based on these features.
> > Ultimately, I want the system to identify a new crash with a certain
> > probability that it belongs to a certain type.
> >
> > I was thinking also that some kind of genetic learning algorithm would
> > be nice for such a system. Someone even mentioned self organizing maps
> > (SOM) as a good tool for tackling such a problem.
> >
> > What I would like to know is if someone can recommend some reference
> > or books that deals with techniques for such kind of feature
> > extraction. There are a lot of books, but I do not know which one
> > would be useful for me. They also tend to be a bit expensive and that
> > is why I need to choose them carefully on my student budget. I do not
> > have much experience with such kind of adaptive algorithms and I would
> > like to have some good references with me.
> >
> > Thanks,
> >
> > Nish