Neural Network Discussion

Everyone,
  I am creating a backpropagated multilayer perceptron ANN using a MEMs
accelerometer to distinguish between automobile types.  My question
is....

Since most likely the ANN is going to "lock" onto the vibration of the
engine would it be in my interest to convert the time data into frequency
data for the input layer?  Does it even matter?  Obvioulsy for a person it
is easier to look at in frequency but when about for a NN? Also,  any
suggestions on how many inputs I should have.  I figured my largest
frequency to be (at idle)

3000 rpm / 60 = 50 Hz.  So a whole second of data only gives me 100
samples.  Too many or too few?  Any tips on figuring this out before I hit
the trial and error stage? 

Thanks.
--Ryan

First, if you are using a neural network, I'd recommend that you also
try using a support vector machine on the same data to see which is
better. See:

http://www.csie.ntu.edu.tw/~cjlin/libsvm/

for a free svm implementation.

The real art to using any machine learning approach, which includes
neural networks and svms, is designing your input vector. There's no a
priori way that I know of to guarantee that one input representation
will out-perform another one. Though if you produce a fitness landscape
(of sets of weights for the network) which is smaller and less "hilly",
that will often lead to better performance after training because [in
simple words] it's easier to find good solutions.

What you probably need to do is research a wide range of possible input
vector formats. And, you might find that you can improve performance
dramatically by including more meaningful transformations of the
original raw data into attributes included in the input vector. Hence,
if you follow my suggestion you'll end up doing a lot of experimental
work looking at various representations of your problem and seeing what
the performance is. I.e. a fancier way of saying trial and error.

The paper by Ross Quinlan on applying Decision Trees to chess end-games
is I think quite illuminating.

http://citeseer.ist.psu.edu/context/52186/0 [reference only]

In it he ends up making up all sorts of attributes of chess boards to
feed into his learning program to make it work well. Personally I think
this exposes the limitations of input languages limited to zero-order
logics rather than really attacking the problem of classifying chess
end-games as won or lost. I.e. he really ends up partially solving the
problem before getting his learning program to finish it off. But, you
might find that doing similar leads to best performance.

If you do take the experimental approach, please make sure that you
follow proper experimental procedure for evaluating the performance of
learning algorithms. E.g. use some form of cross-validation. The
chapter on evaluating classifier performance in the book:

"Elements of Statistical Learning"
(http://www-stat.stanford.edu/~tibs/ElemStatLearn/)

by Tibshirani, Hastie, and Friedman might be useful.

There's another book "Introduction to Machine Learning" by Alpaydin

http://www.amazon.com/gp/product/0262012111/103-1527503-7491050?v=glance&n=283155&n=507846&s=books&v=glance

[apologies for the commercial link]

which also covers experimental procedure.

I'm not a real DSP person, so some of this may be a bit off-topic. But
I hope there's at least something useful or vaguely helpful for you in
this followup.

Cheers,

Ross-c

effimofunk wrote:
> Everyone,
>   I am creating a backpropagated multilayer perceptron ANN using a MEMs
> accelerometer to distinguish between automobile types.  My question
> is....
> 
> Since most likely the ANN is going to "lock" onto the vibration of the
> engine would it be in my interest to convert the time data into frequency
> data for the input layer?  Does it even matter?  Obvioulsy for a person it
> is easier to look at in frequency but when about for a NN? Also,  any
> suggestions on how many inputs I should have.  I figured my largest
> frequency to be (at idle)
> 
> 3000 rpm / 60 = 50 Hz.  So a whole second of data only gives me 100
> samples.  Too many or too few?  Any tips on figuring this out before I hit
> the trial and error stage? 
> 
> Thanks.
> --Ryan

Keeping it in the time domain with an autocorrelation function may be 
better -- I'd experiment with both.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

> 3000 rpm / 60 = 50 Hz.  So a whole second of data only gives me 100
> samples.  Too many or too few?  Any tips on figuring this out before I hit
> the trial and error stage?

Well one tip is to oversample - the engine noise does not only consist
of the fundamental period, but is highly distorted. If you only sample
the fundamental period, your automobile classification algorithm will
have a hard time (try to listen and discern to idling automobiles
situated on the road in front of your house while you are in the
bedroom with windows closed and hiding under the blanket). I would
suggest sampling at the standard audio frequencies of 44.1 or 48 kHz.

Will the automobiles only be idling and stationary? Can you expect to
have the engine ignition process on record as well?

Regards,
Andor

clemenr@wmin.ac.uk wrote:

> First, if you are using a neural network, I'd recommend that you also
> try using a support vector machine on the same data to see which is
> better.

Have to agree there.

A colleague of mine whose research is in learning theory calls neural
networks "neutral notworks".

SVMs are a more rigorous, and justifiable, way to go.

Ciao,

Peter K.

Andor,

>I would
>suggest sampling at the standard audio frequencies of 44.1 or 48 kHz.

   I may oversample some but there are a few reasons that limit the amount
of oversampling that I can achieve.
1)  The maximum bandwidth of the accelerometers is 2500 Hz.
2)  I expect a very small hum and therefore small acelleration so the
wider I set the output bandwidth of the accelerometer the more noise I see
in my ADC and hence the less precise quantization.

>Will the automobiles only be idling and stationary? 
  The automobile will be idling and stationary becuase that is how I feel
I will get the best reults to start with.  If it is not idling it would be
hard to standardize the testing.  If it were moving it just makes it even
worse.  So to start with I will leave it just purring....

>Can you expect to
>have the engine ignition process on record as well?
I could, but again I am not sure I will.

I plan on mounting these accelerometers directly to the engine so there
should be quite a good characteristic frequency based on the engine type
(i.e. inline 4, V6, V8, etc.) .

--Ryan

"effimofunk" <rfrankel@ufl.edu> wrote in message 
news:Z8ydnQhdnsQ-ju7eRVn-vA@giganews.com...
> Everyone,
>  I am creating a backpropagated multilayer perceptron ANN 
> using a MEMs
> accelerometer to distinguish between automobile types.  My 
> question
> is....
>
> Since most likely the ANN is going to "lock" onto the 
> vibration of the
> engine would it be in my interest to convert the time data 
> into frequency
> data for the input layer?  Does it even matter?  Obvioulsy 
> for a person it
> is easier to look at in frequency but when about for a NN? 
> Also,  any
> suggestions on how many inputs I should have.  I figured 
> my largest
> frequency to be (at idle)
>
> 3000 rpm / 60 = 50 Hz.  So a whole second of data only 
> gives me 100
> samples.  Too many or too few?  Any tips on figuring this 
> out before I hit
> the trial and error stage?
>
> Thanks.
> --Ryan

You should consider posting (maybe cross-posting) this 
question to
comp.ai.neural-nets.  You'll probably get some good advice 
from
there as well.

"effimofunk" <rfrankel@ufl.edu> wrote in message
news:WqWdnYJzHbubBe7enZ2dnUVZ_tednZ2d@giganews.com...
> Andor,
>
> >I would
> >suggest sampling at the standard audio frequencies of 44.1 or 48 kHz.
>
>    I may oversample some but there are a few reasons that limit the amount
> of oversampling that I can achieve.
> 1)  The maximum bandwidth of the accelerometers is 2500 Hz.
> 2)  I expect a very small hum and therefore small acelleration so the
> wider I set the output bandwidth of the accelerometer the more noise I see
> in my ADC and hence the less precise quantization.
>
> >Will the automobiles only be idling and stationary?
>   The automobile will be idling and stationary becuase that is how I feel
> I will get the best reults to start with.  If it is not idling it would be
> hard to standardize the testing.  If it were moving it just makes it even
> worse.  So to start with I will leave it just purring....
>
> >Can you expect to
> >have the engine ignition process on record as well?
> I could, but again I am not sure I will.

What if it's Diesel ????? :)



>
> I plan on mounting these accelerometers directly to the engine so there
> should be quite a good characteristic frequency based on the engine type
> (i.e. inline 4, V6, V8, etc.) .
>
> --Ryan
>
>
>

effimofunk wrote:
...
> So to start with I will leave it just purring....
...
> I plan on mounting these accelerometers directly to the engine so there
> should be quite a good characteristic frequency based on the engine type
> (i.e. inline 4, V6, V8, etc.) .

Ryan, I don't understand your application. If you can completely
control the behaviour of the vehicle, and you can mount the sensors
yourself, why don't you just look at the make of the car instead of
mounting the sensors? By the way, this accelerometer seems to be
completely outpreformed by a $5 Radio Shack microphone (2.5kHz
bandwidth?).

I assume you want this application to run under other conditions as
well - what are _those_ conditions? Or is this just for fun?

Regards,
Andor

Andor,
  The reason I am starting the way I am is to reduce the variations in
parameter space associated with the problems you mentioned.  If I cannot
obtain reliable results by mounting it myself and controlling the vehicle
I cannot expect to obtain good results with other features involved. 
 
As far as the radio shack microphone:  The MEMs accelerometers can measure
extremely low-levels of g-forces as well as having a footprint of about a
mm^2.  Enlarging the bandwidth of the filter does not help that much due
to the excessive thermal noise that you let in (i.e. extending the
bandwidth by a factor of 2 increases the noise floor of your system by
3dB).  For this reason it is usually best to use the lowest sampling rate
possible in these types of applications.

>effimofunk wrote:
>...
>> So to start with I will leave it just purring....
>...
>> I plan on mounting these accelerometers directly to the engine so
there
>> should be quite a good characteristic frequency based on the engine
type
>> (i.e. inline 4, V6, V8, etc.) .
>
>Ryan, I don't understand your application. If you can completely
>control the behaviour of the vehicle, and you can mount the sensors
>yourself, why don't you just look at the make of the car instead of
>mounting the sensors? By the way, this accelerometer seems to be
>completely outpreformed by a $5 Radio Shack microphone (2.5kHz
>bandwidth?).
>
>I assume you want this application to run under other conditions as
>well - what are _those_ conditions? Or is this just for fun?
>
>Regards,
>Andor
>
>