Down Sampling Questions: Theoretical vs Practical

gmcauley wrote:
> 
> Thanks Jim for your reply.
> 
> >But I'm guessing that what you are
> >talking about is writing software so that a pair of human eyeballs can
> >look at the data on a computer screen or perhaps look at 2 sets of data
> >for comparison.
> 
> Don't give them in any ideas!! They have not asked me for this yet!!<j>
> 
> >You haven't provided any specific details of what exactly you are trying
> >to do or how you are going to do it.
> 
> Other group members wanted smaller data sets for visually 'epoching' and
> comparing trends between data different data types (eg, eeg vs tissue
> oxygen).  I just wanted to be sure that down sampling would not lead to
> significant information loss in the 'interesting' parts of the data.

Well i still don't know what visually looking at the data exactly means,
but it strikes me that there is a theoretical meaning to "information
loss" and what is "important" and "interesting" and there is also a
practical meaning to those terms. 
	If you sample at 40 KHz you should be able to reconstruct a sin wave at
19 KHz with no loss at all. But in a practical sense there will be
significant "information loss" to the viewer if you are viewing 19 kHz
signal which is sampled at 40 kHZ on a computer screen as opposed to
viewing a display of the same signal sampled at 200 kHz.

	At any rate it seems to me that the best way to avoid information loss
of any kind is to not modify the data at all but to have the ability to
modify the display of that data at will.


-jim

----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

gmcauley wrote:

> [SNIP]
>>You haven't provided any specific details of what exactly you are trying
>>to do or how you are going to do it. 
> 
> Other group members wanted smaller data sets for visually 'epoching' and
> comparing trends between data different data types (eg, eeg vs tissue
> oxygen).  I just wanted to be sure that down sampling would not lead to
> significant information loss in the 'interesting' parts of the data.
> 

I think you are using the phrase "frequencies 20Hz and below" 
differently than this group.

In your reply to me you stated:
      I will probably choose 25Hz - 30Hz as the maximum frequency to
      be sure to get good fidelity of the beta band with max 20-30Hz
      (it seems that people define it differently???)

If http://en.wikipedia.org/wiki/Beta_wave and
http://en.wikipedia.org/wiki/Image:Eeg_beta.svg reasonably typical of 
your data, you may have a problem.

I count 51 zero crossings which implies a fundamental of ~25 Hz. I would 
not feel comfortable with a filter cutoff of <50 Hz. Implies minimum 
sampling frequency of 100 Hz.

gmcauley wrote:


> Other group members wanted smaller data sets for visually 'epoching' and
> comparing trends between data different data types (eg, eeg vs tissue
> oxygen).  I just wanted to be sure that down sampling would not lead to
> significant information loss in the 'interesting' parts of the data.

OK, so what you really want is the "compressed" view of data, right? 
Kind of what they show in the audio wave editors? This has nothing to do 
with Nyquist downsampling.

The idea is very simple: if a pack of samples is compressed into one 
sample for display, calculate one value from that pack. What to 
calculate depends on what is the most important for you. For example, it 
can be a maximum or minimum value, or average, or RMS of the pack.

Vladimir Vassilevsky

DSP and Mixed Signal Design Consultant

http://www.abvolt.com

On Aug 16, 7:20 am, Vladimir Vassilevsky <antispam_bo...@hotmail.com>
wrote:
> gmcauley wrote:
> > Other group members wanted smaller data sets for visually 'epoching' and
> > comparing trends between data different data types (eg, eeg vs tissue
> > oxygen).  I just wanted to be sure that down sampling would not lead to
> > significant information loss in the 'interesting' parts of the data.
>
> OK, so what you really want is the "compressed" view of data, right?
> Kind of what they show in the audio wave editors? This has nothing to do
> with Nyquist downsampling.
>
> The idea is very simple: if a pack of samples is compressed into one
> sample for display, calculate one value from that pack. What to
> calculate depends on what is the most important for you. For example, it
> can be a maximum or minimum value, or average, or RMS of the pack.

What the high quality graphics types do is not just reduce
the "pack" to a single sample, but to a density histogram,
for instance a single pixel wide grey bar slightly darker
at the bottom.  It can be equivalent to a two-dimensional
anti-aliasing of a 2D plot of the original 1D data.  If
needed, max-min bounds or "error" bars can be overlayed
on the resulting graphic in a different color.  e.g. color
can be used to put back some of the information lost by
filtering and down sampling in several different ways.


IMHO. YMMV.
--
rhn A.T nicholson d.0.t C-o-M

Thanks to all for your continued replies.

Sorry, fwiw, I think I can clarify a few things ...


>I think you are using the phrase "frequencies 20Hz and below" 
>differently than this group.
>
>In your reply to me you stated:
>      I will probably choose 25Hz - 30Hz as the maximum frequency to
>      be sure to get good fidelity of the beta band with max 20-30Hz
>      (it seems that people define it differently???)
>

My part of this project is to do some spectral analysis of EEG data.   The
beta band is the highest frequency band that we are interested in.  At the
*time of my original post*, one of our group was using (erroneously?) 20Hz
as the upper end for the beta band.  Since then I have seen (eg, Wikipedia)
25Hz used.  Also 30Hz is used as a lower bound for the gamma band (the next
band above beta).  Since I want to be sure that I can faithfully recover
*all* of the beta band frequencies, I *now* will probably use 30Hz as the
low-pass cutoff.  So, I hope this clarifies the 20, 25-30, and the 20-30Hz
values/ranges used throughout the thread and referred to above.



>Well i still don't know what visually looking at the data exactly means,
>but it strikes me that there is a theoretical meaning to "information
>loss" and what is "important" and "interesting" and there is also a
>practical meaning to those terms.

>At any rate it seems to me that the best way to avoid information loss
>of any kind is to not modify the data at all but to have the ability to
>modify the display of that data at will.


>OK, so what you really want is the "compressed" view of data, right?
>Kind of what they show in the audio wave editors? This has nothing to do
>with Nyquist downsampling.


Other members in the group want to compare the EEG data with other types
of data recorded in parallel (eg, tissue oxygen).  They want to compare
for example if during a low oxygen period (say 20min), if the average EEG
voltage changes with average tissue oxygen.  There tool of choice is the
Excel spreadsheet.  Therefore 50000 instead of 250000 data points will
make it easier to do their statistics and summary bar graphs, etc.  These
are the sort of things I meant when I referred to analyzing the data
'visually' and 'manually'.

For me, I am doing spectral analysis and finding things like spectral edge
frequencies for epochs of interest.  The eeg acquisition software has a
down sampling option, and I want to be able to tell them: it is ok to down
sample but not below x amount, and not until you low-pass filter with a y
cutoff frequency.  Thanks to your kind help, I am feeling confident that x
and y can be chosen to adequately reduce the data and still allow accurate
spectral analysis of the data.

Sorry for not making this clear earlier on.  Your time is much
appreciated.  I try to err on the side of conciseness and not
verboseness.

gmcauley wrote:
> Thank you for your reply.
> 
> 
>> You state that you categorizing traces visually which suggests shape is 
>> important. That suggests that at least the 5th harmonic of your 
>> fundamental is significant.
> 
> I am not exactly sure about the relationship of the harmonics to the shape
> - please enlighten.
> 
> There are perhaps two points related to the visual analysis to help
> clarify our case:  Much of EEG data is historically evaluated visually,
> and there are other data traces (eg, tissue oxygenation) that need to be
> visually (at least initially) related to the EEG.
> 
> 
> 
>> Another question to ask if your sampling system was particularly 
>> design/chosen to measure the process of current interest.
>>
>> Why did original designer/specifier chose 200 samples/sec?
>>
> 
> One of the data types (my involvement) is EEG.  The so called gamma band
> ranges from 26-100Hz.  So I guess this is why 200Hz seems to be used
> often.
> 
>> Are you sure that 20 Hz is actually the highest frequency of interest?
> 
> In our case, we are not really interested in the gamma band.  I will
> probably choose 25Hz - 30Hz as the maximum frequency to be sure to get
> good fidelity of the beta band with max 20-30Hz (it seems that people
> define it differently???)
> 
> So I am really wanting to know I guess for a low-pass cut off of 30Hz,
> what is a reasonable resample rate.
> 
You should get a general-purpose tool like Matlab (which costs $$$$) or 
Scilab (which costs nothing but your time, and not too much of that if 
you're already a Matlab jock).  You should use this tool to try out 
different filters.

I know that EKG traces have specific features that are will messed up 
with ringing caused by too-sharp cutoffs and by phase distortion.  I'm 
not even sure that you _can_ adequately characterize such a waveform 
just by the frequency of the sampling.

I don't know if this is the case with EEG traces, but you may want to do 
a literature search to see if there are any good compression methods 
specifically for EEG.  You're proposing a lossy compression method that 
throws away something you don't think you need (gamma band) and keeps 
something you _do_ think you need (beta band).  Were I you I'd do a 
literature search for good lossy compression methods for raw EEG data to 
see if there's something out there that will be more help yet.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html