Beginner's question on undersampling and aliasing

Now I am sure that what you are doing is sheer nonsense. Besides the 
dumb brute forcing, the obvious sign is the cluelessness, the other 
obvious sign is the secrecy. The earlier you will dismiss your priceless 
ideas, the better.

When people do a serious research, they don't ask the stupid questions 
in the newsgroups. Instead, they learn the subject themselves and/or 
seek for the professional assistance.

VLV


prad wrote:
> I am sorry that you think that. This is for a research subject and not
> homework. I am not sharing details about the data as it would give away
> the novel modeling I am trying to do. Thanks to all those who gave useful
> information and helped me. 
> 
> VLV: Please think before you send a comment like that. 
> 
> 
> 
> Pradeep.
> 

>>
>>Don't you know? This is homework. A stupident is generating a huge data 
>>by software and then trying to make use of that data. Numerical nonsense
> 
> 
>>instead of thinking of the better approaches, as usual.
>>
>>VLV
>>

On Jun 7, 7:11 pm, "prad" <pradeep.ferna...@gmail.com> wrote:
> Ron:
>    Randomized Statistical Sampling is another good idea. Initially I was
> thinking along another line involving random sampling. I was thinking
> about producing random data samples and then performing FFT on these
> samples. In fact, I did it. But since I am not that familiar with DSP and
> FFT, could not really figure out how to interpret the FFT results. In
> fact, most of the links I found on FFT with non-uniformly spaced samples
> were interpolating to find the equally spaced samples and then performing
> FFT. Is this the standard technique for FFT with non-uniformly spaced
> samples? Thanks Ron for this new idea. I will investigate it further.

Actually, this might be a place where trying to use a randomly
sampled low pass filter might be better than nothing.
Essentially create your low pass filter waveform (say a
windowed sinc of some period and width), and then use that
filter waveform in a weighted random number generator.  Use
those weighted random numbers to select sub-samples centered
around the neighborhood of a sample point of interest.  After
some number of sub-samples, if the mean and variance seem to be
converging somewhere after a sufficient number of sub-samples,
then the mean might approximate the value of a decimated sample
of the bandlimited signal perhaps within some statistical
confidence interval.

Does this type of procedure have a name?



IMHO. YMMV.
--
rhn A.T nicholson d.0.t C-o-M

prad,

If someone suggested it, then I've missed it.... here's what I would do:

Because you have so many data points the frequency resolution could be quite 
a bit better than you need.  The fewer contiguous points you use, the 
coarser the frequency resolution.

So, I might take 1024 or 4096 or .... you choose the number ... and compute 
an FFT on just those contiguous samples.  You might do this for various such 
epochs along the total sequence.  While the resolution will be limited, the 
entire frequency range will be covered each time.
If the results are quite different then you know that the spectral character 
of the samples is varying from segment to segment.
If the results are rather similar then the opposite.

Also, you'll be able to see the actual important bandwidth of the 
information - so you might be able to decide that some decimation is OK to 
do without aliasing.

You should be asking yourself this question:
Even though I have a huge number of samples, what is the frequency 
resolution that I require?  The frequency resolution is the reciprocal of 
the temporal epoch that you choose to analyze.

Example:

If you have 1 second worth of samples and the sample rate is 1MHz, then you 
have 10^6 samples.  If you FFT the whole sequence, you will have 1Hz 
resolution over a range 0.5MHz (fs/2).  Maybe 1Hz resolution is overkill for 
your application.

So, 0.1secs of data would be 100,000 samples with 10Hz resolution... and so 
forth.

Pick the temporal length that gives suitable resolution.

I hope this helps.

Fred 

Fred and Ron,

      Thanks for your input. I'll try your suggestions. 


Prad.




>prad,
>
>If someone suggested it, then I've missed it.... here's what I would do:
>
>Because you have so many data points the frequency resolution could be
quite 
>a bit better than you need.  The fewer contiguous points you use, the 
>coarser the frequency resolution.
>
>So, I might take 1024 or 4096 or .... you choose the number ... and
compute 
>an FFT on just those contiguous samples.  You might do this for various
such 
>epochs along the total sequence.  While the resolution will be limited,
the 
>entire frequency range will be covered each time.
>If the results are quite different then you know that the spectral
character 
>of the samples is varying from segment to segment.
>If the results are rather similar then the opposite.
>
>Also, you'll be able to see the actual important bandwidth of the 
>information - so you might be able to decide that some decimation is OK
to 
>do without aliasing.
>
>You should be asking yourself this question:
>Even though I have a huge number of samples, what is the frequency 
>resolution that I require?  The frequency resolution is the reciprocal of

>the temporal epoch that you choose to analyze.
>
>Example:
>
>If you have 1 second worth of samples and the sample rate is 1MHz, then
you 
>have 10^6 samples.  If you FFT the whole sequence, you will have 1Hz 
>resolution over a range 0.5MHz (fs/2).  Maybe 1Hz resolution is overkill
for 
>your application.
>
>So, 0.1secs of data would be 100,000 samples with 10Hz resolution... and
so 
>forth.
>
>Pick the temporal length that gives suitable resolution.
>
>I hope this helps.
>
>Fred 
>
>
>

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in 
message 
news:aqCdnboPO6bS_PHbnZ2dnUVZ_q2pnZ2d@centurytel.net...

>
> So, I might take 1024 or 4096 or .... you choose the 
> number ... and compute an FFT on just those contiguous 
> samples.  You might do this for various such epochs along 
> the total sequence.  While the resolution will be limited, 
> the entire frequency range will be covered each time.
> If the results are quite different then you know that the 
> spectral character of the samples is varying from segment 
> to segment.
> If the results are rather similar then the opposite.

Consider what you would see if you computed a short DFT on a 
low-baud-rate, highly-oversampled FSK signal.  If the DFT is 
short enough relative to the baud rate, you will see the 
Mark and Space frequencies in separate DFT windows.  Would 
you then conclude that the spectral character of the signal 
is varying or would you conclude that the varying spectrum 
characterizes the signal?

> You should be asking yourself this question:
> Even though I have a huge number of samples, what is the 
> frequency resolution that I require?  The frequency 
> resolution is the reciprocal of the temporal epoch that 
> you choose to analyze.
>

A huge number of samples might be the result of 
oversampling.  It might also be the result of long-term 
observation of a phenomenon that  is undersampled.  It 
sounds like the OP is not sure which case applies to his 
data.

"John E. Hadstate" <jh113355@hotmail.com> wrote in message 
news:DL8bi.26179$dy1.22507@bigfe9...
>
> "Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message 
> news:aqCdnboPO6bS_PHbnZ2dnUVZ_q2pnZ2d@centurytel.net...
>
>>
>> So, I might take 1024 or 4096 or .... you choose the number ... and 
>> compute an FFT on just those contiguous samples.  You might do this for 
>> various such epochs along the total sequence.  While the resolution will 
>> be limited, the entire frequency range will be covered each time.
>> If the results are quite different then you know that the spectral 
>> character of the samples is varying from segment to segment.
>> If the results are rather similar then the opposite.
>
> Consider what you would see if you computed a short DFT on a 
> low-baud-rate, highly-oversampled FSK signal.  If the DFT is short enough 
> relative to the baud rate, you will see the Mark and Space frequencies in 
> separate DFT windows.  Would you then conclude that the spectral character 
> of the signal is varying or would you conclude that the varying spectrum 
> characterizes the signal?

John,

Yes, of course I would.  :-)

Fred 

On Jun 10, 1:38 pm, "Fred Marshall" <fmarshallx@remove_the_x.acm.org>
wrote:
> So, I might take 1024 or 4096 or .... you choose the number ... and compute
> an FFT on just those contiguous samples.  You might do this for various such
> epochs along the total sequence.  While the resolution will be limited, the
> entire frequency range will be covered each time.

An fft of 1024 contiguous samples won't tell you too much
about the spectral content of frequencies on the rough
order of one-hundredth to one-billionth of the fft's first
bin, which seems to be where the OP is looking (the first
10e6 dft bins out of circa 10e13 possible?).

It still seems to me that the way to look at such huge
potential data sets (the weight of every rodent in North
America by longitude, or some such) is to start with
some statistical sampling.  What I'm wondering is if
there is a name for the procedure of taking a bunch of
randomly spaced samples and doing a regression fit of
those samples against an set of orthogonal sinusoidal
basis vectors.


IMHO. YMMV.
--
rhn A.T nicholson d.0.t C-o-M