Now I am sure that what you are doing is sheer nonsense. Besides the dumb brute forcing, the obvious sign is the cluelessness, the other obvious sign is the secrecy. The earlier you will dismiss your priceless ideas, the better. When people do a serious research, they don't ask the stupid questions in the newsgroups. Instead, they learn the subject themselves and/or seek for the professional assistance. VLV prad wrote:> I am sorry that you think that. This is for a research subject and not > homework. I am not sharing details about the data as it would give away > the novel modeling I am trying to do. Thanks to all those who gave useful > information and helped me. > > VLV: Please think before you send a comment like that. > > > > Pradeep. >>> >>Don't you know? This is homework. A stupident is generating a huge data >>by software and then trying to make use of that data. Numerical nonsense > > >>instead of thinking of the better approaches, as usual. >> >>VLV >>

# Beginner's question on undersampling and aliasing

Started by ●June 6, 2007

Reply by ●June 9, 20072007-06-09

Reply by ●June 9, 20072007-06-09

On Jun 7, 7:11 pm, "prad" <pradeep.ferna...@gmail.com> wrote:> Ron: > Randomized Statistical Sampling is another good idea. Initially I was > thinking along another line involving random sampling. I was thinking > about producing random data samples and then performing FFT on these > samples. In fact, I did it. But since I am not that familiar with DSP and > FFT, could not really figure out how to interpret the FFT results. In > fact, most of the links I found on FFT with non-uniformly spaced samples > were interpolating to find the equally spaced samples and then performing > FFT. Is this the standard technique for FFT with non-uniformly spaced > samples? Thanks Ron for this new idea. I will investigate it further.Actually, this might be a place where trying to use a randomly sampled low pass filter might be better than nothing. Essentially create your low pass filter waveform (say a windowed sinc of some period and width), and then use that filter waveform in a weighted random number generator. Use those weighted random numbers to select sub-samples centered around the neighborhood of a sample point of interest. After some number of sub-samples, if the mean and variance seem to be converging somewhere after a sufficient number of sub-samples, then the mean might approximate the value of a decimated sample of the bandlimited signal perhaps within some statistical confidence interval. Does this type of procedure have a name? IMHO. YMMV. -- rhn A.T nicholson d.0.t C-o-M

Reply by ●June 10, 20072007-06-10

prad, If someone suggested it, then I've missed it.... here's what I would do: Because you have so many data points the frequency resolution could be quite a bit better than you need. The fewer contiguous points you use, the coarser the frequency resolution. So, I might take 1024 or 4096 or .... you choose the number ... and compute an FFT on just those contiguous samples. You might do this for various such epochs along the total sequence. While the resolution will be limited, the entire frequency range will be covered each time. If the results are quite different then you know that the spectral character of the samples is varying from segment to segment. If the results are rather similar then the opposite. Also, you'll be able to see the actual important bandwidth of the information - so you might be able to decide that some decimation is OK to do without aliasing. You should be asking yourself this question: Even though I have a huge number of samples, what is the frequency resolution that I require? The frequency resolution is the reciprocal of the temporal epoch that you choose to analyze. Example: If you have 1 second worth of samples and the sample rate is 1MHz, then you have 10^6 samples. If you FFT the whole sequence, you will have 1Hz resolution over a range 0.5MHz (fs/2). Maybe 1Hz resolution is overkill for your application. So, 0.1secs of data would be 100,000 samples with 10Hz resolution... and so forth. Pick the temporal length that gives suitable resolution. I hope this helps. Fred

Reply by ●June 10, 20072007-06-10

Fred and Ron, Thanks for your input. I'll try your suggestions. Prad.>prad, > >If someone suggested it, then I've missed it.... here's what I would do: > >Because you have so many data points the frequency resolution could bequite>a bit better than you need. The fewer contiguous points you use, the >coarser the frequency resolution. > >So, I might take 1024 or 4096 or .... you choose the number ... andcompute>an FFT on just those contiguous samples. You might do this for varioussuch>epochs along the total sequence. While the resolution will be limited,the>entire frequency range will be covered each time. >If the results are quite different then you know that the spectralcharacter>of the samples is varying from segment to segment. >If the results are rather similar then the opposite. > >Also, you'll be able to see the actual important bandwidth of the >information - so you might be able to decide that some decimation is OKto>do without aliasing. > >You should be asking yourself this question: >Even though I have a huge number of samples, what is the frequency >resolution that I require? The frequency resolution is the reciprocal of>the temporal epoch that you choose to analyze. > >Example: > >If you have 1 second worth of samples and the sample rate is 1MHz, thenyou>have 10^6 samples. If you FFT the whole sequence, you will have 1Hz >resolution over a range 0.5MHz (fs/2). Maybe 1Hz resolution is overkillfor>your application. > >So, 0.1secs of data would be 100,000 samples with 10Hz resolution... andso>forth. > >Pick the temporal length that gives suitable resolution. > >I hope this helps. > >Fred > > >

Reply by ●June 11, 20072007-06-11

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message news:aqCdnboPO6bS_PHbnZ2dnUVZ_q2pnZ2d@centurytel.net...> > So, I might take 1024 or 4096 or .... you choose the > number ... and compute an FFT on just those contiguous > samples. You might do this for various such epochs along > the total sequence. While the resolution will be limited, > the entire frequency range will be covered each time. > If the results are quite different then you know that the > spectral character of the samples is varying from segment > to segment. > If the results are rather similar then the opposite.Consider what you would see if you computed a short DFT on a low-baud-rate, highly-oversampled FSK signal. If the DFT is short enough relative to the baud rate, you will see the Mark and Space frequencies in separate DFT windows. Would you then conclude that the spectral character of the signal is varying or would you conclude that the varying spectrum characterizes the signal?> You should be asking yourself this question: > Even though I have a huge number of samples, what is the > frequency resolution that I require? The frequency > resolution is the reciprocal of the temporal epoch that > you choose to analyze. >A huge number of samples might be the result of oversampling. It might also be the result of long-term observation of a phenomenon that is undersampled. It sounds like the OP is not sure which case applies to his data.

Reply by ●June 11, 20072007-06-11

"John E. Hadstate" <jh113355@hotmail.com> wrote in message news:DL8bi.26179$dy1.22507@bigfe9...> > "Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message > news:aqCdnboPO6bS_PHbnZ2dnUVZ_q2pnZ2d@centurytel.net... > >> >> So, I might take 1024 or 4096 or .... you choose the number ... and >> compute an FFT on just those contiguous samples. You might do this for >> various such epochs along the total sequence. While the resolution will >> be limited, the entire frequency range will be covered each time. >> If the results are quite different then you know that the spectral >> character of the samples is varying from segment to segment. >> If the results are rather similar then the opposite. > > Consider what you would see if you computed a short DFT on a > low-baud-rate, highly-oversampled FSK signal. If the DFT is short enough > relative to the baud rate, you will see the Mark and Space frequencies in > separate DFT windows. Would you then conclude that the spectral character > of the signal is varying or would you conclude that the varying spectrum > characterizes the signal?John, Yes, of course I would. :-) Fred

Reply by ●June 11, 20072007-06-11

On Jun 10, 1:38 pm, "Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote:> So, I might take 1024 or 4096 or .... you choose the number ... and compute > an FFT on just those contiguous samples. You might do this for various such > epochs along the total sequence. While the resolution will be limited, the > entire frequency range will be covered each time.An fft of 1024 contiguous samples won't tell you too much about the spectral content of frequencies on the rough order of one-hundredth to one-billionth of the fft's first bin, which seems to be where the OP is looking (the first 10e6 dft bins out of circa 10e13 possible?). It still seems to me that the way to look at such huge potential data sets (the weight of every rodent in North America by longitude, or some such) is to start with some statistical sampling. What I'm wondering is if there is a name for the procedure of taking a bunch of randomly spaced samples and doing a regression fit of those samples against an set of orthogonal sinusoidal basis vectors. IMHO. YMMV. -- rhn A.T nicholson d.0.t C-o-M