Reducing Spectral Leakage of Rectangular Window?

On Apr 17, 2:57&#4294967295;pm, Bryan <bryan.p...@gmail.com> wrote:
> On Tuesday, April 17, 2012 11:41:30 AM UTC-7, brent wrote:
> > On Apr 17, 2:33&#4294967295;pm, Bryan
> > &#4294967295;wrote:
>
> > > You clearly didn't read my response
>
> > Are you female?
>
> ...asks the guy who's been flooding the forum with junk by interacting with Bret. I like to help when people ask questions, but obviously not when people refuse to listen to answers.

If you are feeling under-apreciated I will make sure to buy you a
corded appliance for your next birthday.

On Tue, 17 Apr 2012 11:19:07 -0400, robert bristow-johnson
<rbj@audioimagination.com> wrote:

>On 4/17/12 11:07 AM, westocl wrote:
>> I am trying to do some spectral estimation using the DFT using a
>> rectangular window only and i would like my estimates to be as acurate as
>> possible.  The sequence to be transformed is a time series, not exactly
>> gaussian,.. (and i cant make any real assumptions about stationary and such
>> at thsi point.)
>>
>> What i do have at my liberty is the ability to chose the record length at
>> any given measurment, so heres my question.
>>
>> Could i get better spectral estimates (both amplitude AND phase) by
>> choosing the record length such that the beginning sample call it x[0] and
>> the last sample x[N-1] closely match? I guess trying to emulate what window
>> functions do.
>>
>> That is if i had a choise to do the DFT over 135 samples vs doing the
>> DFT over say 128 samples because at point x[134], that data point looks
>> closer to what i had at x[0].  Would my spectral leakage be less, because
>> the sequence looks more periodic?
>
>yeah, it would likely look better.  what you want is that x[N] (the 
>first sample you're cutting off) looks just like x[0] and you want 
>x[N-1] (also cut off) to look just like x[1], and you want x[N-2] to 
>look like x[2].  and then do a 135-point (or whatever number) DFT, *not* 
>a zero-padded 256-point DFT.
>
>then you can consider the DFT to be identical to the DFS (it is anyway) 
>and that you are transforming one periodic sequence in the time domain, 
>to another periodic sequence in the frequency domain.

Haha

Mark DeArman

On 4/17/2012 11:05 AM, westocl wrote:
> Understood... So the general feel is that I can do atleast somewhat better
> by taking a DFT record length that looks to be visually periodic instead of
> blindly taking a record length?

You haven't said anything about the complexity of doing what you have in 
mind.  It's not a trivial operation.

What if you would consider a simple window - such as a trapezoidal 
window shape?  In the limits it's a rectangle or a triangle.
If you double its length relative to a rectangular window then the 
spectral resolution isn't hurt and the spectral leakage is contained.
Yes, it's altogether a different animal but the results vs. complexity 
may warrant its consideration.

At this point it gets to "what are you wanting to accomplish?" in a 
bigger picture sense.

Fred

On Tue, 17 Apr 2012 13:05:43 -0500, westocl wrote:

>>On 4/17/2012 10:41 AM, Fred Marshall wrote:
>>> In this case you can't have both because of the selected span of the
>>> window.  And, it's likely that the value mismatch is the worst in
>>> terms of spectral spreading .. but I've not figured that out.
>>
>>And then, of course, you have the problem (if it is a problem) that the
>>spectral sample intervals might change for every sample.  That seems
>>inconvenient to say the least.
>>
>>Fred
>>
>>
> Understood... So the general feel is that I can do atleast somewhat
> better by taking a DFT record length that looks to be visually periodic
> instead of blindly taking a record length?
> 
> Everyone in agreance?

Not me.  I think you'll always be able to make things look better on the 
surface, but I don't know if you'll be able to make them better for real.

What are you really trying to find out from the data?  Maybe the FFT 
isn't the best thing to use.

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

On 4/17/2012 3:34 PM, Tim Wescott wrote:
>> Everyone in agreance?
> Not me.  I think you'll always be able to make things look better on the
> surface, but I don't know if you'll be able to make them better for real.

Tim,

As I understand the approach being discussed, it's not a generic 
approach because it can't be.  So, better *is* "better for real".  But 
maybe not "best" in even its own narrow context.

The algorithm seems to be:

1) Grab more data than you need.

2) Truncate the data at some propitious point with the objective of 
matching the values and perhaps the first derivative.  This implies a 
reasonable degree of searching I should think.  Method to be used for 
this remains unstated.

If that's the case then it's data dependent and, thus, not generic.

If that approach is used then the objective serves to avoid one sharp 
discontinuity.

If sharp discontinuities, even one, is avoided then a good deal of 
spectral spreading / "new frequencies" / aliasing is eliminated over 
having NOT done this.

The model case is starting with a sinusoid of arbitrary frequency and 
deciding to capture exactly an integer number of periods.  The result is 
that there will be *no* energy outside.  In this case the method works 
perfectly in meeting the one objective.

I suppose one could do a study:
Perhaps add sin[Kt] to sin[(K/2 + 0.5)t] where K is an even integer so 
that now the best you can do is grab a waveform with a discontinuity. 
The worst case one might imagine would be when there is exactly an 
integer number of periods of sin(Kt) and exactly 1/2 period of the other 
at the end of its windows.  Indeed, that shows that all of the spectral 
spreading is due to the latter and none due to the former.

As the composite waveform becomes more complex, or indescribable even, 
then maybe the best you can do *in this context* is the proposed method.
Again, some kind of study that measures leakage energy vs. arbitrary 
window lengths might be interesting to provide a notion of "is this sort 
of optimum or not?"
But that doesn't mean that ordinary windowing might not be better.

Fred

On Wed, 18 Apr 2012 12:27:31 -0700, Fred Marshall wrote:

> On 4/17/2012 3:34 PM, Tim Wescott wrote:
>>> Everyone in agreance?
>> Not me.  I think you'll always be able to make things look better on
>> the surface, but I don't know if you'll be able to make them better for
>> real.
> 
> Tim,
> 
> As I understand the approach being discussed, it's not a generic
> approach because it can't be.  So, better *is* "better for real".  But
> maybe not "best" in even its own narrow context.
> 
> The algorithm seems to be:
> 
> 1) Grab more data than you need.
> 
> 2) Truncate the data at some propitious point with the objective of
> matching the values and perhaps the first derivative.  This implies a
> reasonable degree of searching I should think.  Method to be used for
> this remains unstated.
> 
> If that's the case then it's data dependent and, thus, not generic.
> 
> If that approach is used then the objective serves to avoid one sharp
> discontinuity.
> 
> If sharp discontinuities, even one, is avoided then a good deal of
> spectral spreading / "new frequencies" / aliasing is eliminated over
> having NOT done this.
> 
> The model case is starting with a sinusoid of arbitrary frequency and
> deciding to capture exactly an integer number of periods.  The result is
> that there will be *no* energy outside.  In this case the method works
> perfectly in meeting the one objective.
> 
> I suppose one could do a study:
> Perhaps add sin[Kt] to sin[(K/2 + 0.5)t] where K is an even integer so
> that now the best you can do is grab a waveform with a discontinuity.
> The worst case one might imagine would be when there is exactly an
> integer number of periods of sin(Kt) and exactly 1/2 period of the other
> at the end of its windows.  Indeed, that shows that all of the spectral
> spreading is due to the latter and none due to the former.
> 
> As the composite waveform becomes more complex, or indescribable even,
> then maybe the best you can do *in this context* is the proposed method.
> Again, some kind of study that measures leakage energy vs. arbitrary
> window lengths might be interesting to provide a notion of "is this sort
> of optimum or not?"
> But that doesn't mean that ordinary windowing might not be better.

That's why I suggested that an FFT may not be the solution to the root 
problem.

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

>On Wed, 18 Apr 2012 12:27:31 -0700, Fred Marshall wrote:
>
>> On 4/17/2012 3:34 PM, Tim Wescott wrote:
>>>> Everyone in agreance?
>>> Not me.  I think you'll always be able to make things look better on
>>> the surface, but I don't know if you'll be able to make them better
for
>>> real.
>> 
>> Tim,
>> 
>> As I understand the approach being discussed, it's not a generic
>> approach because it can't be.  So, better *is* "better for real".  But
>> maybe not "best" in even its own narrow context.
>> 
>> The algorithm seems to be:
>> 
>> 1) Grab more data than you need.
>> 
>> 2) Truncate the data at some propitious point with the objective of
>> matching the values and perhaps the first derivative.  This implies a
>> reasonable degree of searching I should think.  Method to be used for
>> this remains unstated.
>> 
>> If that's the case then it's data dependent and, thus, not generic.
>> 
>> If that approach is used then the objective serves to avoid one sharp
>> discontinuity.
>> 
>> If sharp discontinuities, even one, is avoided then a good deal of
>> spectral spreading / "new frequencies" / aliasing is eliminated over
>> having NOT done this.
>> 
>> The model case is starting with a sinusoid of arbitrary frequency and
>> deciding to capture exactly an integer number of periods.  The result
is
>> that there will be *no* energy outside.  In this case the method works
>> perfectly in meeting the one objective.
>> 
>> I suppose one could do a study:
>> Perhaps add sin[Kt] to sin[(K/2 + 0.5)t] where K is an even integer so
>> that now the best you can do is grab a waveform with a discontinuity.
>> The worst case one might imagine would be when there is exactly an
>> integer number of periods of sin(Kt) and exactly 1/2 period of the
other
>> at the end of its windows.  Indeed, that shows that all of the spectral
>> spreading is due to the latter and none due to the former.
>> 
>> As the composite waveform becomes more complex, or indescribable even,
>> then maybe the best you can do *in this context* is the proposed
method.
>> Again, some kind of study that measures leakage energy vs. arbitrary
>> window lengths might be interesting to provide a notion of "is this
sort
>> of optimum or not?"
>> But that doesn't mean that ordinary windowing might not be better.
>
>That's why I suggested that an FFT may not be the solution to the root 
>problem.
>
>-- 
>My liberal friends think I'm a conservative kook.
>My conservative friends think I'm a liberal kook.
>Why am I not happy that they have found common ground?
>
>Tim Wescott, Communications, Control, Circuits & Software
>http://www.wescottdesign.com
>


> Understood... So the general feel is that I can do atleast somewhat
> better by taking a DFT record length that looks to be visually periodic
> instead of blindly taking a record length?
> 
> Everyone in agreance?

Not me.  I think you'll always be able to make things look better on the 
surface, but I don't know if you'll be able to make them better for real.

What are you really trying to find out from the data?  Maybe the FFT 
isn't the best thing to use.



I want an 'accurate' estimate the spectral phase of the signal (not to get
too much into it.. but thats what i want.) Delving into what is meant by
'accurate' is a brand new can of worms id rather not open. ML estimate,
CLB, LS... dont want to open it.

Wanting the spectral phase the reason i feel that i am forced out of having
windows as an option.  Some of the frequency components may be weak, so
leakage will just destroy the estimate even more so i was trying to avoid
it as much as i could with the liberty of being able to choose the record
lenghth... thats how the question came about.

Sounds like you want a spectral estimator based on prolate spheroidal windows
that were specifically designed to reduce spectral leakage. David Thomson 
made such an estimator a number of years ago. Google for multi-tap spectrum 
estimation. I've seen it used in geosciences for weak signals.

>Sounds like you want a spectral estimator based on prolate spheroidal
windows
>that were specifically designed to reduce spectral leakage. David Thomson

>made such an estimator a number of years ago. Google for multi-tap
spectrum 
>estimation. I've seen it used in geosciences for weak signals.
>


Those things dont care about the spectral phase. I want to estimate the
spectral phase, hense am tied into the Rectangular Window.

Anything else will introduce artifacts into the spectral phase.. im
intrested in mitigating those artifacts, so i really want to 'reduce the
spectral leakage of a rectangular window' all i can.

On Thu, 19 Apr 2012 11:29:43 -0500, westocl wrote:

>>On Wed, 18 Apr 2012 12:27:31 -0700, Fred Marshall wrote:
>>
>>> On 4/17/2012 3:34 PM, Tim Wescott wrote:
>>>>> Everyone in agreance?
>>>> Not me.  I think you'll always be able to make things look better on
>>>> the surface, but I don't know if you'll be able to make them better
> for
>>>> real.
>>> 
>>> Tim,
>>> 
>>> As I understand the approach being discussed, it's not a generic
>>> approach because it can't be.  So, better *is* "better for real".  But
>>> maybe not "best" in even its own narrow context.
>>> 
>>> The algorithm seems to be:
>>> 
>>> 1) Grab more data than you need.
>>> 
>>> 2) Truncate the data at some propitious point with the objective of
>>> matching the values and perhaps the first derivative.  This implies a
>>> reasonable degree of searching I should think.  Method to be used for
>>> this remains unstated.
>>> 
>>> If that's the case then it's data dependent and, thus, not generic.
>>> 
>>> If that approach is used then the objective serves to avoid one sharp
>>> discontinuity.
>>> 
>>> If sharp discontinuities, even one, is avoided then a good deal of
>>> spectral spreading / "new frequencies" / aliasing is eliminated over
>>> having NOT done this.
>>> 
>>> The model case is starting with a sinusoid of arbitrary frequency and
>>> deciding to capture exactly an integer number of periods.  The result
> is
>>> that there will be *no* energy outside.  In this case the method works
>>> perfectly in meeting the one objective.
>>> 
>>> I suppose one could do a study:
>>> Perhaps add sin[Kt] to sin[(K/2 + 0.5)t] where K is an even integer so
>>> that now the best you can do is grab a waveform with a discontinuity.
>>> The worst case one might imagine would be when there is exactly an
>>> integer number of periods of sin(Kt) and exactly 1/2 period of the
> other
>>> at the end of its windows.  Indeed, that shows that all of the
>>> spectral spreading is due to the latter and none due to the former.
>>> 
>>> As the composite waveform becomes more complex, or indescribable even,
>>> then maybe the best you can do *in this context* is the proposed
> method.
>>> Again, some kind of study that measures leakage energy vs. arbitrary
>>> window lengths might be interesting to provide a notion of "is this
> sort
>>> of optimum or not?"
>>> But that doesn't mean that ordinary windowing might not be better.
>>
>>That's why I suggested that an FFT may not be the solution to the root
>>problem.
>>
>>--
>>My liberal friends think I'm a conservative kook. My conservative
>>friends think I'm a liberal kook. Why am I not happy that they have
>>found common ground?
>>
>>Tim Wescott, Communications, Control, Circuits & Software
>>http://www.wescottdesign.com
>>
>>
> 
>> Understood... So the general feel is that I can do atleast somewhat
>> better by taking a DFT record length that looks to be visually periodic
>> instead of blindly taking a record length?
>> 
>> Everyone in agreance?
> 
> Not me.  I think you'll always be able to make things look better on the
> surface, but I don't know if you'll be able to make them better for
> real.
> 
> What are you really trying to find out from the data?  Maybe the FFT
> isn't the best thing to use.
> 
> 
> 
> I want an 'accurate' estimate the spectral phase of the signal (not to
> get too much into it.. but thats what i want.) Delving into what is
> meant by 'accurate' is a brand new can of worms id rather not open. ML
> estimate, CLB, LS... dont want to open it.
> 
> Wanting the spectral phase the reason i feel that i am forced out of
> having windows as an option.  Some of the frequency components may be
> weak, so leakage will just destroy the estimate even more so i was
> trying to avoid it as much as i could with the liberty of being able to
> choose the record lenghth... thats how the question came about.

What do you mean by "spectral phase"???

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com