zqchen wrote:> Consider an unknown DC level in a white Gaussian noise with zero mean > and unknown variance. The value of the DC level is restricted to a > range of [A0, A1], the observations, that is the DC level plus WGN are > also restricted in this range. Then what is the optimal estimator for > the DC level and the variance of the noise?So, the signal + noise is clipped to the range [A0, A1]; is that right? Then make a histogram of values. The maximum of histogram will be at DC level; the shape of the histogram gives you a part of the gaussian hat; estimate the variance from there. Question: - Why it is impossible to have sex in the middle of the Red Square in Moscow? Answer: - Because every idiot bystander will try to give you his invaluable advice. Vladimir Vassilevsky DSP and Mixed Signal Design Consultant http://www.abvolt.com
What is the optimal estimator?
Started by ●December 3, 2011
Reply by ●December 5, 20112011-12-05
Reply by ●December 5, 20112011-12-05
On Mon, 05 Dec 2011 06:15:20 -0800, Rune Allnor wrote:> On 5 Des, 15:02, zqchen <zhiqun.c...@gmail.com> wrote: >> On 12月5日, 下午2时46分, Fred Marshall <fmarshallxremove_th...@acm.org> >> wrote: >> >> >> >> >> >> > Why is the problem statement not a contradiction in terms? >> >> > Pixels are all positive. That's a reasonable assertion. The noise is >> > all positive. That's another reasonable assertion. If so, then >> > there's no way to separate out the "DC" and the noise. A lowpass on >> > the noise will yield a positive value. >> >> > If the noise is white Gaussian, that must be before quantization and >> > limiting then. So, the noise amplitude distribution must have a >> > spike at 255 - maybe big and maybe small. Maybe the value of that >> > spike helps measure the noise variance?? And, the distribution of >> > values will peak at zero also won't it? (because it's zero mean). >> > Anyway, it's not Gaussian any more is it? So when does that >> > description apply? >> >> > To be "DC" it can't change .. ever. >> >> > There was no description of the number of temporal samples of these >> > pixels. There seems to be an assumption that there is more than one >> > sample in time. Maybe reasonable, maybe implied but not stated. >> >> > Fred >> >> It may be viewed as multiple samples from a gaussian distribution with >> unknown mean and unknown variance. And the sample values are clipped in >> some range. The mean also lies in this range. It will get worse if >> assume a distribution other than gaussian, right? > > You can do this in two ways: > > 1) Assume a simple Gaussian distribution which is known to > be wrong for the problem, as Fred explained, but where the estimators > you ask for are known and listed in textbooks. > > 2) Find a different distirbution that conforms to the actual > situation, but where you have to the hard work yourself, as > previously explained by Tim. > > RuneI mentioned option 1, also -- and if you aren't seeing much clipping, then it'll do pretty darn well, and it'll let you see results a lot sooner, perhaps before you die of old age or your company dies from lack of products on the shelf. My option 2 has, for high clipping rates, the potential to be very fragile -- it will be highly dependent on the noise PDF actually being Gaussian, or whatever you are assuming, and as the clipping rates go up the estimated values will vary more and more. It's the sort of thing that, if a customer were insisting on it, I would say something along the lines of "I'll do the work, I'll take your money, and I'll stand behind the theory. But I won't guarantee you anything at all about how it'll work in the real world". But I'd only do it after I explained to the customer that the whole approach was probably a waste of time, money and effort. -- My liberal friends think I'm a conservative kook. My conservative friends think I'm a liberal kook. Why am I not happy that they have found common ground? Tim Wescott, Communications, Control, Circuits & Software http://www.wescottdesign.com
Reply by ●December 5, 20112011-12-05
On 5 Des, 16:13, Jerry Avins <j...@ieee.org> wrote:> On 12/5/2011 1:46 AM, Fred Marshall wrote: > > > > > > > Why is the problem statement not a contradiction in terms? > > > Pixels are all positive. That's a reasonable assertion. > > The noise is all positive. That's another reasonable assertion. > > If so, then there's no way to separate out the "DC" and the noise. > > A lowpass on the noise will yield a positive value. > > > If the noise is white Gaussian, that must be before quantization and > > limiting then. So, the noise amplitude distribution must have a spike at > > 255 - maybe big and maybe small. Maybe the value of that spike helps > > measure the noise variance?? And, the distribution of values will peak > > at zero also won't it? (because it's zero mean). Anyway, it's not > > Gaussian any more is it? So when does that description apply? > > > To be "DC" it can't change .. ever. > > > There was no description of the number of temporal samples of these > > pixels. There seems to be an assumption that there is more than one > > sample in time. Maybe reasonable, maybe implied but not stated. > > I keep coming back to my earlier question. How can there be a > substantial DC level in a signal specified to be zero mean?The signal was indicated by the OP to be an image, which is not zero mean. Rune
Reply by ●December 5, 20112011-12-05
On 12/5/2011 9:13 AM, Rune Allnor wrote:> On 5 Des, 16:13, Jerry Avins<j...@ieee.org> wrote: >> On 12/5/2011 1:46 AM, Fred Marshall wrote: >> >> >> >> >> >>> Why is the problem statement not a contradiction in terms? >> >>> Pixels are all positive. That's a reasonable assertion. >>> The noise is all positive. That's another reasonable assertion. >>> If so, then there's no way to separate out the "DC" and the noise. >>> A lowpass on the noise will yield a positive value. >> >>> If the noise is white Gaussian, that must be before quantization and >>> limiting then. So, the noise amplitude distribution must have a spike at >>> 255 - maybe big and maybe small. Maybe the value of that spike helps >>> measure the noise variance?? And, the distribution of values will peak >>> at zero also won't it? (because it's zero mean). Anyway, it's not >>> Gaussian any more is it? So when does that description apply? >> >>> To be "DC" it can't change .. ever. >> >>> There was no description of the number of temporal samples of these >>> pixels. There seems to be an assumption that there is more than one >>> sample in time. Maybe reasonable, maybe implied but not stated. >> >> I keep coming back to my earlier question. How can there be a >> substantial DC level in a signal specified to be zero mean? > > The signal was indicated by the OP to be an image, > which is not zero mean. > > RuneI was just wondering how there can be zero mean noise in image data? That is, since there are no negative values? Zero mean implies zero noise in that case. Of course, I'm assuming that the image data goes from 0-255 and 0 means *zero*. Fred
Reply by ●December 5, 20112011-12-05
On 12/5/2011 3:47 PM, Fred Marshall wrote:> On 12/5/2011 9:13 AM, Rune Allnor wrote: >> On 5 Des, 16:13, Jerry Avins<j...@ieee.org> wrote:...>>> I keep coming back to my earlier question. How can there be a >>> substantial DC level in a signal specified to be zero mean? >> >> The signal was indicated by the OP to be an image, >> which is not zero mean. >> >> Rune > > I was just wondering how there can be zero mean noise in image data? > That is, since there are no negative values? Zero mean implies zero > noise in that case. > > Of course, I'm assuming that the image data goes from 0-255 and 0 means > *zero*. > > FredI think I get it. The noise is zero mean, and the signal isn't. Well, the noise _was_ zero mean before clipping, and if there's not much clipping, its mean isn't far off zero. In that case, a low-pass filter gives the mean pretty accurately. Since images are (ordinarily) finite, we don't need specify which filter might be optimal, simply add all the pixels and divide by the number of them. That's what the mean is, eh? Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Reply by ●December 5, 20112011-12-05
On 12/5/2011 1:38 PM, Jerry Avins wrote:> On 12/5/2011 3:47 PM, Fred Marshall wrote: >> On 12/5/2011 9:13 AM, Rune Allnor wrote: >>> On 5 Des, 16:13, Jerry Avins<j...@ieee.org> wrote: > > ... > >>>> I keep coming back to my earlier question. How can there be a >>>> substantial DC level in a signal specified to be zero mean? >>> >>> The signal was indicated by the OP to be an image, >>> which is not zero mean. >>> >>> Rune >> >> I was just wondering how there can be zero mean noise in image data? >> That is, since there are no negative values? Zero mean implies zero >> noise in that case. >> >> Of course, I'm assuming that the image data goes from 0-255 and 0 means >> *zero*. >> >> Fred > > I think I get it. The noise is zero mean, and the signal isn't. Well, > the noise _was_ zero mean before clipping, and if there's not much > clipping, its mean isn't far off zero. In that case, a low-pass filter > gives the mean pretty accurately. Since images are (ordinarily) finite, > we don't need specify which filter might be optimal, simply add all the > pixels and divide by the number of them. That's what the mean is, eh? > > JerryBut images are always unipolar as in always positive. In this case the image data varies from 0 to 255. Now, I can imagine zero mean noise added to the image data and THEREAFTER have the composite represented from 0 to 255. In which case the composite will be clipped at 0 AND at 255. But I can't imagine zero mean noise represented by itself by 0 to 255 if zero is zero. To make any sense at all, zero would have to be around 128 for the noise. And then added to image data that's also from 0 to 255. if the temporal window is great enough then Vlad suggested an approach for the noise.... Fred
Reply by ●December 6, 20112011-12-06
On 12/5/2011 9:43 PM, Fred Marshall wrote:> On 12/5/2011 1:38 PM, Jerry Avins wrote:...>> I think I get it. The noise is zero mean, and the signal isn't. Well, >> the noise _was_ zero mean before clipping, and if there's not much >> clipping, its mean isn't far off zero. In that case, a low-pass filter >> gives the mean pretty accurately. Since images are (ordinarily) finite, >> we don't need specify which filter might be optimal, simply add all the >> pixels and divide by the number of them. That's what the mean is, eh? >> >> Jerry > > But images are always unipolar as in always positive. > In this case the image data varies from 0 to 255. > Now, I can imagine zero mean noise added to the image data and > THEREAFTER have the composite represented from 0 to 255. In which case > the composite will be clipped at 0 AND at 255. > But I can't imagine zero mean noise represented by itself by 0 to 255 if > zero is zero. To make any sense at all, zero would have to be around 128 > for the noise. > And then added to image data that's also from 0 to 255. > > if the temporal window is great enough then Vlad suggested an approach > for the noise....Here's the original post: Consider an unknown DC level in a white Gaussian noise with zero mean and unknown variance. The value of the DC level is restricted to a range of [A0, A1], the observations, that is the DC level plus WGN are also restricted in this range. Then what is the optimal estimator for the DC level and the variance of the noise? I see nothing in your message, mine, or the original question that is contradictory if taken together. Am I ossifying? Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Reply by ●December 6, 20112011-12-06
On Dec 7, 12:58=A0am, Jerry Avins <j...@ieee.org> wrote:> On 12/5/2011 9:43 PM, Fred Marshall wrote: > > > On 12/5/2011 1:38 PM, Jerry Avins wrote: > > =A0 =A0... > > > > > > > > > > >> I think I get it. The noise is zero mean, and the signal isn't. Well, > >> the noise _was_ zero mean before clipping, and if there's not much > >> clipping, its mean isn't far off zero. In that case, a low-pass filter > >> gives the mean pretty accurately. Since images are (ordinarily) finite=,> >> we don't need specify which filter might be optimal, simply add all th=e> >> pixels and divide by the number of them. That's what the mean is, eh? > > >> Jerry > > > But images are always unipolar as in always positive. > > In this case the image data varies from 0 to 255. > > Now, I can imagine zero mean noise added to the image data and > > THEREAFTER have the composite represented from 0 to 255. In which case > > the composite will be clipped at 0 AND at 255. > > But I can't imagine zero mean noise represented by itself by 0 to 255 i=f> > zero is zero. To make any sense at all, zero would have to be around 12=8> > for the noise. > > And then added to image data that's also from 0 to 255. > > > if the temporal window is great enough then Vlad suggested an approach > > for the noise.... > > Here's the original post: > > Consider an unknown DC level in a white Gaussian noise with zero mean > and unknown variance. The value of the DC level is restricted to a > range of [A0, A1], the observations, that is the DC level plus WGN are > also restricted in this range. Then what is the optimal estimator for > the DC level and the variance of the noise? > > I see nothing in your message, mine, or the original question that is > contradictory if taken together. Am I ossifying? > > Jerry > -- > Engineering is the art of making what you want from things you can get.Is there any problem in my question?
Reply by ●December 6, 20112011-12-06
On Tue, 06 Dec 2011 15:18:55 -0800, zqchen wrote:> On Dec 7, 12:58 am, Jerry Avins <j...@ieee.org> wrote: >> On 12/5/2011 9:43 PM, Fred Marshall wrote: >> >> > On 12/5/2011 1:38 PM, Jerry Avins wrote: >> >> ... >> >> >> >> >> >> >> >> >> >> >> I think I get it. The noise is zero mean, and the signal isn't. >> >> Well, the noise _was_ zero mean before clipping, and if there's not >> >> much clipping, its mean isn't far off zero. In that case, a low-pass >> >> filter gives the mean pretty accurately. Since images are >> >> (ordinarily) finite, we don't need specify which filter might be >> >> optimal, simply add all the pixels and divide by the number of them. >> >> That's what the mean is, eh? >> >> >> Jerry >> >> > But images are always unipolar as in always positive. In this case >> > the image data varies from 0 to 255. Now, I can imagine zero mean >> > noise added to the image data and THEREAFTER have the composite >> > represented from 0 to 255. In which case the composite will be >> > clipped at 0 AND at 255. But I can't imagine zero mean noise >> > represented by itself by 0 to 255 if zero is zero. To make any sense >> > at all, zero would have to be around 128 for the noise. >> > And then added to image data that's also from 0 to 255. >> >> > if the temporal window is great enough then Vlad suggested an >> > approach for the noise.... >> >> Here's the original post: >> >> Consider an unknown DC level in a white Gaussian noise with zero mean >> and unknown variance. The value of the DC level is restricted to a >> range of [A0, A1], the observations, that is the DC level plus WGN are >> also restricted in this range. Then what is the optimal estimator for >> the DC level and the variance of the noise? >> >> I see nothing in your message, mine, or the original question that is >> contradictory if taken together. Am I ossifying? >> >> Jerry >> -- >> Engineering is the art of making what you want from things you can get. > > Is there any problem in my question?"Consider an unknown DC level in a white Gaussian noise with zero mean and unknown variance." could either be parsed as meaning "a DC level with zero mean, corrupted by white Gaussian noise", or "a DC level corrupted by white, zero mean, Gaussian noise". I think most of us just assumed that you must have meant the latter, because the former does not make sense. -- My liberal friends think I'm a conservative kook. My conservative friends think I'm a liberal kook. Why am I not happy that they have found common ground? Tim Wescott, Communications, Control, Circuits & Software http://www.wescottdesign.com
Reply by ●December 7, 20112011-12-07
On 12/6/2011 3:18 PM, zqchen wrote:> > Is there any problem in my question?Since you mentioned pixels, which are usually only positive-valued, I'd understand better if you were to tell us if the noise is zero-mean before the sampling/digitizing limits are imposed. I suspect that's the case but it's not clear. Example: DC value at 128 before sampling / quantization. Noise zero mean with variance of 90 is added to the DC value. This means that the noisy data is both positive and negative at times. Now sample and quantize between 0 and 255. The negative-going noise is limited at 0. The positive-going noise is limited at 255. The mean of the data before sampling is 128. Since it's 128 then the mean after sampling is apparently also 128. But, if the data were something different, say 64 and the noise is as above, then after sampling there will be more sample points at 0 than at 255. The mean before sampling will be 64. The mean after sampling will be something else I do believe. Does this match with your situation? Fred
