Richard wrote:> Hi Andor, > > What's your method's difference with entropy method?If you have a sequence (X_k) of iid normal distributed random variables, Clay's quantization results in a squence (X_q_k), where X_q is iid uniform distributed on the set {1, 2, ..., N}. I still haven't figured how you are going to mix the X_q with real numbers, so I can't comment on the quantization error using this method.> I guess they are > similar. Can you post it. I am very interested in it ;-)Let's see: you are allowed to specify N points on the real line, call them {x_1, ..., x_N}, and you are given a random variable X that is N(0, 1) distributed (you can transform this to any general normal distribution if required). The process of quantizing is to choose k such | X - x_k | is minimal, ie. q := argmin_{k \in {1, .., N}} | X - x_k |. The quantized value is then the random variable X_q := x_q. An obvious criterion for minimiziation is the mean squared error, ie. MSE := E[ (X - X_q)^2 ]. However, you can use any error weighting function ew(x) and then minimize E[ ew(|X - X_q|) ] (the MSE is the special case ex(x) = x^2). Think of this when you calculate the density function f(x) further down, there might be some ew(x) that makes this particularly easy. The task: set the N points so as to minimize the expectation of the error function. For starters, one can compute the distribution function of the error. Notational shortcuts: y_k := (x_{k+1} - x_k) / 2, and d_k := min( d , y_k ) for k=2, 3, ...., N-1. Then the distribution function F: R+ -> [0,1[ of the error is F(d) := P[ |X-X_q| < d ] = sum_{k=2}^{N-1} ( \Phi(x_k+d_k)-\Phi(x_k - d_{k-1}) ) + \Phi(x_1 + d_1) - \Phi(x_1 - d) + \Phi(x_N + d) - \Phi(x_N - d_{N-1}) where \Phi(x) is the distribution function of the standard normal distribution. You can now compute the density function f(x) of the quantization error by f(x) := d/dx F(x) which you can calculate yourself by noting that d/dx \Phi(x_k + d_k) = 1/sqrt(2 Pi) exp(-(x_k-d_k)^2) 1_{x < d_k} and similarly for \Phi(x_k - d_{k-1}). The term 1_{x < d_k} means 1_{x < d_k} = 1, if x < d_k, else 0. Once you have computed the density, you can then compute the integral E[ ew(|X - X_q|) ] = integral_{-infty}^{infty} ew(x) f(x) dx, or, specifically, the mean square error MSE as integral_{-infty}^{infty} x^2 f(x) dx. Note that the MSE is a function of the points {x_1, ..., x_N}, ie. MSE = MSE(x_1, ..., x_N). Perhaps this is minimizable analytically, otherwise use some numeric method. Regards, Andor