algorithms for star field compression

glen herrmannsfeldt writes:

> alb <alessandro.basili@cern.ch> wrote:
> 
> (snip, I wrote)
>>> So there is a stream of similar images? In that case, you might want
>>> to do some motion estimation like MPEG's do. Figure out how the image
>>> shifted from the previous frame, encode the shift and difference between
>>> the two after the shift.
> 
>> Lots of objects do not move from frame to frame and I think that is 
>> something related to the non-uniformity of the CCD. I think I can get rid of 
>> them simply subtracting two consecutive frames (unfortunately the available 
>> memory is not enough to store two complete pictures onboard, therefore I 
>> need to do this process over a subset of the frame and "calibrate" it in 
>> steps.
> 
> You might look up references to the ICT (Integer Cosine Transform),
> specifically designed for computation limited, (and I believe memory
> limited) systems. Specifically, for image compression on the CDP1802
> processor, which has no multiply instruction. ICT is designed to
> optimize the shift and add needed by minimizing the 1 bits in
> the coefficients.
> 

thanks a lot for the hint, I will try to see what would be the result on the 
raw images already taken from the device (at a rate of 1/30minutes!) and see 
what is the result.

At a first glance it looks like the ICT will be quite helpful to compress 
the data but I haven't figured out yet how can be used to filter out the 
noise, I guess I would need a filter before the transformation is performed.

Actually, what would be nice is to see if "stars" do appear different from 
random noise in a frequency domain, at that point I can set a threshold in 
the frequency domain and apply a filter in the frequency domain.

> -- glen

Vladimir Vassilevsky writes:

> 
> 
> alb wrote:
> 
> 
>>> A lot depends on how noisy is CCD, how blurry is the image and how 
>>> much of stray light gets into the picture. In the simple case, an 
>>> algorithm like you suggested would be sufficient. In more complex 
>>> cases, you would probably have to do edge enhancement and stray light 
>>> artifacts removal before processing.
>> 
>> 
>> At the moment we are only getting raw images, just to have a feeling of 
>> how they look like, and to show them on a screen what I do is the 
>> following:
>> 
>> 1. get the mean value and sigma of the whole image;
> 
> Make a histogram of values; set the thresholds accordingly.

this work has already been done, but when there is stray light coming in it 
is not applicable anymore.

> 
>> 2. map all the values < (mean - 3*sigma) to 0x000
>> 3. map all the values > (mean + 3*sigma) to 0xFFF
> 
> The statistics of images is usually very different from Gaussian; 
> therefore N x sigma rule doesn't apply.

making the values of each pixel it is clear that there is a "pedestal" which 
has a gaussian-like distribution, even when there is stray light coming in. 
I think this is the intrinsic CCD noise from the dark current. Definitely 
the rest of the image is nothing like a gaussian, but I think the N x sigma 
still applies to separate the signal from the noise. Am I wrong?

> 
>> 4. map all pixels in between linearly between 0x000 and 0xFFF
>> 
>> As a result you have a constrast amplification which is similar (I 
>> guess) to the edge enhancement you proposed.
> 
> Google "Sobel filter"
> 

hey thanks for the hint! I must admit there's a lot about image processing 
that I have no clue about. I will certainly need to study a lot.

>> I am not sure I understood what you had in mind with "stray light 
>> artifacts removal".
> 
> I consult for money; if your project is more then idle curiosity, my 
> contact is at the web site in the signature.

Appreciate your offer and inputs, I will certainly consider the possibility 
if needs arise in the future. FYI I believe that most of what we discuss 
here is for my personal education, but my personal education is certainly 
very useful to the people whom I work for.

> 
> 
> Vladimir Vassilevsky
> DSP and Mixed Signal Design Consultant
> http://www.abvolt.com

Clay writes:

> On Jan 28, 11:46&nbsp;pm, alb <alessandro.bas...@cern.ch> wrote:
[...]
>> Given the fact that I'm far from being an expert in image processing or
>> astronometry, I would be more than happy to listen to suggestions and/or
>> references on this topic.
>>
>> Al
>>
>> [1] No stars recognition is required onboard.
> 
> 
> Check out "Phd Guiding" by Stark Labs. They have made some of their
> code open source for autoguiding your telescope.
> 

thanks for the reference, it is indeed a neat tool, but not really what I 
was looking for. It seems to me they need the user to identify the object in 
order to guide the camera towards it.

In my case it would be the software which is identifying the objects.

> Typically when one uses an autoguider, you shoot a dark frame for
> subsequent subtraction from all future frames. You identify a guide
> star (hopefully reasonably bright but not so bright as to saturate the
> ccd (you want the central image to appear peaked like a bell curve)
> and then you put a box around the star and only use the reduced size
> field data for tracking.
> 
> Are you just doing this for fun? There are quite a few CCD cameras now
> available for use as autoguiders and Stark Labs' software is free for
> anyone to use. With a good low noise camera typical tracking may be
> had to under 1/10 of pixel in the autoguider camera. Your mount on the
> other hand be an issue.

It is actually for a Star Tracker which is installed on the ISS. I find it 
fun though!

> 
> Clay
> 
> 
> 
> 
> 
> 
> 

alb <alessandro.basili@cern.ch> wrote:

(snip, I wrote)
>> You might look up references to the ICT (Integer Cosine Transform),
>> specifically designed for computation limited, (and I believe memory
>> limited) systems. Specifically, for image compression on the CDP1802
>> processor, which has no multiply instruction. ICT is designed to
>> optimize the shift and add needed by minimizing the 1 bits in
>> the coefficients.
>> 

> thanks a lot for the hint, I will try to see what would be the result on the 
> raw images already taken from the device (at a rate of 1/30minutes!) and see 
> what is the result.

> At a first glance it looks like the ICT will be quite helpful to compress 
> the data but I haven't figured out yet how can be used to filter out the 
> noise, I guess I would need a filter before the transformation is performed.

I don't know, either. It was that you seem to be memory limited that
reminded me of it. The ICT was designed as a retrofit on Galileo while
it was in space. If one had known in advance, a more powerful processor
may have been used.

> Actually, what would be nice is to see if "stars" do appear different from 
> random noise in a frequency domain, at that point I can set a threshold in 
> the frequency domain and apply a filter in the frequency domain.

-- glen