Monday, February 1, 2010

The other part of the solution

The bottom photo was created using 5 synthetic images with uniform and trivial photometric parameters, and the top photo is the SR image that my code currently produces.

Using the code from http://www.robots.ox.ac.uk/~vgg/hzbook/code/, I now have the ability to automatically compute a homography between two images, given only the images as input. The pipeline uses Harris corners, followed by RANSAC used to fit a homography.

I created a synthetic dataset with uniform lighting parameters (so the photometric parameters are trivial and fixed), and ran Pickup's code using the estimated homographies. The figures attached to this post are of a low-res image and a corresponding super-resolved image. The SR image was created using 5 such low-res images. I believe the gray border in the bottom and right of the SR image to be an artifact of the final viewing angle I choose, and trivially fixed.

Note that the high-res image is of significantly higher quality than the low-res image. It's not clear to me whether this is mainly due to the additional information from the other low-res images, or mainly due to Pickup's code knowing the exact blur kernel used in the synthetic data generation. One way to know for sure is to try the code on real images, but to do that, I'll have to implement photometric registration code.

Note the extreme black and white aliasing on some of the columns in the image. I can reduce the aliasing by increasing the weight of the Huber prior, at the risk of over-smoothing the image.

Because of the extreme sensitivity of the SR process to accurate image registration, improving the registration will probably be more effective than fine-tuning the prior. To do this, I plan to use bundle adjustment, which simultaneously registers several image planes. I will likely use the code at http://vision.ucsd.edu/~vrabaud/toolbox/doc/, which appears to be well-commented and likely high-quality. But first, I need to understand how bundle adjustment works, which will require some background reading.

No comments:

Post a Comment