Defishing fisheye images

When it comes to wide lenses, fisheye lenses are the very widest available. The currently widest rectilinear (non-fisheye) lens for the Micro Four Thirds system is the Panasonic Lumix 7-14mm f/4 zoom lens. At the widest setting, it has a diagonal field of view of 114°. This is quite impressive, however, a full frame fisheye lens has a diagonal field of view of 180°.

Two full frame fisheye lenses for the Micro Four Thirds system are the Lumix G 8mm f/3.5 and the Samyang 7.5mm f/3.5. The Lumix lens is rather expensive, but good quality. The Samyang is cheaper, does not feature autofocus, and has good optical qualities.

A disadvantage of fisheye lenses is of course the fisheye projection: They do not feature rectilinear projections like we are used to, but spherical projections. This means that any line not passing through the centre of the frame will appear bent. The further from the centre a line passes, the more bent it becomes.

However, fisheye images can be transformed to rectilinear images, to look more natural. In this article, I use the free software Hugin to transform the image, also called defishing.

Defishing photos

Here is an example image taken with the Samyang 7.5mm f/3.5 at f/5.6. It features Sergels Torg in Stockholm, with straight lines in three directions in the tiles (click to enlarge):

As you can see, the lines that pass near the centre of the frame are the least bent.

To transform the image to rectilinear, I use the Hugin program to defish it. Hugin can do panorama stitching, but in this case, I use it solely to defish an image.

First, click the "Load images..." dialogue button, and load your image into the program:

Next, insert the details about the lens. Here, I am using a Full Frame Fisheye, with a focal length of 7.5mm and a 2x crop factor:

In my experience, using the lens type "Orthographic" also gives good results. Having done this, it is time to tell the program what kind of output projection we want. This can be done by clicking the "Fast Preview panorama" button in the middle top of the window:

This brings up the dialogue window where you can choose the output projection and field of view. Choose "Rectilinear" projection, and a field of view of something like 140° by 110° in the "Projection" tab:

Finally, it is time to save the output file, which is done in the "Stitcher" interface, since this program is, after all, a panorama stitcher program. First, click "Calculate Optimal Size", and then "Stitch Now...":

This brings up a file save dialogue window. Note that the output file will be enormous, and you need to open it in an image editing program, e.g., The Gimp, to crop and resize it. After cropping and resizing, the image looks like this:

There are a lot of things to note about the defished image:

The lines are now (fairly) straight. I did not use a specialized profile for the Samyang 7.5mm f/3.5 fisheye lens, but a generic profile. So the output image is perhaps not perfectly rectilinear. But it is good enough for most uses, I think.

Also, note that the aspect ratio of the image changed. The original image had an aspect ratio of 1.33 (4/3). The defished image has an aspect ratio of 1.92. This is not a mistake. Keep in mind that the fisheye image is more compressed the further from the centre we get. Hence, in the fisheye projection, we pack more angle of view the further from the centre we get. Since the original image was rectangular, this gets exaggerated after defishing.

Finally, the corners of the defished image have been stretched a lot, and the effective resolution is smaller in the corners than in the centre. This can be illustrated by looking at 100% crops from both the original and defished images, from the lower left corner:

Contrary to intuition, perhaps, the fisheye image actually looks more natural in the corner.

In the corner, we also see that the Chromatic Aberration artifacts get exaggerated after defishing. We can see a more pronounced green fringing around the black figure in the image above.

I have also used the "Orthographic" lens model for defishing an image, and compared it with an image taken with the Lumix G 7-14mm f/4 ultra wide angle zoom lens. The transformed rectilinear image from the Samyang 7.5mm f/3.5 lens is even wider than that of the Lumix wide angle lens at 7mm, and very sharp, too.

Defishing video

My example above looked at defishing a still image. It is also possible to defish video. I have used the Kdenlive video editing software for this effect. Right click on the video timeline, and select "Defish":

This brings up the "Defish" video effect settings:

I've found that using the "Equiarea" type, and an amount of 855 works well. Alternatively, use "Stereographic" and an amount of 825. Or you can experiment and see what you like the most.

Using this technique, I have made an example of a video sequence both without and with defishing. The video shows the street gamblers often seen in the Stockholm old city:



Again, we see that the output video is almost rectilinear. For most practical purposes, this is good enough. Also, in the corners, the resolution is not as good as in the centre, just as we saw in the image example above. This is especially evident when viewing the video at 1080p.

Conclusion

Defishing image and video is easy. The output images are not perfect, but considering how extremely wide they are, I'd say they are pretty good.

To avoid the stretched look in the corners, you could apply somewhat less defishing effect. This will leave some residual barrel distortion, but also leave more resolution in the corners. This could be an option if you have an image with few straight lines.

In this article, I have looked at how much wider the defished output image is compared with using a rectilinear wide angle lens.