Sunday, 28 October 2012

Continuous AF during video with the GH2

The Micro Four Thirds mirrorless cameras use a totally different autofocus method than traditional SLR and DSLR cameras. The M4/3 cameras use a technique called Contrast Detection Autofocus (CDAF), whereas SLR systems typically use Phase Detection Autofocus (PDAF), also called Phase Difference Autofocus.

CDAF relies on jogging the autofocus back and forth to find the setting with the most contrast, in which case the camera assumes the image is in focus. For still images, this works very well. My study shows that both the GH1 and GH2 feature very fast autofocus, especially with the kit zoom lenses. I don't think that SLR cameras can do this much faster, at least not faster in a noticeable way.

The GH2 can sample the contrast 120 times per second (120fps), which allows for a faster autofocus, given that the lens is also capable on reacting fast enough. The GH1 could only sample at 60fps.

The Panasonic GH3, due to be available late 2012, further improves upon this by allowing for 240fps CDAF readout. However, Panasonic says that at the moment, only one lens supports this, the Lumix X 12-35mm f/2.8, and only after firmware update, which was announced today, coincidentally.

Continuous autofocus during video recording, though, is one area where the Micro Four Thirds cameras currently do not perform very well.

One thing to keep in mind, is that when taking still images, the sensor is free to be used for AF before you snap the image. When recording video, though, the sensor is busy scanning your video stream, and cannot be used for high speed CDAF sampling in the same way.

The GH2 can record videos at three different rates: 24fps, 25fps and 50fps (24fps, 30fps and 60fps for American versions). The 24fps, 25fps and 30fps modes are popular for their progressive image stream, and high bit rate. However, keep in mind that in these modes, the camera can only sample the contrast at a low rate, hence, I guess that autofocus during video will also be slower.

The interlaced 1080i mode, 50fps for the European version and 60fps for the American version, can scan the image scene twice as fast, and hence one can guess that the AF is also faster.

The test

I have tried to test this with a controlled case. I rigged up a picture which moves back and forth (using a LEGO Technic mechanism). This is used to test the autofocus performance in both the 50i and 25p modes of the GH2, using three different lenses, the Lumix X 45-175mm f/4-5.6, Lumix G HD 14-140mm f/4-5.8 and Olympus 45mm f/1.8. All of these lenses are marketed as video optimized.

Here is a video showing the test setup, and the outcomes of the tests:



Analysis

By looking at the videos, I think it is quite clear that the focus is kept better when recording at 50fps, compared with 25fps. To be more sure, I looked through the first full cycle when using the Lumix X 45-175mm f/4-5.6. The cycle from the rear position, moving forwards and then backwards again to the same position takes 140 frames, or 5.6 seconds.

At 50fps, I counted 71 frames in focus (51%), while at 25fps, the number of frames in focus was 59 (42%). This is hardly a scientific analysis, but I think it confirms that my theoretical guess also holds in practice: When recording at a higher frames per second rate, the continuous autofocus works better.

Conclusion

When the continuous autofocus performance is important, you could consider video recording at a higher frames per second rate. This gives the camera a better chance of keep up the focus, given that there is sufficient light.

On the other hand, the GH2 camera can only provide the interlaced mode at 50fps/60fps at 1080 lines resolution. The interlaced mode probably gives slightly less perceived sharpness anyway.

Future cameras are expected to provide the progressive mode at 1080 lines resolution (1080p) at 50fps/60fps. With this capability, I would certainly recommend using 1080p at 50fps/60fps when the autofocus performance is crucial, and when there is sufficient light.

Technical improvement potential

One could imagine improvement potential within the technical limitations of the autofocus system. As I have demonstrated in this article, the frames per second rating affects the autofocus effectiveness. As the sensor is used to record the video stream, it cannot at the same time be used for autofocus at a faster rate.

However, it is quite often that you don't use a 360° shutter. A 360° shutter means that the shutter stays open all the time. Hence, at 25fps video recording, the shutter speed is 1/25s. At this fps and shutter speed combination, the sensor is always busy reading the video stream.

However, what if you use a 180° shutter? At 25fps, that would correspond to 1/50s shutter speed. At this rate, there is a 1/50s gap between each frame which could be used for reading out CDAF information, see the illustration below:


Using this method, there is room for one exposure, and two additional CDAF readouts per frame when using a 180° shutter. Perhaps the GH2 uses this technique today, I don't know. These technical details tend to be secrets.

Alternative techniques for AF during video

The Micro Four Thirds cameras so far only use CDAF for autofocus, both for still images, and during video. There are some other techniques out there as well.

The Sony Single-Lens Translucent (SLT) cameras use a permanent (non-flapping) mirror which is semi-transparent. This mirror allows for using traditional SLR style PDAF sensors, meaning that the camera can operate the focus very quickly also during video recording.

This has an additional advantage over CDAF: There is no need to jog the focus back and forth to find the maximum contrast, as the Micro Four Thirds cameras do. Hence, you will see that the cameras do not "overshoot" the focus. When needed, the focus changes to the correct distance at the first attempt, given that there is sufficient light.

The disadvantage of the SLT system is the requirement for a traditional mirror box, making the cameras larger and more complicated, as well as requiring the lenses to be designed for a longer register distance, resulting in larger and heavier lenses.

The Nikon 1 system has an alternative approach. It uses an imaging sensor where some of the photosites are PDAF sensors. This allows for a hybrid CDAF and PDAF system: The camera can choose which system to use depending on the situation. In theory, this is the best of both worlds, as it can keep the register distance short, while allowing for faster autofocus during video recording. I have not tried the Nikon 1 myself, though, so I don't know the real world merits of the system.

According to interviews with designers and engineers, there are no plans to change the autofocus system of the Micro Four Thirds cameras. In theory, it should be possible to improve the CDAF system with better and faster image processing. Hence, we can expect the continuous autofocus to gradually improve with newer cameras.

Saturday, 20 October 2012

Autofocus noise comparison

Traditionally, lens focus mechanisms have been rather simple, just a cam which pushes the lens elements back and forth as the focus ring turns. Moving the lens assembly away from the film/sensor gives you a shorter focus distance.

In the Micro Four Thirds family of lenses, this mechanism is rather unusual. Only the two first pancake prime lenses, the Olympus 17mm f/2.8 and the Lumix G 20mm f/1.7 feature this type of mechanism, in which all the lens elements move during focus.

Most other lenses feature internal focus, meaning that only one or some of the lens elements inside the assembly move during focus. This has many advantages: It is usually more quiet, the lens can be more solid since there are no moving elements on the outside, and it is usually less noisy.

On the negative side, this type of focus mechanism can give you a change of effective focal length as the focus moves. This is not much of a problem for still photography, but if you are focusing during video recording, it can lead to what is called "focus breathing", as the image zooms in and out a bit as the focus jogs back and forth.

Also, internal focus can lead to changing geometric distortion characteristics as the focus changes. For the Lumix G 14mm f/2.5 pancake lens, I have seen that it gives a bit of barrel distortion at close focus.

In this article, I will look at the noise characteristics. When operating the focus, there is some noise, and it can vary between lenses. To test this, I mounted the lenses to the Panasonic GH1 in AF-C mode. When half pressing the shutter release button in this mode, the camera makes the focus jog back and forth continuously. This is a good mode for testing the focus noise.

To measure the noise, I put a smartphone next to the lens, with the microphone 1.5 cm from the lens barrel (about 1/2 inch).

Here is a video demonstration of the noise test:



LensNoise reading
Lumix G 8mm f/3.5 fisheye67 dB
Lumix G 14mm f/2.5 Pancake69 dB
Lumix G 20mm f/1.7 Pancake75 dB
Sigma 30mm f/2.8 EX DN70 dB
Olympus M.ZD 45mm f/1.870 dB
Panasonic Leica Lumix DG Macro-Elmarit 45mm f/2.8 1:1 Macro70 dB
Lumix G HD 14-140mm f/4-5.866 dB
Lumix G 14-42mm f/3.5-5.668 dB
Lumix X PZ 14-42mm f/3.5-5.666 dB
Lumix G 45-200mm f/4-5.666 dB
Lumix X PZ 45-175mm f/4-5.666 dB

Conclusion

As expected, the Lumix G 20mm f/1.7 pancake lens has the most noisy autofocus, with the old fashioned focus mechanism. It is also the slowest to focus in my experience.

As for the zoom lenses, they are all pretty similar in terms of autofocus noise. The Lumix G HD 14-140mm f/4-5.8 was marketed as a video optimized lens with very silent autofocus, but the study shows that the other zoom lenses are pretty much comparable in this respect. In terms of autofocus speed, it is also comparable with the rest. So to me, it seems that the video optimization of this lens is to a large degree a matter of marketing only.

You'll notice that the prime lenses are generally more noisy than the zoom lenses. I would guess that this is due to two reasons mostly: The prime lenses generally have larger apertures, meaning that they have fairly large lens elements that take more effort to move during the focus. Also, the larger aperture means that they must be more precise in terms of focus: With a smaller maximum aperture, the zoom lenses can allow themselves to be more sloppy in terms of where to set the focus, as the depth of field is larger. For a large aperture prime lens, the focus must be more exact.

With the exception of the Olympus 17mm f/2.8 pancake lens and the Lumix G 20mm f/1.7 pancake lens, most Micro Four Thirds lenses are pretty much noiseless, and you should not worry about the focus noise.

Friday, 12 October 2012

Aspherical lenses

There has been a lot of improvement to camera technology lately. The fact that we have affordable consumer digital cameras is one thing, another is the improvement to image quality. Lately, one could argue that the image quality has reached a plateau level, and we are now mostly seeing improvements to features, like video and live view, and to high ISO capabilities.

However, it is easy to forget that we have also had significant development in the lens technology.

Most modern lenses involve some kind of optical compromises. This can be, for example, a large zoom range, a bright (fast) lens, a compact pancake design, very wide angle lenses. These lens types require clever and complicated optical designs.


Now, I don't claim to be an expert in optical design, but from what I understand, using aspherical lens elements is a key to avoiding poor image quality when there are significant optical compromises that must be met.

Today, it is not easy to find lenses that do not complicated like this. Most lenses involve a zoom, a large aperture, a compact design, a wide angle, or even some combination of these. Exceptions from this is the Sigma 30mm f/2.8 EX DN. It is not a compact lens, it is not very fast, at f/2.8. It is not a zoom lens, and finally, at 30mm, which is significantly more than the register distance, it does not require a retrofocal design.

For this reason, the Sigma Sigma 30mm f/2.8 EX DN has a very simple optical design, with only seven spherical lens elements in five groups, and still achieves an impressive image quality. In fact, I would argue, for the ultimate in image quality, one should look to simple lens designs like this: Not too fast, not too wide, and no zoom. These lenses are relatively easy to design, and in theory, perform the best. In practice, though, some lens manufacturers would invest the most prestige in designing fast prime lenses, and for that reason, the may be among the best performers as well.

Traditionally, most lens elements have been spherical, meaning that the surfaces, convex or concave, shaped like parts of a sphere surface. These are most easy to produce, and has been used for centuries, ever since Galileo Galilei produced spherical concave lenses by grinding glass sheets with cannon balls.

Aspherical lens elements have one or two surfaces which are not spherical. They can have some more complicated geometrical shape to optimize the sharpness across the image frame or other optical properties.

To produce large aspherical lens elements cheaply is a key factor in making affordable consumer lenses. So let's see what has happened with Panasonic's lenses over the time span of Micro Four Thirds. Here is the ratio of aspherical lenses used per lens, plot against the announcement date:


Some lenses are not in the list, for example the Lumix G 45-200mm f/4-5.6, Lumix G 100-300mm f/4-5.6, and Lumix G 8mm f/3.5 fisheye, as they have no apsherical lens elements.

In the diagram, we see for example that the Lumix G 14mm f/2.5 pancake lens is rated at 50% since it has three aspherical lens elements, out of a total of six.

In general, we can see that there is a trend towards a higher ratio of aspherical lens elements used.

The next diagram shows the diameter in mm of the largest aspherical lens element per lens:


In the diagram, we see that apart from the premium HD 14-140mm and wide angle 7-14mm lenses launched early in the timeline, there is a tendency towards using larger aspherical lens elements.

Finally, this diagram shows the average total aspherical lens area per lens price in dollars. So a high value means more aspherical lens elements for the price:


This picture shows most clearly that aspherical lenses have become less expensive to produce. The ultra compact Lumix X 14-42mm f/3.5-5.6 is on the top of the list, and achieves an impressive set of features in a compact form factor.

The new value tele zoom lens, the Panasonic Lumix G 45-150mm f/4-5.6 is also high on the list, indicating a high value for money in terms of lens design.

In the context of industrial design, the Lumix G 14-42mm f/3.5-5.6 basic kit zoom is one of my favourites. Considering the low price, and the low weight, I think it performs very well for its class. I have accidentally dropped it on a hard floor a couple of times, with no perceptible impact on image quality or function. I think this illustrates the superb design of this lens.

Thursday, 4 October 2012

Multi aspect sensor

After the recent announcement of the Panasonic GH3, much has been said about the missing multi aspect sensor feature, including on this blog, I have to admit. The GH3 was expected to have this feature, since the GH1 and GH2 had it. Also the premium compact cameras Panasonic LX3, LX5 and LX7 have the oversized multi aspect sensors.

But what does the feature actually do?

Put shortly, the multi aspect sensor is an oversized sensor. It covers a larger area than the normal Four Thirds sensor, which is 17.3mm x 13.0mm. The larger sensor allows for a constant diagonal field of view at 4:3, 3:2 and 16:9 aspect ratios, with the latter normally being used for video. It also means that parts of the corners of the sensor are never used in any of the modes.

It can be illustrated with this picture. The green outline shows the normal Four Thirds sensor size, while the oversized sensor is the larger outline. Non-oversized sensor cameras, like the GH3, use the orange box for video recording, while the GH1 and GH2 use the larger red box:


Here is a short video demonstration which illustrates the advantage of the multi aspect sensor. First, it shows the Panasonic GH2, while switching between photo mode (4:3) and video mode (16:9). When switching to video mode, the image grows a bit in horizontal size, while it shrinks a bit vertically.

Later, it shows the same with the Panasonic GF3, which does not have a multi aspect sensor. When switching to video, the top and bottom row disappear, but the horizontal size remains the same. This is because the camera lacks the multi aspect feature:



Ok, so that was the technical stuff. Now again, what are the consequences of losing the multi aspect sensor?

Video

Most zoom lenses start at 14mm. This is not too wide, but it corresponds pretty much the standard wide field of view for kit zoom lenses. For example, APS-C cameras tend to have zoom lenses that start at 18mm, which also correspond to 28mm equivalent field of view.

If you are used to video recording with a GH2 at 14mm, you will be disappointed with the GH3 at 14mm, though. Since it no longer corresponds to 14mm in video mode, but 15mm. This is calculated as 14mm times 5710/5287, with 5710 being the multi aspect sensor diagonal at 16:9, and 5287 being the normal Four Thirds 16:9 sensor diagonal measured in pixels. See this illustration:


Most Micro Four Thirds standard zoom lenses start at 14mm, which is common for the basic kit zooms. But it's not so impressive, and when the non-multi-aspect-sensor of the GH3 makes it into 15mm in video mode, it can be limiting. With a non-multi-aspect-sensor, you may see the need for the Lumix X 12-35mm f/2.8 or the Olympus 12-50mm f/3.5-6.3, which start at a the wider 12mm focal length.

On the other hand, if you don't like to use the wide angle end of the zoom, but care more about tele, then the GH3 is going to be better for you, of course, as the tele effect of the lenses increase slightly in video mode, as compared with the GH2. On the GH3 in video mode, the kit zoom lens corresponds to 45mm in the longest setting, rather than 42mm on the GH2.

Still images

If you like to take photos in the 4:3 aspect ratio, then there is no difference whatsoever between the GH2 and the GH3. The resolution is exactly the same.

However, the GH1 and GH2 had the option of using the 3:2 and 16:9 aspect ratios for photos as well, while still retaining the same diagonal field of view. If you intend to use the images in this format, then you would use the lens image circle more efficiently with these cameras, and the GH2 will give you better resolution to boot. Here is a comparison table:

Photo resolutionGH2GH3
4:34608x3456 (16MP)4608x3456 (16MP)
3:24752x3168 (15MP)4608x3072 (14MP)
16:94976x2800 (14MP)4608x2592 (11MP)

These differences are not that important, surely, but there is in fact a significant difference.

On the other hand, with a camera like the GH3, it is best to stick with the 4:3 format when photographing, and crop the image later, if needed. And that saves time and hassle while photographing, which is not the worst thing you could do.

Tuesday, 2 October 2012

Panasonic interview re GH3

Imaging Resource recently published an interview with some Panasonic staff, mostly engineers. The interview was centered around the technology of the Panasonic GH3, their long awaited new top camera model.

Here are some of my comments and analysis of the interview:

Dynamic range

It is not unusual that new camera models are advertized as having a better dynamic range than previous models. I think that the dynamic range was a shortcoming of both the GH1 and GH2 models, and so I would be very happy to see a significant improvement to the GH3.

The engineers were quoted to say that the saturation level of the sensor photosites was 45000 (in theory), but that the actual number would be lower. They also compared this with the fill capacity of typical compact cameras, which is 8000.

So, what does this mean? When light hits a pixel, electrons are released, and counted by the sensor. The more electrons released and counted, the lighter tone this photosite will report. When 45000 electrons are released, the pixel is saturated, and cannot report any brighter tone. We have then overexposed the pixel.

For simplicity, let's say now that the theoretical number, 45000, is also the actual number of electrons of fill capacity. Keeping in mind that one stop of light difference represents a doubling of the light amount, i.e., the number of electrons, we can count the number of stops of dynamic range this will give:

Tone segmentNumber of electrons
0 (black, underexposed)0
1 1
2 2-3
3 4-7
4 8-15
5 16-31
... ...
15 16384-32767
16 32768-45000
17 (white, overexposed)more than 45000

So we see that there is a theoretical maximum of 16 stops of tones in the dynamic range. Actually, the true number would be 15.5 stops, calculated as log2(45000). However, in practice the number of stops will be much smaller, of course. First of all, the number 45000 was quoted as the theoretical fill capacity. And what the actual fill capacity is was not stated.

Second, noise is going to affect this evaluation. Not only due to technical shortcomings, but also due to the physical nature of the light and electrons.

For example, it is natural to assume that the electron count is Poisson distributed. This means that the standard deviation in the first two groups, 1 and 2, is 1 electron and the square root of 2 electrons, respectively. Hence, categorizing between the groups 0-2 is going to be almost pure chance only, even with some noise reduction techniques. It is perhaps only group 4 that can be categorized correctly with some reasonable significance. And on top of this comes the extra noise added by the sensor equipment.

So how many stops of dynamic range can we expect? It is probably impossible to calculate based on this information only. But if they are correct that 8000 is a figure for point-and-shoot cameras which is comparable to the 45000 figure, then the GH3 can be expected to have 2.5 stops more of dynamic range compared with a point-and-shoot camera, calculated as the difference between log2(45000) and log2(8000). And 2.5 stops improvement over a compact camera does not sound very impressive, honestly.

As a side note, the table above explains why some people use the Expose To The Right (ETTR) technique for improved noise performance. This philosophy states that you should expose the subject as much as possible, without losing details in the light parts. The light parts, to the right in a histogram, correspond to the lower part of the table. In this section of the histogram, there are more electrons behind the measurements, hence, it is reasonable to believe that the significance of the tone segment grouping is higher. And this means less noise.

Using ETTR will of course give you too bright images, but you can open them later in a RAW editing program, and reduce the brightness, while retaining the good noise performace.

ETTR only makes sense when you have less dynamic range in the subject than the sensor can handle, which is perhaps not so often.

Autofocus

Many new Micro Four Thirds camera models have been launched stating that they are the quickest in terms of autofocus, and the GH3 is no exception. In the interview, they refer to a 240 frame-per-second mode. This is easily misunderstood.

It does not mean that there is a video mode which records video quickly, suitable for making slow motion videos. Rather, it refers to the sample rate used for the Contrast Detection Auto Focus (CDAF) system.

The CDAF system relies on sampling the contrast of the subject while changing the focus of the lens. The faster it can sample the contrast, the faster it can reach optimal focus.

The GH2 also improved upon the GH1 by having a maximum of 120 fps sample rate, to be compared with 60 fps for the GH1. As I saw in my AF study, this made the GH2 beat the GH1 in bright light. In dim light, however, the GH2 had to revert to a slower sampling rate, and the focus speed of the two was virtually identical.

With the GH3, I expect that the 240 fps option will only work in fairly bright light, and that the focus speed gain due to this feature will be limited in dim light.

They also stated that using the 240 fps option requires that the lens is also capable of reacting very quickly. At this time, they said, only the Lumix X 12-35mm f/2.8 lens is capable of using this mode, and only after a firmware update to be issued later.

Multi aspect sensor

The interview also confirms the rather sad news that the GH3 will not have a multi aspect sensor. Both the GH1 and GH2 had this feature.

The multi aspect sensor feature means that the sensor is over-sized. At any one time, the whole sensor area is never used. But various parts of the sensor area can be used, to enable 4:3, 3:2 and 16:9 aspect ratios, while retaining the same diagonal field of view. This uses the lens imaging circle more efficiently, especially in video mode.


The engineers do not state any reason for dropping this feature, beyond saying that a sensor was not available. With the GH1 and GH2, I guess that they had separate production runs for these camera sensors, even though they were not volume models. This must have been an expensive sensor to make, relative to the volume of the cameras.

The GH3, costing even more than the predecessors, will probably not become a volume model, either. So I am guessing the reason is economics: It will be too expensive to order special production runs for this single camera model, and they think this extra cost is not worth the benefits of the multi aspect sensor.

I would say this is consistent with the LX5 and LX7 premium compact camera models. The LX5, launched in 2010, had a sensor slightly larger than the common 1/1.7'' size, to accomodate the multi aspect feature. The LX7, launched in 2012, on the other hand, reverts to the usual 1/1.7'' sensor size, probably to make the sensor production cheaper. Producing sensors in odd sizes (as with the GH1, GH2 and LX5) is certainly more expensive than common sizes.

On the other hand, the LX7 does retain the multi aspect sensor feature, but now with a smaller total image circle, so that it still fits into the common 1/1.7'' sensor size.

I have written more about the Multi Aspect Sensor here.