Canon announced today that they’ve developed a 120MP (that’s a 13,280 x 9,184 pixel image) APS-H format sensor that has a laundry list of features. The sensor can be completely read out in about 10ms, resulting in a 9.5 FPS frame rate, it can do full HD video from the whole sensor (I assume) or one of several 1/16th area sections. Fortunately, or unfortunately as it may be, this is just a prototype and not something Canon has any immediate plans for.

The full press release can be read on Canon’s website. What follows are my thoughts on this sensor and what it could mean for photography down the road.

Small Pixels, Big Picture

From the scant details in the press release, we can make a few estimates about the sensor. For starters, the sensor packs 120 million pixels, resulting in a 13,380 x 9,184 image, in a 29.2mm x 20.2mm area. Simple math tells us, that the pixels are approximately 2-microns across.

2um pixels alone aren’t something new, many of the better performing current generation point and shoots have pixels that size, including Canon’s PowerShot G11 and S90. What is unprecedented, however, is the move to make a sensor with that pixel pitch that big.

Since this is an engineering prototype, it’s clear that part of this is Canon demonstrating they have the technology to manufacture a part this big and the capabilities to drive or use it. Lots of pixels, after all, means lots of data that needs to be moved and processed, doing that at almost 10 FPS isn’t anything to laugh at. The real question, or at least a good hypothetical one, is what that means for photographers in a few years.

Why aim for small pixels, aren’t bigger pixels better?

The balance between pixel size and noise is determined by a couple of factors including area, well capacity and quantum efficiency. However, the only factor in resolution is pixel size.

In general, when there is very little signal (light), larger pixels do “better” since they collect more signal and as a result should have less noise. However, when there is a lot of signal, smaller pixels will resolve more detail. This split can be seen in how Canon and Nikon orient their studio and “low light” cameras, studio cameras has more pixels and a limited ISO top end, low light cameras have fewer pixels and a much higher ISO top end.

What this thinking doesn’t consider is that you can make a smaller pixel appear bigger if you combine it with its neighbors. In short, you can make a small pixel camera behave more like a big pixel camera, but you can’t do things the other way around. This parity can be seen if one compares results of the 5D mark 2 and the Nikon D700. The D700’s bigger pixels perform better in low light, but when you reduce the 5D mark 2’s image to the same size, the results are virtually identical.

Currently, designers try to balance the resolution advantages of small pixels with the noise advantages of big pixels. However, if the sensor can pack enough pixels to break the current 1 imaging pixel equals 1 image pixel condition things could change in interesting ways.

With a whole heck of a lot of pixels, designers could choose to bin pixels all the time. That is, treat each RGBG quad as a single image pixel. It reduces the image resolution at least by a factor of 4, however, noise goes down, and image quality goes up. Moreover, the anti-aliasing (blur) filter can be disposed of without worrying about moiré so the camera should be able to resolve more fine details.

The other alternative is to progressively bin as the signal to noise ratio or ISO increases. This has the advantage of producing the highest possible resolution image in situations where there is enough signal to support that, and producing increasingly less noisy, though lower resolution, images as pixel noise increases. Binning based on ISO isn’t something that’s unheard of either, Phase One’s P+ medium format backs do this to achieve their highest ISOs, so do many P&S cameras.

Bits, FPS, and Bandwidth

The real problem, and what makes this sensor largely infeasible as a commercial product for the foreseeable future, is the amount of bandwidth that this sensor can consume. Assuming it keeps a 14-bit output reading out 120MP sensor 9.5 times a second requires almost 2 GB/s in bandwidth. At that rate, a 32GB flash card would be filled in less than 2 seconds and with only 16 or 17 pictures.

2GB/s isn’t an insurmountable problem; most modern PCs are capable of moving much more than that between their system memory and CPU. Even the DDR memory used in Canon’s current top end camera’s in theory, at least with a wide enough bus, could handle the data. The problem in a camera, however, is twofold. First, is power; simply put moving lots of data requires a lot more power than moving a lot less data, and this sensor would be moving more than 7 times as much data as the fastest production camera Canon currently makes.

Simultaneously, there the problem with storage, at 9.5 FPS, this sensor could fill a 32GB flash card in less than 2 seconds, with only 16 images. Well, at least it could if the buffer was big enough. The fastest flash cards currently available can’t write any faster than about 100MB/s. At that speed, it would take almost 30 seconds to write a single 120MP 14-bit image (assuming no compression).

It’s a Tech Demo, Don’t get Excited Yet

Of course, ultimately this announcement wasn’t intended to demonstrate what Canon expects to put in a camera in a couple of years. This is a demonstration of their capabilities in designing and building bleeding edge imaging sensors, and while it’s not ultimately useful on its own, it is very much necessary to spend the time and money on R&D that results in this kind of thing even if you only use part of what you learned.

There are a few things, however, that concern me about this announcement. For starters, there’s the frame rate. 9.5 FPS is quite fast, and while Canon doesn’t go into detail about how parallel the readouts are, it’s clear that in addition to reading in parallel the sensor is being read out quite quickly as well. The problem is, speed and noise, at least when it comes to analog to digital conversion, don’t play together very well. Conversions can be done quickly, but doing so increases the amount of noise in the conversion.

This problem is most obviously seen in the EOS 1D mark 4. Independent tests have shown the sensor could be capable of >15 stops of dynamic range if the conversion circuitry wasn’t driven to meet the 161MP/s requirements of 10FPS shooting. Now this might not be an issue for this prototype sensor, it may not have even been a goal of this prototype, and of course, Canon may have found a way to build an ADC that performs better at the required speeds.

While this prototype sensor is certainly impressive, photographically speaking FPS and pixels mean less to me than noise, dynamic range, and color accuracy. What would really get me excited is to see Canon announce is a sensor with good by current standards resolution (say ~20MP full frame) and 20 stops of dynamic range. Even better, would be for them to say they’re going to ship it in a camera in the next year.