I’ve been meaning to write in some depth about the challenges of storing digital photos for quite some time, unfortunately, this article isn’t it. However, with rising camera resolutions, and the respective increase in image file sizes, finding room to store, backup and properly manage a large volume of image files is becoming increasingly expensive and difficult.

Beyond practicing shutter discipline and being aggressive with our editing, there is one more aspect that is worth considering, and that’s compression.

Lossy vs. Lossless Compression vs. Photographers

Compression comes in two basic flavors, lossy and lossless. The difference between the two is that lossless compression produces a file that can exactly reproduce the original source material, while a lossy file throws “data” away to produce a more compressible file.

Lossy compression has always had something of a bad wrap in the photographic world, even when it didn’t actually throw away anything worth keeping. Strictly speaking, most photographers, and most of the people who will be most vocal about not using lossy formats don’t really understand what’s going on behind the scenes. That’s not to say we should all start shooting in JPEG now, because we shouldn’t. But we shouldn’t be so quick to write off “lossy” compression when it’s properly modeled, and fits the needs.

I don’t proprot to be an expert in compression, but without going into excruciating detail, it suffices to say, if the “lossy” algorithm is properly modeled to exploit the behavior of light, the distribution of data in an image, or the psychology and physiology of the human viewing the image it’s certainly possible to produce a “lossy” compression algorithm that is visually indistinguishable from a lossless one.

Lossy DNG

So what does lossy DNG actually do. Well I can tell you it’s not a DNG version of Nikon’s Lossy NEF. That is it doesn’t simply use a non-linear mapping function to convert 12 or 14 bits of data into 8 or 9. In fact, it’s considerably more complicated than that, on the flip side, it does produce some incredibly tiny files with no perceptible loss of image quality and still some headroom for post processing.

So how does it work?

According to an Adobe employee’s comments in this thread, it works something like this. The RAW file is demosaiced, to produce a RGB file similar to a TIFF or BMP. The color channels are then remapped using a 1D perceptual mapping algorithm, so that their values are spread across the full range of possible values. Some noise is added—dithering—in specific places as necessary to insure that the now remapped data doesn’t become blocky. The image is then compressed using the same compression algorithm that JPEG files normally uses.

The result is a file that’s often 1/3 or 1/4 the size of the source file, that retains more fidelity than a JPEG would and has sufficient data to allow the image to be processed to a limited degree but more extensively than a JPEG.

In addition to the processing and file size advantages, being stored in a DNG container also means that the file retains the DNG’s checksums to preserve image integrity.

Finally, Adobe did handle the two edge cases, converting a JPEG to a lossy DNG and re-compressing a lossy DNG as a lossy DNG. In both cases, the image data is copied bit-for-bit instead of being decompressed and recompressed.

Size v. IQ

Given that it’s lossy the real question is how much do I really lose versus how much do I really gain. Well lets start with a simple case, in the past when I wanted to save space with a snapshot kind of image, I would export the image as a high quality JPEG, to the same folder and replace the raw with the JPEG.

The RAW from my 10.1MP 1D mark 3, and this was a best case ISO 100 file, was 11.31MB. The JPEG produced by Lightroom that I would have used in the past, is 5.4MB. The lossy-DNG is only 3.83MB. But the real question is how do the images compare?

• RAW
• Lossy DNG
• aRGB JPEG 100
• aRGB JPEG 83

Sample images in RAW, lossy DNG, aRGB JPEG 100, aRGB JPEG 83. (Click image to see full size PNG).

Judging the comparative quality of images is a tad more difficult. To my eye, I can’t detect a difference in detail between a 5MB JPEG 100, a 2MB JPEG 83, a 3.3MB lossy DNG, and the origioal 14-bit RAW. There is a color shift in the two JPEGs due to the color space they’re rendered in (the lossy DNG is still in the same capture color space as the RAW).

Of course these images are unprocessed, and certainly not heavily processed, which isn’t something I intend to be doing with any regularity with how I plan on using lossy DNGs. However, how the lossy-DNGs handle processing is something that I want to look at eventually. More specifically, I’m interested to see how the noise added during the quantization process ultimately effects image noise—though I’m not really expecting it to.

For me at least, lossy DNGs are definitely a useful tool that wasn’t there in previous Lightroom Versions. So far they seem to be smaller and offer more processing headroom than a quality 100 JPEG, and preserve the checksum feature of regular DNGs. Of course the real utility is based in a large part on how much storage you have and how much of your library is less than “serious” business. Granted, it’s not like any appreciable amount of my library is going to end up being converted from lossless RAWs to lossy DNGs, but as it stands on average every 3 files that I compress, I get space for another uncompressed RAW. Which in my opinion is a better storage solution than just throwing the images away.

Instead what I’ve found works best is to shoot shorter exposures of each of the various elements, then composite them together in Photoshop.

Shooting the Display

In my experience, there’s no trick to in shooting a fireworks presentation. In general:

• Use a Tripod
• Use a cable release
• Set the camera to the lowest ISO, 100 or 200
• Use an aperture between f/8 and f/22 depending

The only real trick is balancing the aperture and exposure time, with the brightness of the actual fireworks. This is something that usually takes me the first couple of frames to get right.

Too wide of an aperture and bright fireworks, or most if they’re near buy, will blow out. Additionally, it will make any smoke illuminated by the fireworks brighter; which, in my opinion, is undesirable. Too narrow of an aperture and the dimmer, usually blue, fireworks will not reproduce well.

Usually it takes me a couple of shots to get the exact settings dialed in, which is usually okay when you consider  that the first couple of bursts typically aren’t the meat of the show.

The other major consideration is if you’re trying to expose for a foreground element, like a building. In which case your range of viable exposure values is considerably more limited. Then again, if you composite the shot like I do, you can make a proper exposure for the foreground before or after the display and just composite it in.

Processing

My workflow starts by bringing the images I shot into Lightroom. I’ll make minor exposure tweaks, usually bringing the blacks up to remove sky glow from the horizon. At this point it’s less than ideal to make too many changes in develop, though it’s possible some additional changes will be necessary before the image is complete, many things can be fixed up in Photoshop without serious issues.

After I’ve made my minor corrections and decided on the images I want to composite, it’s time to start the compositing.

1. Using control (command) select the images in the grid that you want to composite.
2. Right click on one of the images and select Edit In→Open as Layers in Photoshop
   
3. Set up the base layer
   1. For an image with a background/foreground,
   1. Move the best exposed background/foreground image to the bottom most layer
   2. For an image with no background/foreground
   1. Make a new layer
   2. Fill it black
   3. Place it as the lower most layer in the document.
4. Set the upper firework layers’ blend modes to screen.

At this point, if smoke wasn’t too bad to start with, or minimizing it isn’t desirable, you’re pretty much done.

However, in my experience there’s usually quite a bit of smoke that needs to be removed, or at least reduced. This is especially true on calm days or for demonstrations that have a large proportion of low-level material.

In this case, I’ve found the easiest way to do that is to adjust the layers blend mode, specifically the black side of the “This layer” slider.

The finished results, with and without the added step to reducing the smoke in the scene.

Who knew?

Apparently, not me, that’s for sure.

That’s not to say that I didn’t know about focus stacking. I’ve played with the product Helicon Focus, and while I didn’t ultimately find it worth the money based on my needs, it does have some nifty features. Nevertheless, that’s not what this is about…

Why Choose focus stacking over stopping down?

Three reasons:

• More depth of field than you can get even when stopped down as far as the lens will go.
• More control over bokeh than you can get by stopping down.
• More depth of field without giving up optimal sharpness.

The first case is the typical case presented for using focus stacking. The you’re stopped down to f/22 but you still don’t have enough depth of field for the shot. Typically, this is combined with macro photography, where depth of field, even at tiny apertures, is incredibly shallow.

The real interesting case, for me, is the second one. This is the case the partially replaces the need for a tilt shift lens, but lets you keep smooth out of focus backgrounds where you want them.

While most people think of depth of field as how much will be in focus, you can also think of it as controlling how fast something will become out of focus as the distance between it and the point of focus increases, and simultaneously controlling how out of focus something will be.

I think the photos below demonstrate this quite nicely. The objective, as far as I’m concerned, is to get a completely in focus flower on as clean of a background as possible.

Shooting at f/16 yields enough depth of field to get the flower completely in focus, but the background becomes extremely busy and distracting.

Conversely, shooting at f/2.8 the background has been cleaned up, but there isn’t enough depth of field to get the whole flower in focus.

Combining multiple images shot at f/2.8 with slightly shifted focus points, however, has the effect of increasing the depth of field on the subject while simultaneously keeping the bokeh quality of a fast aperture shot.

Problems with Focus Stacking

Like everything in photography, focus stacking is a tool that can help in some situations and may not work at all in others. It’s simply not the end-all be-all solution to depth of field problems.

Much like stitched panoramas, the stacked images are brief snaps in time taken over a much longer time than even a single stopped down exposure. As a result, moving subjects can be repeated, broken, or the whole process may not be possible. Ultimately, this means that like stitched panoramas, focus stacking is inherently limited by the subject.

Additionally focus stacking costs some resolution. This is due to the need to align and correct images slightly to account for the shifts in position. In fact, these shifts are completely unavoidable; as shifting, the focus will slightly alter the subject’s magnificent. How much resolution is lost depends on the image, the lens used, and whether the background is conducive to repair or extension in post processing.

For example, the images in this article were shot with a Sigma 150mm f/2.8 macro lens on a Canon EOS-1D Mark 3. After alignment and cropping, the 10.1 MP source images ended up producing a 8.9 MP file.

The final consideration is that if you’re doing this in the field, there’s no way to automate the image collection with current SLRs.

The 30s Tutorial for Lightroom and Photoshop Users

After importing your images into Lightroom, select the images that you want to stack, right click one of them, and select “Open as Layers in Photoshop…”

Once the images have been loaded into Photoshop there are 2 steps. First select all the layers and click Edit -> “Auto-Align Layers…” when the Auto-Align Layers dialog box comes up you make sure that Geometric Distortion is checked. Click OK and let PS do its thing.

After PS has aligned the layers, go to the Edit menu and pick “Auto-Blend Layers…” This will bring up the layer blend dialog, which has all of 3 options, create a panorama, stack images, and a check box for seamless tones and colors. Since this is a stack, we want Stack images. Additionally, it’s probably a good idea to keep the “Seamless Tones and Colors” option checked, as that will correct any minor issues in color variations between the stacked images. Again, click OK, and let Photoshop do its thing.

In my opinion the key elements are clouds, color and foreground, though not necessarily in that order. Beach sunsets pose a special problem since a foreground of interest is usually much harder to come by unless you’re lucky enough to find a good-sized bit of driftwood. Clouds are always problematic, especially in the winter and spring in Florida where the skies are usually clear and completely devoid of clouds.

Last weekend I spent the day on Sanibel Island on Florida’s Gulf coast. The following image is one of many images shot during the not all that impressive sunset.

For all the image lacks, there is a serene quality in its muted pastel colors and evenness of light. However, it lacks pop and given the lack of interesting clouds and a strong foreground it could just as easily end up on the cutting room floor. Fortunately, the lack of pop can be easily fixed with some minor adjustments in post processing.

Contrast

Contrast affects many things in how we perceive images. Local contrast and apparent sharpness are directly related; the higher the local contrast is the more sharp an image appears. On the other hand, the overall contrast of the scene really affects the pop and mood.

Just looking at the image it’s clear that it lacks contrast. It’s even more obvious when looking at the histogram, where the bulk of the image data is in the center and doesn’t actually extend out to the edges.

Naively the lack of contrast could be addressed by adjusting the contrast slider. Doing so will darken the darker areas, which is what we want to happen, however, the contrast slider will also brighten the highlights. The contrast slider can be thought of as controlling the wideness of the histogram, adding contrast stretches the darks towards black and the lights towards white. Reducing the contrast pushes the darks towards white and the lights towards black—making everything grayer.

However, in this image specifically, the real problem with the contrast is the lack of blacks, not so much the highlights. Moreover, if you look at the peak in the reds and oranges in the histogram, which incidentally corresponds with the sky, it’s not necessarily desirable to push those further to the right, brightening them.

While not precisely the same, the blacks and exposure sliders can be thought of as acting like halves of the contrast slider. The Exposure control pulls the histogram towards white, and the black control pulls the histogram towards black.

In this case, increasing the black level will increase the overall contrast. It will also stretch the entire histogram towards the left. The whole histogram shift has a secondary advantage, as it will increase the saturation in the sky and ocean.

Saturation

Saturation sells. Okay it’s considerably more complicated than that, but by and large, most people looking at images will perceive a more saturated image as better, I’m no exception. That said; don’t take this to mean that cranking the saturation to 11 is a desirable thing. In many cases, it’s not.

Ultimately, I think I like the slightly more saturated sky, as it feels that it brings out the pinks and oranges much better. The final rendition with the simple tweaks to blacks and saturation has much more impact than the image did right out of the camera, or even what was visible at the actually being there.

The key idea here, I think, is to understand the differences between how contrast, exposure, and blacks affect the image and when to apply them. Unfortunately, like most things in photograph there’s no hard and fast rule that can be given or followed, instead the one has to use their best judgment when looking at the image and the histogram. In fact, an additional point may be to look at the histogram, as it’s a useful tool when deciding on what adjustments need to be applied.

For a photographer, the minimum viewing distance is limited only by the length of his nose. – Unknown

There are many rules of thumb for determining just how much resolution or pixels per inch (PPI) one needs to print for acceptable quality images. There are rules of thumb, and they are handy however, like most things in photography these things can be accurately quantified with relatively simple math but the numbers can be deceptive.

Some Rules of Thumb

Rules of thumb are usually a good place to start. However, like all rules of thumb in photography they should really be treated as suggestions not absolutes.

When it comes to resolution, the most common rule of thumb goes something like this: If the picture is 8” away, you need a source file with 480 pixels per inch (PPI), if the picture is 12” away you need 300 PPI, if the image is 24” away you need 180PPI.

What the rule doesn’t do is offer any idea as to how far away one would be from an image on average. For that, you need another rule of thumb.

One of the most common that I hear is that you should assume the image is viewed from a distance equal to the length of its diagonal. My personal experience with this is that doing that tends to be conservative for some, if not most, print sizes. For example, I tend to view 8x10s at about 18” not 13”.

The Science

What this ultimately comes down to is a question of the resolving power of the human eye. That has been measured to be 1 arc minute or 1/60^th of a degree, for a person with 20/20 vision.

Using that measurement, we can work out the size of a spot a person with 20/20 vision would be able to see distantly at a given distance. Incidentally, this is not much different from the foundations for depth of field calculations.

A little trigonometry gets us started on the path to the spot size. Where D_spot is the spot size, and D is the distance to the print.

Using the spot size, we can compute the DPI as the reciprocal of the spot size. Putting it all togeather we end up with:

Plugging some numbers into the above formula, we get the following PPIs. For 8 inches, you need 430 PPI; for 12 inches, you need 287 PPI; for 18 inches, you need 191 PPI; and for 24 inches, you need 144 PPI. Not exactly what the rule of thumb listed but not horribly far off either.

As far as math goes, that’s it. The trick is figuring out how to interpret the numbers that come out of the equations.

In Practice

In practice, the hardest thing to estimate is viewing distance, at least unless you have control over it somehow, like by having a desk or couch in front of the wall where the print is hanging. More than anything else, viewing distance has the largest impact on resolution especially when the distance is less than about 3-1/2 feet; dropping from more than 400 PPI to 100PPI in the 35 inches from 8 to 43 inches, but only dropping from 100 PPI to 45 PPI over the next 35 inches.

Yet it’s debatable if you’d ever want to print an image at 45 PPI or even 100 PPI. I know if I was selling a fine art print, I wouldn’t be comfortable with making a 24×36 from an 80PPI source even though the calculations suggest that is all that’s necessary at a viewing distance equal to the image’s diagonal.

Ultimately, it’s important to remember that these formulas provide the absolute lowest resolution you would ever want to print at, not necessarily my recommendations or even recommended resolutions. More resolution, if you have it available, is always going to give more “margin of error” since not everyone will abide by the estimates you might have started with.

I’ve posted a PPI calculator that will do the math for you as well as provide pixel and megapixel estimates for a given print size and viewing distance. In some ways it can be eye opening to see how little resolution is really visible for viewing distances longer than an arm’s length, or what’s actually needed if you went by the diagonal viewing distance rule of thumb.

Like most things in photography, there’s no easy answer. If you can control the viewing distance, there’s a lot of room for PPI to drop especially for large prints. However, doing so compromises their quality when viewed closer than that. How close or how far one has to consider that, though, is ultimately up to the photographer.

In Windows XP, the fix is as simple as restarting the Tablet service though the Services control panel. In fact it was easy enough, and the OS insecure enough, that a simple CMD script could do it as well.

net stop tabletservicewacom
net start tabletservicewacom

It was tedious enough that I use to have that in a cmd script file, but I could call from the hot-key application I used. However, it could also be called from a simple shortcut or though one of the media/shortcut keys on most modern keyboards.

Windows Vista and Windows 7 change that. The new privilege system means that you can’t simply restart a service without some form of privilege escalation, even for admin users. In addition, the separation of privileges has apparently prompted Wacom to introduce a “Wacom_TabletUser” process that appears to interface that user’s settings with the system-level tablet driver.

Unfortunately when the system’s “Wacom_Tablet.exe” (the process behind the TabletServiceWacom service) crashes the user process doesn’t. At this point, simply restarting the tablet service leaves the orphaned user process blocking communicate and the tablet frozen.

The solution is to stop the TabletServiceWacom service, kill the the Wacom_TabletUser.exe process, and restart the tablet service.

The trick is scripting that so it can be done without having to navigate the OS with a keyboard. UAC is what makes this complicated.

Option 1: Use the Elevate Power Toy and a CMD script.

The elevate command isn’t built into Windows. You’ll need to download it from here. Installation is straightforward enough, extract the EXE that’s provided, and install the “ElevateCommand.inf”.

The commands needed are show below. Copy and paste the following into a text file and change the extension to “.cmd” and you’re good to go.

elevate net stop "TabletServicWacom"
taskkill /F /IM "Wacom_tabletuser.exe"
elevate net start "TabletServiceWacom"


Running this will cause a UAC prompt to appear, and then reset the tablet driver.

Option 2: Use AutoHotkey to do this on a hot-key shortcut.

Actually there’s two options here, one is to run the cmd script from option 1 via a hot-key, though that’s not nearly as elegant if you ask me.

The second alternative is to download this compiled AutoHotkey script and execute it as a hot-key action. The script is compiled to require administrator privileges and will generate a UAC prompt.

Of course there’s one more option, you could always reboot when the tablet driver craps out. This was what I was doing until just recently, and it’s horribly annoying if you ask me so I don’t really count it as much of an option.

For example, to reject an image you can hit X, to reject and advance you hit SHIFT+X.

This works with all the keys on the main keyboard but not the numpad.

Camera Setup

A while back, I wrote an article on the way I setup my camera when shooting HDRs and multiple exposure images.

Sizing your brackets is something of a question. Some reputable sources suggest the bracket size (i.e. how many stops between exposures) depends on the camera. That’s probably the ideal solution. That said, I’ve never seen a difference between 1 stop and 2 stops on my 40D, so I don’t use that. When I’m shooting HDRs, I set the bracket size based on how much I need to go under to get the brightest highlight detail I back. In other words, if I think I need to shoot 2 stops under to keep the sky from completely blowing, I use a 2 EV bracket. Remember, I want dark things to be dark in the final image, just not black, so massively overexposing them isn’t helpful to me.

With respect to focus, one thing is important; focus must not change between exposures. Where to focus is another problem, even with a lot of depth of field choosing whether to focusing at infinity, the hyperfocal point and even on a subject are complex enough to warrant an article on their own. Even when I have lots of depth of field, I still tend to focus on the most important subject.

Processing

For me all processing starts with importing into Lightroom. For HDRs, I don’t do any processing to them beyond the import defaults. From there, it’s off to Photoshop.

Photoshop is what I use to do HDRs; I don’t really do enough to justify buying special purpose software for it. If you’re coming from Lightroom via the merge to HDR edit in option, you should be at the Merge to HDR window, if not, you’ll want to load your images and merge them to HDR. I do this in 32-bit mode and then do the HDR tone mapping by switching the image to 16-bit (or 8-bit) color as a separate step. This gives me an opportunity to set an undo point if I need to go back and fix something in the tone-mapping step.

In the case of this image, the tone mapping was done using the Local Adaption method. After that’s done, it’s just a matter of tweaking the image to get the desired look.

For this, I created a selection of the top half of the image, and feathered it by about 100 pixels. This has the effect of creating a graduated split filter, but in mask form, so I can do just about anything I want in the two halves.

To the top-half of the image, I applied a curves adjustment that darkened the mid-tones and darks to bring out some cloud detail. To the bottom-half of the image, I applied a curves layer to slightly brighten the mid-tones and darks and a warming photo filter to make the rocks less blue.

While I may have been able to do these adjustments when doing the HDR tone mapping, I find that this gives me a bit more control. In addition, in my experience, you just have to play with the adjustments; there are no set rules for what to do.

In the right circumstances an HDR can be the best solution to dealing with huge dynamic range scenes especially when there is an irregular border between the brightest and darkest areas rendering traditional split neutral density filters useless.

Method 1

A quick Google search for removing Autoplay handlers in Windows 7 turned up two locations in the registry:

1. HKEY_LOCAL_MACHINE \ Software \ Microsoft \ Windows \ CurrentVersion \ Explorer \ AutoplayHandlers \ Handlers \ <handler>
2. HKEY_LOCAL_MACHINE \ Software \ Microsoft \ Windows \ CurrentVersion \ Explorer \ AutoplayHandlers \ EventHandlers \ <EventName>

These apply to Windows 7 and the specific Autoplay Handler for a 64-bit system is Lightroom3BetaAutoPlayHandler64, I assume for a 32-bit install of Lightroom 3 Beta that the last 2 numbers would be 32 instead but I don’t have any 32-bit installed of LR3 Beta to check with.

Indeed there was an entry in the first location named Lightroom3BetaAutoPlayHandler64 and multiple references to it in the second location under MixedContentOnArrival, ShowPicturesOnArrival, and UnknownContentOnArrival. Deleting these, however, did not remove the now broken entry from the Autoplay window when inserting a media card.

Further searching in the registry turned up 2 more locations where this key existed. The essayist way to find them is to search for Lightroom2BetaAutoPlayHandler64.

One set of keys is in Handlers and EventHandlers under HKEY_CURRENT_USER \ Software \ Microsoft \ Windows \ CurrentVersion \ Explorer \ AutoplayHandlers. The others are likely in either the default user or any other users under HKEY_USERS. Searching is still the best way to find them. Removing the entries from any other place they pop up should clear out the extraneous entry in the Autoplay dialog that pops up when you insert a media card.

I don’t understand why I ended up with both global and user specific instances of the Lightroom 3 Beta Autoplay handler, or why the uninstallation of Lightroom 3 Beta 2 didn’t remove them even though the rest of the application was removed, but there you go.

Method 2

I’ve had this happen to me again when upgrading from Lightroom 3.4 to Lightroom 3.5. In the process I think I’ve identified a slightly better way to deal with rebuilding the registry keys. Remember, before deleting anything in the registry it would be prudent to export the affected keys for a backup.

1. You start with the same procedure above, go into the regedit and visit the keys I noted above
2. Delete all the Lightroom autoplay handlers you can find (whether it for 3 or 3 beta, or anything else).
3. In regedit search for Lightroom3 and delete any autoplay handlers that show up there.
4. Exit Regedit.
5. Start Lightroom
6. Go to Edit → Preferences → General Tab
7. Uncheck “Show import dialog when a memory card is detected.”
8. Click OK
9. Go to Edit → Preferences → General Tab
10. Check “Show import dialog when a memory card is detected.”
11. Click OK

This will prompt Lightroom to rebuild the registry entries for the autoplay handlers, and they should be the correct ones for Lightroom carrying on into the future. Oddly the autoplay handlers it generated for me were Lightroom3BetaAutoplayHanlder not Lightroom3Autoplay handler. However, the autoplay dialog shows the correct information. What remains to be seen is whether this will translate to a correct upgrade to the next point release.