Review: Deshaker Video Stabilizer
Posted: August 22, 2008 at 6:33 am, by Isaac

As I mentioned earlier, VirtualDub has become a big part of our production pipeline, and it’s largely due to the power of Gunnar Thalin’s excellent plugin Deshaker. Deshaker is the best image stabilization tool I’ve ever used. I’ve gotten slightly better results with the 3D tracker SynthEyes, and Avid’s built-in stabilizer is more workflow-integrated, but there’s nothing that can touch it in terms of the combination of power, speed, automatability… and of course, price.

It also has features that I haven’t seen anywhere else, like the ability to detect and repair the distortions that can come from cameras with a rolling shutter. It can handle interlaced video, which a lot of tools can’t, and it has the ability to reconstruct images to prevent edge flicker, which a lot of tools don’t. It even has scene detection built in so you can deshake footage that’s already been edited together (although this is obviously not ideal). Take a look at a very simple scene below. Even minimal stabilization adds a tremendous amount of production value.

As you can see, it’s best at taking out intermittent high-frequency shakes; the unavoidable fast jerks that you get when trying to handle a tiny consumer camera. If I cranked up the smoothness settings (which are adjustable on every axis, including zoom), it would be even more stable, plus I could isolate the area that I want tracked to just the sky, so the rippling ocean waves aren’t confusing the plugin. This is the only problem with Deshaker – it is so configurable that almost any shot can be properly stabilized, but it’s not automatic enough to get the perfect results every time when batch processing a whole drive full of clips.

However, I have managed to come up with base settings that give decent results for the kind of shooting we’ve been doing with the HV20, and I adjust them for the light conditions of each set. Each shot could be tweaked further for a better result, but for the most part I’m happy with what I get by applying the same presets to an entire directory of clips, and each time I adjust the presets to accommodate a new set, I learn better how to use Deshaker to its full potential. Everything is adjustable, from every aspect of the tracking process to the individual controls for correcting the image and compensating for frame edges, an area where it really shines.

This unstabilized shot is pretty smooth for handheld, but the pan could be a lot more consistent and it would be better to keep the rider in the center of the frame. Unfortunately, stabilizing the jerky pan would reveal the edges of the frame. Deshaker has an adaptive zoom feature which will enlarge the image so that the edges aren’t visible, but that will crop out a lot of detail and soften the image. The solution is an intelligent edge compensation, or image reconstruction. By comparing each frame to the frames that come before and after, Deshaker can actually rebuild the image and extend the frame out to where it should be, like this:

click to enlarge

This is a particularly good example of how great Deshaker’s image reconstruction algorithm is, but bear in mind that this is an accurate result since the spectators’ heads are relatively still and the background is static. Still, it’s not completely perfect. If you look closely you can see some smearing near the bottom of the image, and you’ll also notice that the men on the right are looking at where the horse was, and on the left they’re looking at where the horse will be.

But Deshaker can now pan around inside that image without ever revealing edges, and it shouldn’t have to pan far enough that we’re seeing vastly out-of-date frames. Also, there’s a feature than can extend the color from the edge pixels, which looks like this. The only downside is that image reconstruction is pretty time-intensive – to rebuild this panorama out of about 120 extra frames took almost 10 seconds per final frame, but most shots only need a few extra frames to fill the gaps, so our main edit box generally churns through HD video at about 5-6 fps (after a 9-10 fps analysis pass).

So, give it a try. It’s easy to learn, doesn’t cost anything, and shouldn’t add too much extra time to a production schedule. Everything else that we use VirtualDub for, like the deinterlacing and the denoising, could be done just as well in other programs, but Gunnar’s plugin is one-of-a-kind.

Permalink     Print     6 Comments

All Hail VirtualDub
Posted: August 20, 2008 at 10:28 am, by Isaac

Generally, our production pipeline involves editing all the footage down in Premiere to the final cut and then, depending on length and complexity, on-lining the project or project sections in After Effects for all color correcting, stabilizing, deinterlacing, frame rate adjustments, or whatever else is required. This the best way to maintain video quality, since you go straight from the source files to the corrected final in one shot, without multiple generations of adjustments and transcoded video files.

However, sometimes there isn’t time for all that, and sometimes video needs to be converted or processed prior to the editing phase (also, with a good codec, transcoding once or twice isn’t too bad). After Effects is great for this too, obviously, since it’s easy to set up batches of files with preset effects, but sometimes this is overkill, and that’s where VirtualDub comes in.

VirtualDub is a free video utility developed by Lee Avery. It’s not an editor per se, but it can append multiple video files or trim portions out of them. Its main strengths are capturing, processing, and compressing digital video. It can do anything from stripping or adding audio streams to AVI files without recompressing them, or heavy-duty image remastering using third party filters.

Because VirtualDub was originally built to capture and remaster television programming, most of the filters are for removing things like VHS color banding, ghosting from broadcast interference, and network logos, but there are also plenty of great film-related tools for de-flickering, de-jittering, and de-graining damaged telecine transfers. And that’s not all; since VirtualDub is a completely open source project, plenty of developers are contributing new tools all the time.

There are softeners, sharpeners, deblockers, super-resolution resizers, pan-and-scan filters, various color correction tools (limited by the fact that they can’t be animated, but still handy), deinterlacers, subsampling interpolators, vignetting fixers, etc. Perhaps the most powerful is Gunnar Thalin’s Deshaker, which is one of the best video stabilizers I’ve ever used, and merits a blog post all of its own.

So, for our latest project, which didn’t really require a serious on-lining process, we used Virtualdub to process our video before editing. All of our B camera footage (shot with the HV20) was deinterlaced, then denoised, then stabilized, and then had the audio tracks removed in a single generation, resulting in footage that was a much closer match to our A camera (the XL H1).

Once you set up a filter chain in Virtualdub, it’s very easy to apply all those changes to an entire directory of files at a time. More complicated, multi-pass settings may require other tools for the really easy batch processing of clips, but it is still be faster than After Effects for automated processing.

Filmmakers shouldn’t let VirtualDub’s humble origins dissuade them for making it a majpr part of their production pipeline. Whether it will be used for simple codec conversions or cleaning stock footage or reconfiguring entire libraries of video, it has some of the most powerful filters available, the fastest batch interface, and the lowest system requirements of just about any processing tool, and it’s free.

Permalink     Print     No Comments