As I mentioned earlier, VirtualDub has become a big part of our production pipeline, and it’s largely due to the power of Gunnar Thalin’s excellent plugin Deshaker. Deshaker is the best image stabilization tool I’ve ever used. I’ve gotten slightly better results with the 3D tracker SynthEyes, and Avid’s built-in stabilizer is more workflow-integrated, but there’s nothing that can touch it in terms of the combination of power, speed, automatability… and of course, price.
It also has features that I haven’t seen anywhere else, like the ability to detect and repair the distortions that can come from cameras with a rolling shutter. It can handle interlaced video, which a lot of tools can’t, and it has the ability to reconstruct images to prevent edge flicker, which a lot of tools don’t. It even has scene detection built in so you can deshake footage that’s already been edited together (although this is obviously not ideal). Take a look at a very simple scene below. Even minimal stabilization adds a tremendous amount of production value.
As you can see, it’s best at taking out intermittent high-frequency shakes; the unavoidable fast jerks that you get when trying to handle a tiny consumer camera. If I cranked up the smoothness settings (which are adjustable on every axis, including zoom), it would be even more stable, plus I could isolate the area that I want tracked to just the sky, so the rippling ocean waves aren’t confusing the plugin. This is the only problem with Deshaker – it is so configurable that almost any shot can be properly stabilized, but it’s not automatic enough to get the perfect results every time when batch processing a whole drive full of clips.
However, I have managed to come up with base settings that give decent results for the kind of shooting we’ve been doing with the HV20, and I adjust them for the light conditions of each set. Each shot could be tweaked further for a better result, but for the most part I’m happy with what I get by applying the same presets to an entire directory of clips, and each time I adjust the presets to accommodate a new set, I learn better how to use Deshaker to its full potential. Everything is adjustable, from every aspect of the tracking process to the individual controls for correcting the image and compensating for frame edges, an area where it really shines.
This unstabilized shot is pretty smooth for handheld, but the pan could be a lot more consistent and it would be better to keep the rider in the center of the frame. Unfortunately, stabilizing the jerky pan would reveal the edges of the frame. Deshaker has an adaptive zoom feature which will enlarge the image so that the edges aren’t visible, but that will crop out a lot of detail and soften the image. The solution is an intelligent edge compensation, or image reconstruction. By comparing each frame to the frames that come before and after, Deshaker can actually rebuild the image and extend the frame out to where it should be, like this:
click to enlarge
This is a particularly good example of how great Deshaker’s image reconstruction algorithm is, but bear in mind that this is an accurate result since the spectators’ heads are relatively still and the background is static. Still, it’s not completely perfect. If you look closely you can see some smearing near the bottom of the image, and you’ll also notice that the men on the right are looking at where the horse was, and on the left they’re looking at where the horse will be.
But Deshaker can now pan around inside that image without ever revealing edges, and it shouldn’t have to pan far enough that we’re seeing vastly out-of-date frames. Also, there’s a feature than can extend the color from the edge pixels, which looks like this. The only downside is that image reconstruction is pretty time-intensive – to rebuild this panorama out of about 120 extra frames took almost 10 seconds per final frame, but most shots only need a few extra frames to fill the gaps, so our main edit box generally churns through HD video at about 5-6 fps (after a 9-10 fps analysis pass).
So, give it a try. It’s easy to learn, doesn’t cost anything, and shouldn’t add too much extra time to a production schedule. Everything else that we use VirtualDub for, like the deinterlacing and the denoising, could be done just as well in other programs, but Gunnar’s plugin is one-of-a-kind.
That’s amazing! I checked out the video before I read the rest of the post, so it was pretty exciting to see DeShaker in action and then realize it’s a free plugin for a program I already use.
I’ve been using Deshaker for well over a year now and have managed to get stunning results for some sequences – especially one shot from behind of my wife skiing, which went from unwatchable to hypnotic. However, I have just bought my first HD camera (Xacti HD1000) and find I get some odd results occasionally – a kind of wobbly shimmering effect in the stabilized video – should some of the settings be changed for HD ?
Well, the Xacti has a rolling shutter, so Deshaker can fix some of the wobbles if you tell it that the footage was shot with a rolling shutter, but for the most part all you can do is make sure the shutter speed is set very high on the camera to minimize the amount of the wobbling. Pob lwc!
It is my understanding that high shutter speeds in fact exacerbate rolling shutter artifacts, and that you should use a 180° or 360° shutter, so that motion blur hides some of the wobble/skew.
Unfortunately, motion blur only hides slight wobble of moving objects; if the camera is moving and the entire frame is warped, all motion blur will do is soften the edges of anything that’s warped. If you are trying to stabilize the footage, motion blur will confuse the tracker, and even if it does work you’ll get motion blur happeneing even on frames where the movement has been stabilized.
I would like to get a result like your large image at the bottom. A very large frame that accommodates the widest and most vertically expansive view available cumulatively from a sequence of frames. I could then set VirtualDub to crop the borders as I saw fit. I like Deshaker plugin a lot but I often prefer eliminating the shake by converting to a fixed POV as opposed to smoothing the shake by going to a smooth pan.
Can you tell me what settings you used to get the large image?