The final touch to the project, which pulls everything together and does more to communicate the mood of the scene is the musical score. The music needs to match the scene, both visually and emotionally, and also fit in with the sound effects and dialog. Our score was (unfortunately) created without any real instruments whatsoever, and composed, performed, and mixed by three people on a single computer running Cubase.
Rather than recording the analog audio feed from our electric piano, the computer recorded the MIDI signals created by each keypress. This enabled us to go back into a performance and adjust each note’s position, length, velocity, and expression, or move them up and down across the scale, or change their tempo. This is particularly important for adjusting each track to better match the instrument that is playing it.
click to enlarge
Each of these performances sits in a track, and can also be manipulated on the timeline, as well as copied and pasted, extended, shortened, and affected by filters without destroying the MIDI data. At any time, any note can still be changed. We now have all the parts for the orchestra laid out and assigned, and it’s time to attach instrument sounds to them.
click to enlarge
The sound bank that we use is called the Garritan Personal Orchestra which contains audio samples of all types of instruments playing all of their respective notes. The software replaces our keyboard’s A# with a violin or piccolo’s A#, and adds several special controls to better simulate the individual characteristics of actual instruments.
The last step, after all the parts have been recorded and adjusted, is to mix all the tracks together. This requires special attention to volume, EQ, and stereo placement, but the finishing touch is to add just the right amount of reverb to the performances that they sound like they were recorded in an actual studio. The Garritan package includes an excellent ambiance plugin with superb control.
And here’s the final score, with the final visuals. This mix contains only the music, with no sound effects or dialog. Watch carefully how the music changes to fit the mood of the scenes, and how it matches the action.
So far, I’ve only talked about creating the images for our film, but sound and music are extremely important, and can be quite time consuming. One of the reasons that the animatic was so important for our tight-deadline project was that it gave us something to write music to before any animation was done. As as I’d finished the animation edit, we could start adding the sound effects. I’ve probably posted enoughcharts over the past month, but here’s one more:
That is the timeline for the whole project. Ideally, the scripting period should have been longer, but you can see that by creating an animatic at the beginning of the project, the composers were able to use the entire month. To a lesser extent, so was the sound effects department. The storyboards showed the action pretty well, and from that we could see what sounds we would need; swords clashing, cannons firing, and creaking ships. Most of these sounds we found on the internet, but many we recorded ourselves. All of them needed to be adjusted and sweetened so that they could work together.
We used the many powerful filters of Adobe Audition to turn fuzzy, static-filled, mono internet clips into cleaner, more expansive stereo tracks. Our own recorded sound effects needed tweaking as well, and we added ambience, subtle reverb, and stereo offsets to simulate a more “open sea” type of environment. We built this library out until the first edit was finished, and then were able to begin matching audio cues to visual actions and create a more realistic mix. This is extremely important for animated projects, since there isn’t any live sound captured on set.
As of today, www.outside-hollywood.com is twelve months old. As of this post, there are 64 entries in the archives, which means that I’ve somehow managed to average more than one post per week… even if they haven’t always turned up regularly.
I’d like to thank all the readers who frequent this site, and everyone who has emailed in questions, comments, and news reports. I have greatly appreciated your encouragement and participation, and am looking forward to another year of posting about projects and developments in film and video.
The final stage of creating the visuals is compositing. That involves combining the backgrounds and foreground animation together, and adding any special effects and color correction. This is easier with 3D animation than live action because animated elements can be rendered separate from a background, and combined according to depth. This is helpful because atmospheric effects are important.
In addition to backgrounds and character animation, I created some moving mist to make up the smoke of battle, some smoky wisps for closer shots, and the smoke from the cannons. We also need several shots of debris, for when the cannonfire hits the sides of the ships, and several different splashes for cannonballs that miss and men overboard. To add life to the island, we also needed seagulls.
Above is Shot 32. You can look at the rendered image, which is what Lightwave exports. It also exports a depth map, what Lightwave calls the z-buffer. The z-buffer displays how far away objects are from the camera. You can see that the captain, the object closest to the camera, is black, the pirate ship is grey, and the sky is white. Then we render out the background image, which fits together with the foreground.
Using the depth map, I can add my mist layer in so that it goes behind the captain and gets thicker as it gets further away. The depth map also lets me add a blur that simulates depth of field. The further a pixel is from the camera, the more blur is applied to it. Then I add my detail smoke over the top to tie everything together, and it’s time for the final color correction pass. Here’s one more shot:
Shot 28 makes a lot use of the z-buffer since it needs to simulate an enclosed space full of smoke. I could have rendered it with actual 3D volumetric smoke, but that would have taken too long. Simple 2D smoke applied in Adobe After Effects works almost as well. Where the z-buffer is black, the smoke is transparent, and where it is white, it is more opaque. That gives us our first smoke pass. I can also use the z-buffer to place an animated element, in this case cannon smoke, behind a specific object in the frame.
I then changed the color of the smoke to better simulate the light shining in through to gun ports, and added a red glow around to torches to look like they were illuminating the smoke layer. All this shot needs after that is the depth of field blur and some color correction, and it is finished. With only fifty shots, it didn’t take long to finish compositing the entire film, and all the image work was done. Of course, while I was working on that, my siblings were hard at work on the sound effects and music, which I’ll talk about next.
Well, once the animation of the characters, ships, and foreground images got taken care of, it was time to create the backgrounds. First off, we needed an ocean. With square plastic blocks there are only a few ways to make an ocean, but in 3D I had the advantage of being able to control infinite plastic blocks.
This clip is my first 3D water test. I have 3D blocks moving up and down, controlled by an animated displacement map. If they get high enough, the top of the block turns white, so the peaks of my waves have white crests. Once I figured out how to do this, it was simpler than it looks. There were some downsides, though. While I can in theory control infinite blocks, my tight deadline meant that I probably wouldn’t have the time to render more than about 40,000 per scene.
That’s a lot of blocks. Fortunately, the storyboard (coming in handy once again) showed me exactly how many scenes needed close-up shots of the 3D water. There were actually not that many. In most shots, the ocean is much further away and I could cheat it with some simple 2D water. For that I created these two textures, which tile and loop, and mapped them to a single, flat, ocean polygon. The grey image is the bump map, and the blue one is color. Here is what it looks like animated. Not as nice as the 3D water, but fine for long shots.
Of course, we also need a sky. On the open sea the sky is a large part of the environment, and it would also dictate the lighting for the shots. As much as possible, I wanted to visual style of this short to mimic the great swash-buckling illustrations by Howard Pyle and N. C. Wyeth, who always had strong warm light, long cool shadows, and high puffy clouds. Of course, they also had a lot more swash and buckle than I can squeeze out of rigid little plastic men, but at least I can do puffy clouds.
The picture above is a quick color sketch I did in Corel Painter to experiment with a few things. When I was happy with it, I painted a panoramic sky image that could be mapped onto a giant cylinder that would be the backdrop for my ocean. It took some experimenting before I got the sunset colors to work in 360 degrees, but eventually I got it right. A real downside with this project was that I only got to do one matte painting, but at least it was a big one that was in every single shot.
click to view detail
And now for the island. This was the easiest model to build, thanks to the efforts of Rob and Alexx, coders of a Lightwave plugin called The Legoizer. This plugin can take a regular polygonal model and convert it into cubes very quickly. It took me about five minutes to build a very rough Island shape, and then ten minutes for Lightwave to convert it into a brick sculpture.
Cover that with a few hundred swaying plastic palm trees, add a cascading waterfall, and it looks pretty good. And then all the background elements are done; close-up water, distant water, sky and island. Now it’s just a simple matter of loading the camera animation from the character scenes into my background scenes so that the motion matches, and rendering them out. Then the foreground and background elements can be composited together, and the visuals for the short film are finished.
With all the foreground elements created, it was time to begin animation. Now I’m not going to go into too much detail of the nuts and bolts of 3D animation since there aremanywonderfulanimationresources on the internet already, and I’m trying to teach more about the production process we used than the nitty gritty technical stuff.
click to enlarge
However, it’s similar to stop motion and even live action in that you have a 3-dimensional stage to lay out your scene in. I can load the ship into Lightwave, put the sailors on the deck, and position lights and camera just like real life. However, there is a bit more control in the computer, since I can scrub back and forth through my frames as I animate to check my progress – something that’s tricky to do with claymation.
Thanks to the storyboard and animatic, I knew exactly what needed to be in each shot, and exactly how many frames long it should be, so for shot 12, I have the Cap’n, Bosun, and Cabin boy standing on deck with a telescope for 77 frames. The Bosun is speaking, and so I need to make sure that his head and body movements match the his animation and dialog. Animating such simple characters is harder than I thought, since they only have seven joints, and each of these can only turn in one axis.
I also need to animate the camera, since I don’t want any static camera positions. The ship itself needs to move as well, and so everything that is not stationary needs keyframes in order to move. Watch this clip to see how that camera is animated. At frame 0 it is in one place, and at frame 77 it has moved slightly to the left. You can also see the ship rocking back and forth, which took three keyframes. This clip is from the camera’s view and shows how the characters are moving.
With such simple animation, it didn’t take long to animate and render the characters and foreground elements for every one of my 50 shots. I loaded those into Adobe After Effects and put them on top of the animatic. This was the second edit, and we needed it to add sound effects to. We had begun to write the music to the animatic, but specific sound effects needed to be matched to the animation.
There are a few shots missing, and a glitch or two here and there, but the editing works, and the animation seems to be working. Next step: Backgrounds.
