PostProduction

PostProduction for Visual Effects is covered in this podcast.

Here's the timeline:
2:10 Client turnovers
6:24 Effects editorial
8:41 Scanning
12:08 Effects turnovers
15:21 Artists - Matchmover, Animator, TD, Compositor, Rotoscoper, Painter, CG Modeler
19:10 Work day
22:17 Dailies
24:34 Client reviews
26;33 Temp screenings
28:09 Balancing work and schedule
29:36 Filmmaker

Transcript
Today I'll be discussing Post-Production for Visual Effects. This will be an overview with later podcasts to cover each specific area in more detail.

As with all of these podcasts I'm providing information based on my experience. Every effects house and film will be slightly different so use this as a starting point.

Effects Pre-Production
By the time the actual work is to start on the visual effects most of the CG models that will be animated will have been built, chained and painted. The various pipelines should already be set up. A pipeline is the workflow and the support for that workflow with specific tools. A shot may be passed assembly line style from one person to another. It may also be worked on by more than one person at the same time. The images and the data is passed from one software package to another. Animation may have to be done in one piece of software and then exported to be lit in another piece of software and then rendered using yet a different software. Creatures with fur may have to go through a different render pipeline than non-furred creatures. The output images are then composited in yet another piece of software. The idea is to make this transfer of data as easy and seamless as possible. This may involve writing special databases and scripts to help move the data and keep track of it. If this is an established effects company most of this may already be in place but there always seems to be some changes required. The CG supervisor is usually working on this pipeline while the film is being shot.

If there is an animated CG creature most of the voice recordings are also done by this point.

Client Turnovers
The first step of the postproduction process is the client turnover.
When a sequence has been edited the director, editor and producer turn over the visual effects shots to the effects company. The effects company team includes the effects supervisor, animation supervisor, effects producer and typically an effects coordinator. Everyone gathers around the editing machine, usually an Avid, while the director explains what's required in each shot. A 10-minute sequence in the film can easily take 2 hours to review and discuss due to the complexities of the shots and action.

Even though the effects shots have been storyboarded in pre-production there are always additional details now that the background plates have been shot. There are always some changes as well in the shooting and editing process. A single planned shot may have been broken into to 2 shots or two shots may have been combined into one big shot. In some cases shots have been dropped for pacing reasons and a few may have been added to help the story telling.

Depending on the shot it may be difficult to make the edits, especially for a shot where most of the scene will be added later. If a shot will be all virtual as a standalone miniature shot or a CG virtual shot then they will be cutting in a storyboard or an animatic to use as temp placeholder for the real shot to come later. They may have cut in the reference take so they can see how the action flows but they will have selected their real take based on the actors performance. In other cases they want to use a reference take and you'll need to remove the other actor or monster stick.

One thing I try to do when possible is to create mockups before the edit. I did this quite a bit on Van Helsing. A QuickTime of a shot would be copied to my iPod while we were shooting and then I'd tried to do a basic mockup at night back at the hotel and dump it back out. This tended to be mainly 2D mockups but was enough for them to check the take and get a sense for the action. It also allowed me to see how well something was working and to determine if we may have to make adjustments in the future.

The editor or his assistant may do simple mockups on their editing system. A shot may be split or a simple bluescreen key may be done. If animatics are generated it's sometimes useful to do a render of just the foreground or background. If it's a background this can be comped behind a bluescreen and if it's a foreground piece this can be comped over the real background. All of this is done to provide a better sense for which take and timings work best. In some cases they may pick alternate takes if there's a problem later with animation timing.

Since each shot is covered in great detail with much finger pointing at the video screen and pantomiming by the director we usually video tape these sessions in addition to taking notes. It's important during these sessions to get as much information and detail as possible since any miscues could be costly in time and money.

Since the number of visual effects shots these days is going up and the amount of postproduction time is getting shorter it's very common to do these turnovers while the film is still being shot. This is usually spelled out in the contract since the film has a fixed release date. The delivery date for all the shots to be completed by the effects company is called the finals date and can be 2 weeks to 2 months before the release of the film. The editor will start making an assembly from the dailies and the director will review this after a day of shooting or on the weekends. Once the director is happy with a full sequence the turnover will happen at the end of a shooting day or on a weekend as well.


Effects Editorial
Once the client turnover is done the show's editorial department usually ships a copy of the edited sequence. This may be on a tape but ideally is in Avid project or some means for us to assemble our own edited version. It's important to have not just the visual effects shots but the surrounding live action shots as well. This will allow checking the in-progress effects shots in context so we can check the timing, actions and the match.

The effects editor creates a breakdown, line up or cross over sheet. This will list the sections of film that need to be scanned and any additional elements required. The standard references of the lighting sphere and other items are some of the additional elements. The effects or animation supervisor may want to scan additional reference footage.

The editor has the option of extending a live action shot at any point because there's usually plenty of additional footage on the head and tail of the selected cut. With visual effects though you end up with a shot just as long as specified. To provide a little latitude for offsetting or increasing the shot length an effects shot usually includes additional frames at the head and tail of the shot. These extra frames are known as the handle. This could be 2 frames at each end or up to 16 frames at each end.

The more frames you have, the more flexibility in the edit to make an adjustment but these added frames add an additional cost to each and every shot. 16 frames of handle would mean a total of 32 frames of extra work that likely won't be used. That's a second and a half. That's a lot since the typical effects shot is 5 to 8 seconds in length. A 4-frame handle is probably most typical. The average shot length and required handles is spelled out in any effects contract. The size of the handle can be discussed on a shot by shot basis since in some cases the extra frames may require a lot of paint out or other work.

Scanning
Once the information is together and the original negative is available it's scanned into the computer. Some effects houses do this themselves and others send this out to a service to be done. Normally I try to obtain timing clips from the DP (director of photography) so we can do some basic color balance when scanning. A timing clip is a few frames of a shot or shots that the DP has worked with the lab to create and is an example of the color and brightness for the final shots. This shows the DP's vision of what the end result should look like. Some places wait until the end of the process to do a full color balance.

Scans are usually specified by their resolution, bit depth and format.
There are a number of formats and possible aspect ratios when shooting: 1:85, 2:35 anamorphic, Super 35, etc. It's normal to try to scan a bit more of the frame if it exists such as in 1:85 on the top and bottom (and one side if it's academy format) This allows a bit more room to move the clips.

Typical resolution for feature films is approx 2k or 2048 pixels across. Some films are being done at 4k or 4096 pixels across but that requires 4 times the amount of data and processing. The bit depth refers to how many levels of lumance for each color. 8 bits is 256 levels and provides over 16 million colors for red, green and blue. 10 bits or higher is now more common since it provides finer increments of color. It's also important that these scans be logarithmic to provide the best use of those bits. Some people think films have to be more than 2k and have deep color but the fact is quite a few well known films were done with 8-bit and less than 2k resolution. The file formats are usually in a Cineon format but there are many others, especially for in house scanning.

High-resolution images take up a huge amount of storage space. Even a simple 8-bit, 2k image will take 10 megabytes per frame. Since film runs at 24 frames per second and since there are usually multiple film elements per shot that starts adding up quickly. When possible images are sometimes compressed in a lossless compression. This will reduce the amount of storage required and reduce the amount of data to be copied between computers. These scanned shots are dumped to a high-density data tape or across a network.
Even though effects houses may have terabytes worth of storage shots are usually brought online (loaded on to hard drives) when the shot is about to start and they're normally removed as soon as the shot is finaled.

After a shot is scanned it typically goes through dirt removal. A team of people sit at computer workstations looking for dirt and scratches that have to be painted out, typically by hand, before the shots are ready for production.

Effects Turnovers
When a few shots have been scanned and are ready to be worked on there are turnovers for the actual people working on the shots. A shot usually goes through a number of departments, some of which work simultaneously on the shot and other departments that may have to wait until the previous department finishes. A single turnover may happen with all the key people who will be working on the shot or it may be broken into multiple turnovers for each department.

The shot will be displayed on a computer or video monitor and the animation or effects supervisor will step through the key issues much as the director did. It's important for all those in the turnover meetings to have a pen and paper for notes. This would seem obvious but a lot people try to remember every detail only to find out they forgot something a week later.

In some cases a section of video of the director turnover may be shown just to make sure there's no miscommunication.

Every shot and every element will have an ID of some type. Possibly the first 3 characters of a sequence name followed by the shot number and then possibly a letter or letters signifying the element itself. All this is written up by editorial and provided to the people working on the shot. Larger houses also have a database with info and access related to each shot. This is critical since one of the bigger tasks in doing visual effects is keeping track of all the pieces.

A single shot can take one person a few hours or can take a number of people in 6 departments a few months to complete. A few linear weeks are probably typical with it being touched by 5 or 6 people. Most people are assigned 2 or 3 shots so they can continue to work even while waiting for feedback or additional elements on one of their shots. Each person is assigned a basic schedule and deadline for their portion of the work.

For a large sequence in the film there may be a sequence lead for technical directing and a lead for animation. The lead TD will establish the basic lighting for the sequence based on the live action and will act as mentor to the other td's and compositors working on the shots in that sequence. They'll also be involved with specific R&D required for a sequence. The animation lead will try to establish the basic character action for the sequence and help to mentor the other animators as required.

A show could have a dozen people but more than likely will have 50 to 300 people working on it. A large effects house may have 2 to 5 shows going on at one time. It's important that schedules and plans are laid out and that sequence leaders be able to manage their team. These crews are made up of true artists that are at the core of the effects work you see. They have a balance of artistry and craftsmanship.

Some of these artists include:

MatchMover
A matchmover is provided with information about what's critical in the shot and what's not critical. In some cases a rough matchmove is requested first so it can be passed on to the animator to get started on. The match mover is provided with measurements, photos and additional information from the set for a specific shot.
Once the matchmove is done a grid or spheres are rendered as an overlay with the background to check the work.


Animator
The animator on the shot will take the matchmove and start roughing out the animation. Ideally the sequence lead animator may block out or rough in the animation over a sequence of shots. A low res single color plastic render will be created for each shot. This is overlayed with the background plates and cut into the sequence. Once this is done it is sent to the editor and director for review.

Technical Director
The Technical Director or TD is the person that does the lighting and rendering of the CG creature or model. The also deal with any 3D technical issues on a shot. On the credits they're usually listed as Digital Artists since the Directors Guild controls anyone who get's a director credit of any type.
The TD can start the lighting as soon as he or she receives the first animation from the animator.

Compositor
The compositor is the person who handles the 2D work including compositing or combining all the different pieces of film or elements, including computer generated elements. They start their work prepping the different images, requesting paint and roto work as need be and then take any rendered CG images from the TD. The compositor adjusts the edges and color balance of each element to make sure they blend seamlessly.

Rotoscoper
The roto person creates hand drawn mattes as needed. If a shot has a CG creature run behind a table someone has to draw the outline of the table and create a high contrast matte that will be used to place the image of the table over the image of the creature. In the case of something like Dragonheart the actor in the foreground has to be rotoscoped while moving so we can place the dragon behind them.

Painter
The painter goes through at the start and paints out any special rigs or other items that shouldn't be in the shot. They try to use the clean plate if it exists.
When a shot is finished it may require some slight touchup on a few frames where the render or composite wasn't quite right. There's much more handwork involved in doing visual effects shots than most people realize.

CG Modeler
Actually modeling usually involves a number of people. People who do the model, others who paint the texture maps and others to build the shapes, skin and skeletons for animation purposes. Hopefully this has been done but additional model shapes may be required, especially for facial animation, while the shots are being worked on.

Some of these tasks such as TD and compositor may be combined into one artist. It's possible for a single person to do most of the shot themselves but this may not always be the most efficient.

Work Day
Once the shot has been turned over each individual goes back to their workstation to work on the shot. As they progress they let other's know there is an update. In many cases they may check their work into a database. This is similar to software development where files can be checked in and out.

When an animator is going to do additional work it's important for them to have the latest version of the matchmove. Likewise it's important for the TD to have the latest animation to light and render.

In the old days you had to finish any work by the end of the day (such as opticals or motion control) to get the film into the lab. In the digital age you try to get everything done before the end of the day as well so the computers can render over night.

The TD may be checking single frame renders during the day as he or she makes adjustments to the lighting. The animator will be looking at wire frames or plastic renders to check animation and the compositor is working on a few key frames to make adjustments.

One of the things that these people also do during the day is to calculate the amount of computing power required. If you have a 300 frame shot and each frame will be taking 1 machine hour to render that needs to be taken into account. A large effects house will have a render farm made up of the individual machines as well as a cluster of specialized rendering computers. A single shot can be spread over multiple computers over the network. A frame could be sub-divided in to multiple sections to be rendered or each frame can be passed to a different computer. There may be dozens of shots being rendered overnight.

Everyone that needs rendering usually logs their request into a database and the CG supervisor will have to review the requests versus amount of computing available and determine how the work will be done. Some of the overnight work is done at lower resolution so animation and basic rendering can be checked without using up too much computing power. If it's a final render then it will have to be a full resolution with motion blur and fur rendering and anything else. These types of renders may take a few days and could be scheduled to run on the weekends.

Additional elements
In preproduction and during filming a list is made of other specific elements that need to be filmed. These could be smoke from chimneys, fire for a building or water splashes that have to match a creature jumping into the water. During the compositing process a few more elements may be found to be necessary to finish the look of the shot. These will have to be filmed and scanned before the compositor can complete the shot. Ideally the effects company has past footage and generic elements for at least some of this.

Dailies
On every show the dailies are handled a bit different. The number of shots and the length of time to review and comment can make it very time consuming.

In the morning the effects supervisor and animation supervisor may meet and quickly review the shots together in private. If the animation supervisor has any comments about rendering, compositing and lighting issues he or she would make it here and visa versa.

Dailies can be held in a large theater with high hres video projection with the entire crew or could be in a smaller room with just the sequence people or could just be a one on one review of the work with the supervisor and the artist. All of this is a balancing act between making sure the artists get feedback on their own work as well as seeing the related work and at the same time avoiding tying them up for all morning sitting and watching dailies.

Animation and TD/Compositing dailies are usually separate. Hopefully the effects editor has already cut the dailies into the edit so it's possible to review the shot in context.

I usually try to review the shots on my machine before dailies so I have some notes and can quickly cover a lot of issues. Some people want to wait for a review until they've fine-tuned their shots but this may mean a week until any progress is seen. If for some reason there's a mistake or miscommunication while the artist is polishing the shot then all that work will have to be re-done.

As with the turnovers all artists should bring a pen and pad of paper to take careful notes during dailies. When possible the TD and compositor should explain what they've done and what they still have yet to do. Because of time constraints most of the focus will be on work that still has to be done or improved rather than praising the artist for sections that have been completed.
During the course of the day the supervisors may check in with the artist or may get email from an artist requesting a review or a question.

Usually there is a coordinator assigned to each supervisor. They help to follow-up with the artists and track down information as needed.

Reviews
Because of the complexities and time of doing visual effects it's critical to make sure to frequently review the shots in context. Some people try to wait until they're finished with the shot but there's a strong likelyhood the director will want to change something and they may not be able to make a critical decision until they see the surrounding shots. Each director has a different ability to visualize the shots. Some have difficulty in viewing plastic renders and making animation decisions based on these. This is a learning process for working with each director.

Once or twice a week the supervisor will determine which shots to show the client. These may be shots in progress where a confirmation is requested, or shots where a question has come up or a shot, which is considered a final by the supervisor. Usually there's an animation final on a shot and then later a TD/compositing final. The supervisor will go to the editing room to show the director and editor the new work. If the director is in another city a DVD may be sent or a live video link is used. The latter is much preferred. The video link allows the artists to be present and hear the comments right from the director.

It's still possible for changes to be made at any point. If changes are outside the scope of the original budget for the shot then a change order may be called for where a new or updated budget is submitted for approval.

Once the shot has finaled on video by the director it needs to be rendered full res if it hasn't already been and then be filmed out. The director views this film projected on a full size screen to determine if it's a final final. When a shot is finaled a few frames are saved at full res for PR purposes and the final shot and key elements are backed up and removed from the server.

Temp screenings
The director probably has to show the film to the studio a few times during the course of postproduction. Of course the director will want as many of the shots done or in at least some form of completion even it's with temporary shots. This is better than showing an actor sword fighting with a ping pong ball on a stick or actors on a bluescreen pointing to a non-existent object.

These temps are also used to help the composer and sound effects people since this work will be ongoing even while the visual effects work is still in progress.

As the film gets closer to release it becomes important for the studio to do test screenings. Unfortunately test audiences don't have too much tolerance for plastic renders of dinosaurs and other temporary shots. If the test screening is a wild success the studio may authorize more effects shots. If a test screening does poorly they may want to rethink and redesign some of the effects shots.

Trailers
The director and marketing people select shots that they think would be good in a trailer. A trailer being the ad for the film shown in theaters and on TV. Unfortunately they sometimes select shots that aren't even scheduled to start for several months. As a result trailer shots can be rushed and the version for the final film will be noticeably better. It's best not to judge the effects work too much based on the trailer.

Balancing
For those of us working in visual effects it becomes very easy to fixate on every detail in a shot. This is especially true since we can change everything in a shot. We loop the footage over and over on a computer monitor or large screen at full magnification. We know where to look so it's easy to spot any imperfections. We have to remind ourselves if we don't spot something in 2 or 3 times through then the shot's done.

The same problem holds true with the directors as well. If they film a jet fighter flying by for real then they will select a take and move on. If it's a motion control model or a CG model then the director usually gets caught up in making the jet roll a specific amount at a specific frame even if the original take matched the live action version. Just because there is infinite control doesn't mean there should be an infinite number of takes.

All this is important because in order to generate hundreds of effects shots in a given amount of time you have to keep moving ahead. If you spend half the postproduction time on a fourth of the shots then the other shots will be rushed and the quality level will drop. It's important to develop a pace that can be maintained during the production or you'll end up working long weeks and hours the closer you get to your finals deadline.

Filmmaker
If you're doing your own film and even all of visual effects there's still some principles to apply.

Before you begin the full postproduction process make sure you have your pipeline working. It's better to work out the kinks before you have 30 shots and a looming deadline. Run a short test shot from your editing software to your 3D software to your compositing software and then back to the editor. Use your manual and online help to make sure you understand the different steps involved.

If this is video, export from your editing software with some additional handles. Keep a log book of the shots including timecode and other information. Avoid re-render and recompressing when possible and use lossless compression such as the QT animation codec if you plan to manipulate the images through more than one software system.

Find out the required field order. Most video footage, especially DV is set for lower field. If this is 24p video make sure you understand how to remove and how to add pulldown. Some effects composting software can do this but you may want to do the pulldown removal in the editor if you plan to use multiple programs.

Do test sections and frames and set your programs to render overnight for rendering if you can.

If you're having other people help with the shots then do a turnover with them and be clear about what you need. Figure out a naming convention and how you plan to pass images back and forth. Firewire drive, networking, etc.

Try to keep an honest assessment of your work and the time required. Is the quality up to the level you need? Can you finish all the shots in the time you have? Use a schedule to help keep track of your time and your target.

Try to rough in the animation or basic composites for a sequence so you can be sure it's working in your film before you spend a lot of time fine tuning the shots.