Why do visual effects costs so much?

Why do visual effects costs so much?

This question comes up constantly even from those who should know better. Many simply wish to ignore the reasons. If you haven’t done so yet please read The Value of Visual Effects post to put this perspective. Discussing costs doesn’t mean anything unless you place a value on it.

Here is a typical response in Hollywood:

Peter Berg, Director of BATTLESHIP, was interviewed on a podcast recently.

Berg: But the money is all going to… the business to be in is ILM. That’s whose making all the money..

Masters: The effects houses.

Berg: Yeah in particular ILM. I mean and they do great work but its what these films cost because you’ve got these giant visual effects components and they dictate the prices on them.


There a number of flaws in this thinking.

Myth 1: Visual effects are the majority of the costs. This may or may not be true depending on the specific project. Normally visual effects are less than 1/2 of the project even for 'visual effects films'. 1/5 to 1/3 is a more likely scenario. But if a movie is made almost entirely with visual effects in every shot and they’re complex visual effects it will be a correspondently larger portion of the budget.

Myth 2: High costs means high profits. Just because a line item is expensive doesn’t mean that there is a corresponding profit. Amount of costs and profit are independent issues. Just as a movie itself may be expensive but that doesn’t guarantee a huge profit.

Myth 3: ILM or any other visual effects company dictate the prices. Obviously they have to quote a price but they can hardly charge whatever they wish or make huge mark-ups. The market doesn’t allow it.  If that price is too high the studios simply go elsewhere.

Myth 4: All visual effects have huge mark-ups and profits. Even in the days before digital effects producers were all convinced that visual effects companies were raking them over the coals because they had what the producers wanted. Producers were (and still are) convinced visual effects companies were simply adding huge markups because they could. These people are convinced if they only knew some of the mystic technobabble they could get the work done for a fraction of the price.


Reality:
Short version: Visual Effects is incredibly time consuming and labor intensive work done for very little profit. In some cases it may actually cost less for the studio than the real costs incurred. Changes and compressed schedules  increase the costs further.

Related posts:
Visual Effects are inexpensive
The Miracle of Visual Effects

Full version:
Visual effects is a very competitive market worldwide. There are a lot of visual effects companies and all of them eager to get work so competition alone does not allow any company to have huge mark-ups.

The visual effects industry suffers from tax incentives in other states and countries. Some as much as 40% off. Note that these figures are not reduction of taxes but actually funds applied directly or indirectly to a movie. That means it’s not a level completive field and any visual effects company in California has to drop their prices if they wish to compete with other companies on the basis of price. Many visual effects companies actually underbid the work when required to try to keep money coming in. This means in some cases it’s costing the studio less than it actually costs. Same thing with the tax incentives. In those cases the local tax payers are in fact helping to fund the movie and thereby lower the studios expense for the visual effects.

The amount of work for visual effects companies fluctuates widely so when it’s all said and done on a yearly basis most visual effects companies make razor thin profits or may be further in debt. Many visual effects companies have gone out of business. Some of the ones still operating do so because they have the backing of a large corporation or individual with deep pockets. Some companies are able to stay afloat because they are in a country or area with tax incentives. There’s good work done all around the world and price is not the only factor studios look at but the visual effects industry would be much different if there were no tax incentives.

If visual effects were really such a profit center the studios and investors would be breaking down the door to buy or create their own visual effects companies. Some studios have had their own visual effects departments in the past. Disney had Secret Lab and Image Movers at different times. They closed both. The only major studio currently with a visual effects component is Sony with their Sony Imageworks. Sony had moved a number of jobs to Albuquerque, New Mexico a few years ago in an attempt to get in on the New Mexico tax incentives. They’ve now closed that and are moving many jobs to Vancouver for the tax incentives there. There may be a future where all visual effects for Hollywood based movies are all done out of this country simply due to tax incentives. This is the outsourcing that isn’t talked about.

See the Digital Domain IPO documents showing one of the more successful California companies. They’re trying to make animated films, getting involved in for profit schools and getting money from Florida in an attempt to make money since visual effects is not cutting it for them.

[Update: 7/21/2012  New Digital Domain news item
Textor said investors are punishing Digital Domain because of the low-margin nature of its visual effects business.
...
In the first quarter of 2012, Digital Domain reported a loss of $14.8 million on revenue of $31.1 million.  

Digital Domain is one of the largest and more successful visual effects businesses in the U.S. And it lost $14.8 million in a single quarter. Hardly the idea of a big profit center. What if they had billed the full out of pocket costs? ]

[Update: 9/11/2012 Digital Domain has closed it's Florida facility and has spun off their visual effects group for a much smaller amount and filed for chapter 11 for the main holding company. This after going with a full IPO less than a year ago. No, visual effects is not a big profit center. ]


Most of the larger animation companies create their own content and receive the profits from how well the films do. Visual effects companies at times have a small percent of a film but most of the time the companies are doing work for hire.

Live Action Movies:
The blame for high priced movies seems to be placed directly on visual effects by some people but lets look at non-visual effects films.
Why are movies so expensive to make even without visual effects?

That’s a common question for laypeople. Why should it costs millions to make a movie let alone $100 million or more? How long can it possibly take to shoot a 2 hour movie? And isn’t it all done by a handful of movie people?

Time:
A movie typically takes a year to make from green light (approved funding) to theaters. That’s after possibly spending years in development. There’s a pre-production stage (building sets, casting, working on the script, finding locations, etc) that can be a few months, production (filming) can take 40 to 100+ shooting days. A 100 shooting day schedule is 20 weeks. The shoot days are the most expensive days of the film because there’s a large crew working so the more shoot days the more expensive the film, all things being equal. And time is the gold standard for shooting such that production is guiding the director to do it as quickly as possible and to remain on schedule. Once shooting is finished the film goes into post-production. This is when editing happens along with sound mixing, composing, recording of the music and this is when most of the visual effects work is done. This is also the most likely compressed stage since the movie has already been booked into theaters and that date cannot be changed. So studios tend to provide shorter post-production schedules than in the past. sometimes this requires the director to edit and lock sequences even before the filming is finished just so there is enough time to do the visual effects.


Costs:
Even non-visual effects films can be very expensive. Ignore high actor salaries and marketing costs. Even looking at just below the line costs (crew) on a large production. Camera crew, grips, electrics, wardrobe, makeup,  audio crew, etc. And consider all the people not on the set that are required to make a movie - art dept. with carpenters, painters and others making sets, orchestra, sound mixers, sound effects, etc. Watch the credits of a large non visual effects film and see how many people are listed. Those are skilled, experienced, talented people being paid a reasonable rate for the services they provide in this freelance world. That’s hundreds of people. It’s as if you started a company, staffed it up and ran it for a year before any revenues appear. That’s expensive.

Labor:
Few people actually sit down and consider the enormous cost of labor.  I’ll be making up some numbers so don’t take these as actuals or even averages. Let’s suppose we had 300 people working on a project at an average rate of $80,000 a year. The average income in the US in theory is around $30,000-$55,000 a year but most in the film industry don’t work full time over the year and this also has to take into account overtime which most jobs don’t have so don’t assume even someone making $80,000 rate ends up with $80,000 at the end of the year.  That’s 300 people at $320 a day = $96,000 a day in expenses just for labor. Now you can adjust the number of people or the income either way but the fact is labor is more expensive than most people realize. Even in this example that’s $480,000 a week (almost 1/2 million dollars) A 6 month project with these people will rack up $12 million just in labor costs. It would be $15 million if the average rate was $100,000. Add all the extras costs (building, supplies, support, rentals, etc) and adjust for overtime and work required. In the case of a movie add in the high priced talent, directors and other above the line costs along with transportation, locations, equipment and other costs as well as marketing.

Take a look at the company you work for. How many people work there and what’s their average pay? This will help you calculate how much money needs to be coming in just to break even on labor alone. Don’t forget most workers get benefits so the cost to the company per person is above and beyond just their rates.

The visual effects work:
Each visual effects shot (5-8 seconds typically) is unique. It's the same as having to do a complete new setup in live action which requires changing the camera position and the lighting. In many cases even shots can be equivalent to the doing a full 'company' move where the live action crew has to travel and setup at a new location. a time consuming and expensive process.

In a previous article I noted that shots are like snowflakes since no two are identical.  I compared them to oil paintings. Another analogy is to consider the visual effects crew is constructing a building based on nothing but a sketch on the back of a napkin and the specifics of where walls go and what carpet is selected for each different room is in constant flux. Creativity combined with technically challenging work, fixed bids, changes and deadlines does not make the process easy. I don't think there is another industry like visual effects that has this same type of business model.

Visual Effects expenses:
Beyond what most businesses have as basic expenses (buildings, desks, basic services, etc) visual effects companies have to provide at least one high end computer per worker. These tend to be top of the line fast computers with a lot of memory, large hard disks and advanced graphics cards and graphics tablets. Each needs  to be loaded with high end graphics software. In addition there’s typically at least one large room with racks and racks of computers for rendering the images you see. These are the render farms and usually have special power and air conditioning needs that make them very expensive. All of these computer systems require advanced and costly wiring along with an IT department to support it all. Hundreds or thousands of shots at 24 high resolution images a second times dozens of elements equates to a lot of storage space that needs to be maintained and archived to avoid losing critical work.

Visual effects companies have screening rooms and editors with editing systems so they can be synced to the production. They have special security systems as mandated by the MPAA to keep all studio materials under safeguards. Many companies have at least a small insert stage to shoot elements and odds and ends as required. A visual effects company may also have a motion capture stage, 3D scanners and model shop or have to sub-contract this type of work when required.

Hidden crew:
But even with these expenses the labor is the largest cost for doing visual effects. This labor is the hidden crew. Most directors and producers seldom see much of the visual effects crew because these crews work away from the studio. Even for those directors that visit the  companies they likely only see part of the crew and since most visual effects these days are spread out over multiple companies (shorter time schedules) the work is being done all over the world (thanks in part to tax incentives).

On a live action set most of the crew is standing there, ready to work. It’s obvious that decisions need to be made to keep these people productive and there is a schedule for what needs to be accomplished everyday. Since most of the visual effects crew is out of sight, they’re out of mind. There’s a disconnect to making decisions and changes and what that relates to in terms of costs and time. On a set if the director turns the other way and wants an elaborate set built the cost and crew time is evident to all. In many cases the notion may be dismissed as not being worth it. A director is constantly making decisions and makes a number of compromises at all steps of production. Should the director shoot another 10 takes of the actor to try to get a better take? Should they try a totally different angle or camera move? Make adjustments to the set? At some point the director has to move on if they want to make their day and complete the film in the number of shooting days budgeted.

Disconnect:
But none of that is evident when working with visual effects. Adding a 100 shots or doing 100 takes of an animation only takes a request. What impact it has on the schedule and costs is seldom considered. Because of the competitive nature of visual effects and the fact that the number of potential clients (the studios) is less than a dozen, companies forego many change orders, thereby lowering the price of doing the actual visual effects work even more.

And this is another disconnect. Just about everyone else working on a production is  employed by the production company directly. Should they be required to work more days or hours they tend to be paid. And crew members are union members except for visual effects workers. Visual effects people are primarily employed by a visual effects companies and not the production companies. Most are freelance and have to switch projects to keep working. Artists may or may not be paid overtime. They also may or may not get benefits. All depends on the company and location.

Color correcting, making film prints and other services are done at a lab. Sound mixing is done at a sound company. These types of companies typically work on a time and material basis. If the director wishes to spend another day making adjustments then that cost is obvious and billed accordingly. A visual effects company bids on a fixed bid even with partial information. Even with accepted change orders this tends to whittle down any planned profits.

Visual effects crew size:
The number of people working on visual effects varies with the scope of the work. It's not unusual for a major visual effects film to require hundreds or more people. On larger projects the size of the visual effects can easily eclipse the size of the rest of the crew. Next time you're at a large visual effects film sit through the entire credit list. There are a lot of people listed under visual effects. And usually there will be quite a few companies listed. Note that you won't actually see the full list of all visual effects people who worked on a film. The end credits is usually a partial list since the studios only allow so many credits to each company depending on the contracts. There may be people from visual effects not listed that worked for a year but it's likely someone who was an assistant to an assistant getting coffee for a week on the set is in the credits.

What do all of these people do:
(I'll have a post in the future listing the various positions. The Visual Effects Society lists over 200 job titles) [Update: Here's that post that sits many, but not all, key positions. In some cases there may be a few dozen people in the same position, working on different shots. ]

A visual effects company is like a mini-studio with a wide range of artists and craftspeople.  In pre-production artists are drawing and designing, modelers are sculpting every set, actor, prop and location required to be rendered. Everything is built from scratch. Every computer graphics actor or creature needs a skeleton and skin that moves correctly. Texture painters are painting everything down to the finger nail or leaf. Specialists are focused on how the clothing moves.

If the project was shot on film a small team will be going through every frame and painting out the dirt. Yes, there are a number of jobs in visual effects that require working on every frame. By hand. Others are hand tracing actors in specific scenes so new backgrounds can be replaced. Anywhere something had to be removed (stunt rig, etc) needs someone to hand paint it out. Remember that film is running at 24 images a second. Visual effects on a 2 hour movie could be  a lot of frames (172,800 frames). Most movies probably around 2000 shots - visual effects films likely have at least 500 or more shots. In some cases every shot in the film is worked on.

Match movers have to create a computer camera that exactly matches what the real camera did. Animators have to animate any computer graphics actors or creatures. Lighters and others will have to light scenes just like a cinematographer. All of these images have to be calculated and rendered. And then compositors work to combine all of these images into the final shot to be used in the movie. In addition visual effects requires a supervisor, producer, editor and numerous other support people.

Visual effects time:
Visual effects is one of the first departments involved on a project in pre-production  (or should be ) and continues until the end of post-production. This can be a year or more on large projects. Remember that a large animation film probably averages 3 years. Post-production time is frequently being squeezed into 4 to 9 months. Because post-production time is less than it used to be most of the crews start working overtime from the start. The films release date is locked so it's not unusual for the last few weeks or even months require working 80+ hour work weeks. 24 hour work days are not unheard of.

Computers:
The question of computers comes up as well. Don't computers do all the work anyway? NO. A camera doesn't film a scene by itself and a word processor doesn't write a script. The computer is simply a tool. And no, there is no magic button on the keyboard to do the work.

Computers are getting faster so why aren't visual effects a lot cheaper and faster to make? Just as a faster computer doesn't allow a writer to now create a novel in a day it doesn't allow a visual artist to do a shot in a shorter time. Much of the time is spent thinking, planning and working. Any speed ups or reduction in costs due to improvements of techniques and people is eclipsed by the next project since it requires more shots that are even more complex in a shorter period of time. Many movies are about taking it up to the next notch for the audience. That next notch absorbs any gains. Newer requirements also demand extra time and labor - 3D, 4k resolution, and/or higher frame rates.

Base cost:
All of this totals up to a  large price to have a very large crew of very talented, skilled and experienced people working long hours over this length of time. Consider  that the companies had to provide a fixed bid based on how many people and how much time was required, simply based on storyboards and some previs. (rough animations) They also have to estimate how much a new and never before seen effect for the film will cost. Calculated in these budgets is the assumption of how many takes will be required by the director.

The reality:
If the company estimated 4 takes per shot it's just as likely the director will require 12 takes for every shot. And that's just the start of things that happen to the 'profits' of visual effects. On a location in a forest clearing the director may decide to replace the sky for the entire sequence. Seems a simple enough task given all the other work the companies are doing but now people have to try to separate the existing sky in every shot. Some of that may be possible using semi-automated techniques but more than likely it will require a team of rotoscopers (artists who trace to create mattes). Every leaf and branch needs to be traced and any actor (and their hair) have to be separated. Maybe a prop doesn't work. That's added to the list as well to be added or replaced later in visual effects. If a stunt or special effects action doesn't quite work or isn't as big as the director wanted, that's added to the visual effects shot list.

When projecting the dailies in a theater after the shoot is finished, the studio may notice they don't care for the makeup or may spot wig netting or other flaws in the footage. That's added to the list of visual effects shots. Boom mics and crew members in shots are also added to the list. Some shots that were originally planned to have no visual effects now require them based on the edit. Based on test screenings there may be a need for reshoots or new scenes to be shot. Since the sets may have been struck or the actors may not have time to go back the location, these are shot as greenscreens with the visual effects company responsible to add the backgrounds and match all the other footage.

There are times when visual effects are held to a different standard than live action. On a location if a stunt car rolls over for a couple of takes the director may accept it and move on. The shot works fine in the movie. It may not be exactly what the director had in mind but it tells the story. Since visual effects can control everything down to the pixel with precise adjustment the director and studio may wish to tweak and adjust down to the last minutia. If the stunt car was done as a visual effects shot it may be requested to make it roll 6 frames earlier and to roll 5 degrees more. On the 20th take it may be decided to hit a specific parking meter which needs to bend at a specific angle.

Most of this feedback and the changes requested are sent to the hidden visual effects crews. Since these people are seldom seen (with the exception of some people in Skype conferences) it's easy to lose sight of the labor costs being incurred daily. Also unseen is the amount of work and overtime being put in to making the visual effects.

The need to get all of the shots done by the finals deadline drives the pacing. It's easy to spend months on a small number of shots and tweak them and then end up rushing the last batch of shots. The quality of visual effects shots in most cases is directly proportional to the time allowed to finish and polish them. Rushed shots will have flaws and that's why companies sometimes push for a CBB (Could Be Better) shot status. It's not perfect but it can work in the movie. If time permits some or all of these will revisited but that's better than the alternative of tweaking shots the first batch of shots and having a week to finish half the movie.

Remember this all started with a fixed bid for a specific number of shots in a specific time frame. The deadline never changes so the companies now find themselves having to do many more visual effects shots with increased complexity all in the same time frame. In some cases there may be clear changes that enable submitting change orders. But in many cases such as increased noodling and takes, it's a gray area. That tends to erode much of the profits the company was able to build into the original bid.

The amount of work flowing into a visual effects company can also fluctuate a lot. They may finish a very large project and then have a few months with no or little work. Many of the crew are laid off but there's still a need to keep a core team along with paying for all the overhead costs. Any profits made on projects will have to pay for these lulls in production.

Extra costs:
But that's not the end of the story. Many decisions may be delayed. There's usually no one keeping the post-production moving in the manner that the live action was. Sequences aren't turned over while edits and re-edits are done. The studio and director may have conflicting ideas. Test screenings popup with little notice. Results of test screenings require more changes. Sequences are added. Shots and sequences that had been approved 2 months earlier need to be redone with major changes. In some cases the majority of the visual effects happen in the last month or two of the film. What had been scheduled for 6 months must now be done in a fraction of that time and it must be done well. The deadline remains fixed since it must be in theaters as scheduled. And it's not like they can just hire another 1000 people to help out at that stage and make up for the added work. No, the original team will have to put even more hours a day and work 7 days a week. The more hours worked the lower the productivity of the workers. Some work may be farmed out to other companies when possible as 911 emergency calls are made. But that comes with a price of time and money as those companies get up to speed.

To put this in perspective think how much it would cost a live action production to scrap a month of shooting and be required to build new sets and reshoot on new sets, all in the period of a week.

If the artists are compensated for the overtime then prices really start to skyrocket. And more than likely after this big push most of these artists will be laid off until the next rushed project comes through the door.

And that is why visual effects cost so much.



Related posts:
Visual Effects are inexpensive
The Miracle of Visual Effects
Visual Effects - The Big Picture
VFX Business Models
How VFX is perceived by at least one DP
People, not computers, create visual effects
Getting the most out of your VFX Budget

Visual effects service - The Big Picture

(Note: I’m on the board of the VES but all posting to this blog are mine own and do not represent the VES) (As always, I could be wrong about anything)

If we’re re-examining the current VFX situation we need to take a step back and look at the whole process.

I think most of us think of visual effects as a service. But is it?

History
Years ago the studios had their own visual effects department with people on staff and basic optical and animation equipment. When the studios closed those departments, labs and small optical companies took their place. I can remember looking over lab and vfx company price lists in the mid-70’s. A dissolve was so much per foot. A matte was so much per foot with minimum cost of X dollars. Some of the places had small insert stages they would rent for so much per day. If you needed something special they could give you a quote but the majority of the work was on a time and materials basis.

For larger vfx projects the productions themselves would set up a full department somewhere. This was the case with 2001, Logan’s Run, Close Encounters, Star Wars and other films. The production would lease a building and set it up from scratch with the people needed to run and operate it. For Close Encounters we were in an industrial building in Marina del Rey. They had to custom make the matte painting stands and other equipment as need be. They purchased or leased optical printers and an animation camera. Everyone working there was paid by the production. This is in fact how productions usually run. They become their own company that has a group of people that round up or build whatever is required for the film. In live action this would be the construction of the sets, special rigs, etc. Once those shows were done the one off facility they setup would usually be closed down and the crew laid off. In the case of ILM, since George Lucas had such success with Star Wars and thought he might like to do more things for himself and his friends he re-created it in Northern California. Apogee was formed by John Dykstra and others using the original ILM building and some of the same gear(?) used on Star Wars. Many of the Close Encounters people went to Universal Hartland where Universal setup up a facility to handle Buck Rogers and Battlestar Galactica.

When we formed Dream Quest most of the initial projects we did on a time and materials basis, especially if it was large. If someone wanted us to shoot motion control we charged for the stages and the crew for so much per week. This saved us in a number of cases where the director or the vfx supervisor they had hired made changes or threw out work because of a change.

Today if you’re working on a commercial the cost of a telecine is so much per hour and so much for tape, etc. The work on the Flame system or equivalent is on a per hour basis. They may provide a rough estimate but it’s always up to the client how much time is used. If the agency wants to tweak something all day that’s fine. They get billed for it and the video house doesn’t have to worry about making a budget.

Fixed Bids
Somewhere along the way the studios wanted a fixed bid on vfx work for feature films. Estimates were no longer good enough and the vfx companies would now have to stick to the budget. This changes a number of dynamics. In the eyes of some studios and directors the vfx people were no longer working directly for the production, they were working for the vfx company. The vfx crew became another step further from the film crew. Outsiders. Now there’s always the ping-pong of trying to please the director but not going over budget. Filing change orders and having discussions with the studios regarding the costs now became standard process. At times the vfx company can be pushed into a corner. The vfx supervisor was now that guy from the vfx company. The name of the vfx company became the main selling point. The vfx supervisor, not so much. If the client doesn’t like the first proposed company supe then another one at the company will be swapped out on a whim. The crew having to work overtime was now the vfx company’s problem. We still had crazy hours at times in the ‘old days’. On Star Trek: The Motion Picture I worked several weeks straight, 12 hrs a day, 7 days a week. (and a few 24hr days) But the studio knew it and actually visited the facility. Today, they’re now removed from those details.

So are there any other areas of film production that are completely farmed out to a 3rd party company besides VFX? All the other main leads tend to be hired directly. DP, Production Designer, Wardrobe, etc. Even though most Special Effects people have companies I believe most are hired as a team of people or at least paid on a time basis. Sound mixing or the DI? I assume these are on a time and material basis as well. The previs team is frequently brought in to work in the same office down the hall from the director. They typically bill by the man days or hours. Most of the set construction I see is done by a team of people working for production with special projects (cars, etc) farmed out.

When I think of a service I think of a dentist, a car shop where they work on your car or a plumber that comes to your house. In these cases they do work but don’t tend to produce anything. The costs are based on time and materials.

Custom manufacturing?
Should vfx be considered as custom manufacturing? We actually create something when we finish our work, whether it’s from scratch or a montage of material provided. That’s what the studios want, not the actual service part.

Here is where things get crazier. Each shot is unique like a snowflake. It’s own little world of issues, handwork and tweaks. You try like anything to make shots as consistent as possible and to be able to run them through the exact same process but it’s never full automated. For all the talk about computers in our business it’s still a very labor-intensive process. The number of people and the time required to do a shot from start to finish would astound most outsiders.

If you go to most manufactures and request custom work you will be required to make specific requirements in writing. (I.e. you want cabinet style 32 but in this specific color of blue. You want a custom cake that says Happy Birthday. It will be yellow cake with vanilla ice cream and chocolate frosting.) And that is what you will get. They seldom show you the work in progress or have your input at every single stage. The other thing is a custom manufacture will tell you when it will be done. They dictate the schedule. In the film business it’s the opposite of all of this. The studio specifies when the delivery will be. It’s almost always less than the time that would have been arrived at by a normal scheduling process for the facility.

On a VFX project you start with the script, which provides a wide-open interpretation of what the final visuals will look like. In pre-production the director hopefully approves concept art, does storyboards and ideally previs. While most previs lays a good foundation the number of nuances and changes required for the final shots can be enormous. The director usually wants something never seen before that will require a lot of R&D. Not just custom but a totally unknown look or process that needs to be invented. Just how much time and money will that take? The vfx companies have to provide a bid for all of this before the film is even shot. During shooting things will change. During post-production things will continue to change.

This is a creative process so there will be changes but think of it this way: The vfx company is making a 1000 custom oil paintings that technically have to be delivered on a hard date for a fixed price (at least initially). This process could costs in the 10’s of millions of dollars, make up half of the film budget and fill up half of the screen time. There are some rough thumbnails but not enough information to simply deliver the finished paintings. The director is involved at every step of the process for every single painting. In some cases, for every brush stroke. Some directors only want to see the final pieces. In these cases you can end up with ‘no, now that I see it I don’t’ want apples in the painting, I want pears’. So much for the time and effort to create the initial painting. If a director changes one painting that may change two dozen that are almost finished. Remember, the due date will not move, regardless of the changes. And of course shots are not paintings but moving images so time and motion presents another infinite number of possibilities.

How many other areas does the director really work in this much minutia? Normally when they’re working with Directors of Photography, Production Designers, etc they discuss and try to get in sync regarding the general look and style they want. The director may be asked about the color of the pillows on a set but at some point they pass on the taking care of the details to their key creatives. The director is unlikely to ask to change the 3rd brick from the right on the set or ask the DP to reduce a specific light by ½ stop. And yet at times it can be that way when working with visual effects.

With visual effects the director has unlimited control. Every pixel of every frame can be changed. If production has an on set stunt or action the director shoots what takes they feel are appropriate and will select one. The fact that the stuntman’s hand is raised a little doesn’t cause problems. The best take will be selected and production moves on. With the advent of digital visual effects that’s not the end of the story. What would have been fine previously in any movie is now something to be scrutinized and analyzed by the director, editor and studio. Now it may be an added shot for the vfx crew to fix that hand position. And while they’re working on the shot can they change that thing back there and that other thing over there? A shot with a jet may get a request to roll the jet another 3 degrees. Will the audience notice 3 degrees? Will it make it a better shot? Obviously if production paid and shot a real jet they would be unlikely to schedule another shoot day simply to get the jet to roll 3 degrees more.

On the set the director knows it will take a certain amount of time to make a change so they always have to balance that because time is their gold standard. They have so many days to shoot the show and have only 2 days scheduled for this set and need to shoot 20 setups a day. With vfx that time balance is thrown out the window. Most of the work is done after filming. The amount of time and effort to make the change is all hidden. It’s happening elsewhere by unseen people. It’s no longer the director or producers responsibility to complete this phase of production on time; it’s up to the vfx company. To add to this difficulty is the fact that the live action shoot can and does go over schedule. Problems during shooting may now require additional, unplanned work to be done by the vfx company. But the vfx company cannot go over schedule. They are the end of the road so every delay during shooting, every added fix, shot or change needs to happen by the deadline. That’s the finals date that was set before the vfx company even started bidding on the show. Not only does the vfx company have to do all the work they initially agreed to do in that time, they have to absorb most production issues that have accumulated and rippled down the pipeline since the pre-production began. Add into that mix the requirement by the studio to make last minute changes, possibly based on test screenings, possibly based on an idea of an executive.

Are there other non-film businesses setup like vfx companies in terms of the requirements and client involvement? That would be useful to look at and learn from. Unfortunately I can’t really think of anything on the scale or dealing with the same types of issues. Many construction projects are of course custom and involve a lot of money and people. However they have blueprints that have been signed off on. They have colors that were selected to paint the walls and the client has approved the carpet and the tiles. Sure there will be some changes but the majority of the work is usually very well specified. Any major changes will involve a change of completion date or will require client to pay a large fees to have it accelerated.

Summary
Visual effects is a very labor-intensive business. The labor is made up of dedicated and highly skilled and trained people. There’s the requirement to complete hundreds of works of unique, never before seen, art (shots), based on rudimentary starting points, that are constantly being scrutinized and changed. And this all has to be done for as much adherence to a fixed bid as possible and above all has to be finished on the deadline, - no ifs, ands or buts.

I do want to go on record that I support all the directors I work for and that I’m all for anything that can make a film better. All vfx artists want the best possible film. What I hope this posting will illustrate is just how complex this issue is. We have art, technology and commerce all colliding. The vfx companies are put in a tough situation and the vfx artists are put in a tough situation to try to balance this all out. The end result is any process or structure that will help balance this issue to create the best creative and to make it reasonable for the vfx artists will be a welcome relief.

Related posts:
Pass me a nail
Risk and subsidies
Oh, the mess we're in!

PostProduction

PostProduction for Visual Effects is covered in this podcast.

Here's the timeline:
2:10 Client turnovers
6:24 Effects editorial
8:41 Scanning
12:08 Effects turnovers
15:21 Artists - Matchmover, Animator, TD, Compositor, Rotoscoper, Painter, CG Modeler
19:10 Work day
22:17 Dailies
24:34 Client reviews
26;33 Temp screenings
28:09 Balancing work and schedule
29:36 Filmmaker

Transcript
Today I'll be discussing Post-Production for Visual Effects. This will be an overview with later podcasts to cover each specific area in more detail.

As with all of these podcasts I'm providing information based on my experience. Every effects house and film will be slightly different so use this as a starting point.

Effects Pre-Production
By the time the actual work is to start on the visual effects most of the CG models that will be animated will have been built, chained and painted. The various pipelines should already be set up. A pipeline is the workflow and the support for that workflow with specific tools. A shot may be passed assembly line style from one person to another. It may also be worked on by more than one person at the same time. The images and the data is passed from one software package to another. Animation may have to be done in one piece of software and then exported to be lit in another piece of software and then rendered using yet a different software. Creatures with fur may have to go through a different render pipeline than non-furred creatures. The output images are then composited in yet another piece of software. The idea is to make this transfer of data as easy and seamless as possible. This may involve writing special databases and scripts to help move the data and keep track of it. If this is an established effects company most of this may already be in place but there always seems to be some changes required. The CG supervisor is usually working on this pipeline while the film is being shot.

If there is an animated CG creature most of the voice recordings are also done by this point.

Client Turnovers
The first step of the postproduction process is the client turnover.
When a sequence has been edited the director, editor and producer turn over the visual effects shots to the effects company. The effects company team includes the effects supervisor, animation supervisor, effects producer and typically an effects coordinator. Everyone gathers around the editing machine, usually an Avid, while the director explains what's required in each shot. A 10-minute sequence in the film can easily take 2 hours to review and discuss due to the complexities of the shots and action.

Even though the effects shots have been storyboarded in pre-production there are always additional details now that the background plates have been shot. There are always some changes as well in the shooting and editing process. A single planned shot may have been broken into to 2 shots or two shots may have been combined into one big shot. In some cases shots have been dropped for pacing reasons and a few may have been added to help the story telling.

Depending on the shot it may be difficult to make the edits, especially for a shot where most of the scene will be added later. If a shot will be all virtual as a standalone miniature shot or a CG virtual shot then they will be cutting in a storyboard or an animatic to use as temp placeholder for the real shot to come later. They may have cut in the reference take so they can see how the action flows but they will have selected their real take based on the actors performance. In other cases they want to use a reference take and you'll need to remove the other actor or monster stick.

One thing I try to do when possible is to create mockups before the edit. I did this quite a bit on Van Helsing. A QuickTime of a shot would be copied to my iPod while we were shooting and then I'd tried to do a basic mockup at night back at the hotel and dump it back out. This tended to be mainly 2D mockups but was enough for them to check the take and get a sense for the action. It also allowed me to see how well something was working and to determine if we may have to make adjustments in the future.

The editor or his assistant may do simple mockups on their editing system. A shot may be split or a simple bluescreen key may be done. If animatics are generated it's sometimes useful to do a render of just the foreground or background. If it's a background this can be comped behind a bluescreen and if it's a foreground piece this can be comped over the real background. All of this is done to provide a better sense for which take and timings work best. In some cases they may pick alternate takes if there's a problem later with animation timing.

Since each shot is covered in great detail with much finger pointing at the video screen and pantomiming by the director we usually video tape these sessions in addition to taking notes. It's important during these sessions to get as much information and detail as possible since any miscues could be costly in time and money.

Since the number of visual effects shots these days is going up and the amount of postproduction time is getting shorter it's very common to do these turnovers while the film is still being shot. This is usually spelled out in the contract since the film has a fixed release date. The delivery date for all the shots to be completed by the effects company is called the finals date and can be 2 weeks to 2 months before the release of the film. The editor will start making an assembly from the dailies and the director will review this after a day of shooting or on the weekends. Once the director is happy with a full sequence the turnover will happen at the end of a shooting day or on a weekend as well.


Effects Editorial
Once the client turnover is done the show's editorial department usually ships a copy of the edited sequence. This may be on a tape but ideally is in Avid project or some means for us to assemble our own edited version. It's important to have not just the visual effects shots but the surrounding live action shots as well. This will allow checking the in-progress effects shots in context so we can check the timing, actions and the match.

The effects editor creates a breakdown, line up or cross over sheet. This will list the sections of film that need to be scanned and any additional elements required. The standard references of the lighting sphere and other items are some of the additional elements. The effects or animation supervisor may want to scan additional reference footage.

The editor has the option of extending a live action shot at any point because there's usually plenty of additional footage on the head and tail of the selected cut. With visual effects though you end up with a shot just as long as specified. To provide a little latitude for offsetting or increasing the shot length an effects shot usually includes additional frames at the head and tail of the shot. These extra frames are known as the handle. This could be 2 frames at each end or up to 16 frames at each end.

The more frames you have, the more flexibility in the edit to make an adjustment but these added frames add an additional cost to each and every shot. 16 frames of handle would mean a total of 32 frames of extra work that likely won't be used. That's a second and a half. That's a lot since the typical effects shot is 5 to 8 seconds in length. A 4-frame handle is probably most typical. The average shot length and required handles is spelled out in any effects contract. The size of the handle can be discussed on a shot by shot basis since in some cases the extra frames may require a lot of paint out or other work.

Scanning
Once the information is together and the original negative is available it's scanned into the computer. Some effects houses do this themselves and others send this out to a service to be done. Normally I try to obtain timing clips from the DP (director of photography) so we can do some basic color balance when scanning. A timing clip is a few frames of a shot or shots that the DP has worked with the lab to create and is an example of the color and brightness for the final shots. This shows the DP's vision of what the end result should look like. Some places wait until the end of the process to do a full color balance.

Scans are usually specified by their resolution, bit depth and format.
There are a number of formats and possible aspect ratios when shooting: 1:85, 2:35 anamorphic, Super 35, etc. It's normal to try to scan a bit more of the frame if it exists such as in 1:85 on the top and bottom (and one side if it's academy format) This allows a bit more room to move the clips.

Typical resolution for feature films is approx 2k or 2048 pixels across. Some films are being done at 4k or 4096 pixels across but that requires 4 times the amount of data and processing. The bit depth refers to how many levels of lumance for each color. 8 bits is 256 levels and provides over 16 million colors for red, green and blue. 10 bits or higher is now more common since it provides finer increments of color. It's also important that these scans be logarithmic to provide the best use of those bits. Some people think films have to be more than 2k and have deep color but the fact is quite a few well known films were done with 8-bit and less than 2k resolution. The file formats are usually in a Cineon format but there are many others, especially for in house scanning.

High-resolution images take up a huge amount of storage space. Even a simple 8-bit, 2k image will take 10 megabytes per frame. Since film runs at 24 frames per second and since there are usually multiple film elements per shot that starts adding up quickly. When possible images are sometimes compressed in a lossless compression. This will reduce the amount of storage required and reduce the amount of data to be copied between computers. These scanned shots are dumped to a high-density data tape or across a network.
Even though effects houses may have terabytes worth of storage shots are usually brought online (loaded on to hard drives) when the shot is about to start and they're normally removed as soon as the shot is finaled.

After a shot is scanned it typically goes through dirt removal. A team of people sit at computer workstations looking for dirt and scratches that have to be painted out, typically by hand, before the shots are ready for production.

Effects Turnovers
When a few shots have been scanned and are ready to be worked on there are turnovers for the actual people working on the shots. A shot usually goes through a number of departments, some of which work simultaneously on the shot and other departments that may have to wait until the previous department finishes. A single turnover may happen with all the key people who will be working on the shot or it may be broken into multiple turnovers for each department.

The shot will be displayed on a computer or video monitor and the animation or effects supervisor will step through the key issues much as the director did. It's important for all those in the turnover meetings to have a pen and paper for notes. This would seem obvious but a lot people try to remember every detail only to find out they forgot something a week later.

In some cases a section of video of the director turnover may be shown just to make sure there's no miscommunication.

Every shot and every element will have an ID of some type. Possibly the first 3 characters of a sequence name followed by the shot number and then possibly a letter or letters signifying the element itself. All this is written up by editorial and provided to the people working on the shot. Larger houses also have a database with info and access related to each shot. This is critical since one of the bigger tasks in doing visual effects is keeping track of all the pieces.

A single shot can take one person a few hours or can take a number of people in 6 departments a few months to complete. A few linear weeks are probably typical with it being touched by 5 or 6 people. Most people are assigned 2 or 3 shots so they can continue to work even while waiting for feedback or additional elements on one of their shots. Each person is assigned a basic schedule and deadline for their portion of the work.

For a large sequence in the film there may be a sequence lead for technical directing and a lead for animation. The lead TD will establish the basic lighting for the sequence based on the live action and will act as mentor to the other td's and compositors working on the shots in that sequence. They'll also be involved with specific R&D required for a sequence. The animation lead will try to establish the basic character action for the sequence and help to mentor the other animators as required.

A show could have a dozen people but more than likely will have 50 to 300 people working on it. A large effects house may have 2 to 5 shows going on at one time. It's important that schedules and plans are laid out and that sequence leaders be able to manage their team. These crews are made up of true artists that are at the core of the effects work you see. They have a balance of artistry and craftsmanship.

Some of these artists include:

MatchMover
A matchmover is provided with information about what's critical in the shot and what's not critical. In some cases a rough matchmove is requested first so it can be passed on to the animator to get started on. The match mover is provided with measurements, photos and additional information from the set for a specific shot.
Once the matchmove is done a grid or spheres are rendered as an overlay with the background to check the work.


Animator
The animator on the shot will take the matchmove and start roughing out the animation. Ideally the sequence lead animator may block out or rough in the animation over a sequence of shots. A low res single color plastic render will be created for each shot. This is overlayed with the background plates and cut into the sequence. Once this is done it is sent to the editor and director for review.

Technical Director
The Technical Director or TD is the person that does the lighting and rendering of the CG creature or model. The also deal with any 3D technical issues on a shot. On the credits they're usually listed as Digital Artists since the Directors Guild controls anyone who get's a director credit of any type.
The TD can start the lighting as soon as he or she receives the first animation from the animator.

Compositor
The compositor is the person who handles the 2D work including compositing or combining all the different pieces of film or elements, including computer generated elements. They start their work prepping the different images, requesting paint and roto work as need be and then take any rendered CG images from the TD. The compositor adjusts the edges and color balance of each element to make sure they blend seamlessly.

Rotoscoper
The roto person creates hand drawn mattes as needed. If a shot has a CG creature run behind a table someone has to draw the outline of the table and create a high contrast matte that will be used to place the image of the table over the image of the creature. In the case of something like Dragonheart the actor in the foreground has to be rotoscoped while moving so we can place the dragon behind them.

Painter
The painter goes through at the start and paints out any special rigs or other items that shouldn't be in the shot. They try to use the clean plate if it exists.
When a shot is finished it may require some slight touchup on a few frames where the render or composite wasn't quite right. There's much more handwork involved in doing visual effects shots than most people realize.

CG Modeler
Actually modeling usually involves a number of people. People who do the model, others who paint the texture maps and others to build the shapes, skin and skeletons for animation purposes. Hopefully this has been done but additional model shapes may be required, especially for facial animation, while the shots are being worked on.

Some of these tasks such as TD and compositor may be combined into one artist. It's possible for a single person to do most of the shot themselves but this may not always be the most efficient.

Work Day
Once the shot has been turned over each individual goes back to their workstation to work on the shot. As they progress they let other's know there is an update. In many cases they may check their work into a database. This is similar to software development where files can be checked in and out.

When an animator is going to do additional work it's important for them to have the latest version of the matchmove. Likewise it's important for the TD to have the latest animation to light and render.

In the old days you had to finish any work by the end of the day (such as opticals or motion control) to get the film into the lab. In the digital age you try to get everything done before the end of the day as well so the computers can render over night.

The TD may be checking single frame renders during the day as he or she makes adjustments to the lighting. The animator will be looking at wire frames or plastic renders to check animation and the compositor is working on a few key frames to make adjustments.

One of the things that these people also do during the day is to calculate the amount of computing power required. If you have a 300 frame shot and each frame will be taking 1 machine hour to render that needs to be taken into account. A large effects house will have a render farm made up of the individual machines as well as a cluster of specialized rendering computers. A single shot can be spread over multiple computers over the network. A frame could be sub-divided in to multiple sections to be rendered or each frame can be passed to a different computer. There may be dozens of shots being rendered overnight.

Everyone that needs rendering usually logs their request into a database and the CG supervisor will have to review the requests versus amount of computing available and determine how the work will be done. Some of the overnight work is done at lower resolution so animation and basic rendering can be checked without using up too much computing power. If it's a final render then it will have to be a full resolution with motion blur and fur rendering and anything else. These types of renders may take a few days and could be scheduled to run on the weekends.

Additional elements
In preproduction and during filming a list is made of other specific elements that need to be filmed. These could be smoke from chimneys, fire for a building or water splashes that have to match a creature jumping into the water. During the compositing process a few more elements may be found to be necessary to finish the look of the shot. These will have to be filmed and scanned before the compositor can complete the shot. Ideally the effects company has past footage and generic elements for at least some of this.

Dailies
On every show the dailies are handled a bit different. The number of shots and the length of time to review and comment can make it very time consuming.

In the morning the effects supervisor and animation supervisor may meet and quickly review the shots together in private. If the animation supervisor has any comments about rendering, compositing and lighting issues he or she would make it here and visa versa.

Dailies can be held in a large theater with high hres video projection with the entire crew or could be in a smaller room with just the sequence people or could just be a one on one review of the work with the supervisor and the artist. All of this is a balancing act between making sure the artists get feedback on their own work as well as seeing the related work and at the same time avoiding tying them up for all morning sitting and watching dailies.

Animation and TD/Compositing dailies are usually separate. Hopefully the effects editor has already cut the dailies into the edit so it's possible to review the shot in context.

I usually try to review the shots on my machine before dailies so I have some notes and can quickly cover a lot of issues. Some people want to wait for a review until they've fine-tuned their shots but this may mean a week until any progress is seen. If for some reason there's a mistake or miscommunication while the artist is polishing the shot then all that work will have to be re-done.

As with the turnovers all artists should bring a pen and pad of paper to take careful notes during dailies. When possible the TD and compositor should explain what they've done and what they still have yet to do. Because of time constraints most of the focus will be on work that still has to be done or improved rather than praising the artist for sections that have been completed.
During the course of the day the supervisors may check in with the artist or may get email from an artist requesting a review or a question.

Usually there is a coordinator assigned to each supervisor. They help to follow-up with the artists and track down information as needed.

Reviews
Because of the complexities and time of doing visual effects it's critical to make sure to frequently review the shots in context. Some people try to wait until they're finished with the shot but there's a strong likelyhood the director will want to change something and they may not be able to make a critical decision until they see the surrounding shots. Each director has a different ability to visualize the shots. Some have difficulty in viewing plastic renders and making animation decisions based on these. This is a learning process for working with each director.

Once or twice a week the supervisor will determine which shots to show the client. These may be shots in progress where a confirmation is requested, or shots where a question has come up or a shot, which is considered a final by the supervisor. Usually there's an animation final on a shot and then later a TD/compositing final. The supervisor will go to the editing room to show the director and editor the new work. If the director is in another city a DVD may be sent or a live video link is used. The latter is much preferred. The video link allows the artists to be present and hear the comments right from the director.

It's still possible for changes to be made at any point. If changes are outside the scope of the original budget for the shot then a change order may be called for where a new or updated budget is submitted for approval.

Once the shot has finaled on video by the director it needs to be rendered full res if it hasn't already been and then be filmed out. The director views this film projected on a full size screen to determine if it's a final final. When a shot is finaled a few frames are saved at full res for PR purposes and the final shot and key elements are backed up and removed from the server.

Temp screenings
The director probably has to show the film to the studio a few times during the course of postproduction. Of course the director will want as many of the shots done or in at least some form of completion even it's with temporary shots. This is better than showing an actor sword fighting with a ping pong ball on a stick or actors on a bluescreen pointing to a non-existent object.

These temps are also used to help the composer and sound effects people since this work will be ongoing even while the visual effects work is still in progress.

As the film gets closer to release it becomes important for the studio to do test screenings. Unfortunately test audiences don't have too much tolerance for plastic renders of dinosaurs and other temporary shots. If the test screening is a wild success the studio may authorize more effects shots. If a test screening does poorly they may want to rethink and redesign some of the effects shots.

Trailers
The director and marketing people select shots that they think would be good in a trailer. A trailer being the ad for the film shown in theaters and on TV. Unfortunately they sometimes select shots that aren't even scheduled to start for several months. As a result trailer shots can be rushed and the version for the final film will be noticeably better. It's best not to judge the effects work too much based on the trailer.

Balancing
For those of us working in visual effects it becomes very easy to fixate on every detail in a shot. This is especially true since we can change everything in a shot. We loop the footage over and over on a computer monitor or large screen at full magnification. We know where to look so it's easy to spot any imperfections. We have to remind ourselves if we don't spot something in 2 or 3 times through then the shot's done.

The same problem holds true with the directors as well. If they film a jet fighter flying by for real then they will select a take and move on. If it's a motion control model or a CG model then the director usually gets caught up in making the jet roll a specific amount at a specific frame even if the original take matched the live action version. Just because there is infinite control doesn't mean there should be an infinite number of takes.

All this is important because in order to generate hundreds of effects shots in a given amount of time you have to keep moving ahead. If you spend half the postproduction time on a fourth of the shots then the other shots will be rushed and the quality level will drop. It's important to develop a pace that can be maintained during the production or you'll end up working long weeks and hours the closer you get to your finals deadline.

Filmmaker
If you're doing your own film and even all of visual effects there's still some principles to apply.

Before you begin the full postproduction process make sure you have your pipeline working. It's better to work out the kinks before you have 30 shots and a looming deadline. Run a short test shot from your editing software to your 3D software to your compositing software and then back to the editor. Use your manual and online help to make sure you understand the different steps involved.

If this is video, export from your editing software with some additional handles. Keep a log book of the shots including timecode and other information. Avoid re-render and recompressing when possible and use lossless compression such as the QT animation codec if you plan to manipulate the images through more than one software system.

Find out the required field order. Most video footage, especially DV is set for lower field. If this is 24p video make sure you understand how to remove and how to add pulldown. Some effects composting software can do this but you may want to do the pulldown removal in the editor if you plan to use multiple programs.

Do test sections and frames and set your programs to render overnight for rendering if you can.

If you're having other people help with the shots then do a turnover with them and be clear about what you need. Figure out a naming convention and how you plan to pass images back and forth. Firewire drive, networking, etc.

Try to keep an honest assessment of your work and the time required. Is the quality up to the level you need? Can you finish all the shots in the time you have? Use a schedule to help keep track of your time and your target.

Try to rough in the animation or basic composites for a sequence so you can be sure it's working in your film before you spend a lot of time fine tuning the shots.