Why do visual effects costs so much?

Why do visual effects costs so much?

This question comes up constantly even from those who should know better. Many simply wish to ignore the reasons. If you haven’t done so yet please read The Value of Visual Effects post to put this perspective. Discussing costs doesn’t mean anything unless you place a value on it.

Here is a typical response in Hollywood:

Peter Berg, Director of BATTLESHIP, was interviewed on a podcast recently.

Berg: But the money is all going to… the business to be in is ILM. That’s whose making all the money..

Masters: The effects houses.

Berg: Yeah in particular ILM. I mean and they do great work but its what these films cost because you’ve got these giant visual effects components and they dictate the prices on them.


There a number of flaws in this thinking.

Myth 1: Visual effects are the majority of the costs. This may or may not be true depending on the specific project. Normally visual effects are less than 1/2 of the project even for 'visual effects films'. 1/5 to 1/3 is a more likely scenario. But if a movie is made almost entirely with visual effects in every shot and they’re complex visual effects it will be a correspondently larger portion of the budget.

Myth 2: High costs means high profits. Just because a line item is expensive doesn’t mean that there is a corresponding profit. Amount of costs and profit are independent issues. Just as a movie itself may be expensive but that doesn’t guarantee a huge profit.

Myth 3: ILM or any other visual effects company dictate the prices. Obviously they have to quote a price but they can hardly charge whatever they wish or make huge mark-ups. The market doesn’t allow it.  If that price is too high the studios simply go elsewhere.

Myth 4: All visual effects have huge mark-ups and profits. Even in the days before digital effects producers were all convinced that visual effects companies were raking them over the coals because they had what the producers wanted. Producers were (and still are) convinced visual effects companies were simply adding huge markups because they could. These people are convinced if they only knew some of the mystic technobabble they could get the work done for a fraction of the price.


Reality:
Short version: Visual Effects is incredibly time consuming and labor intensive work done for very little profit. In some cases it may actually cost less for the studio than the real costs incurred. Changes and compressed schedules  increase the costs further.

Related posts:
Visual Effects are inexpensive
The Miracle of Visual Effects

Full version:
Visual effects is a very competitive market worldwide. There are a lot of visual effects companies and all of them eager to get work so competition alone does not allow any company to have huge mark-ups.

The visual effects industry suffers from tax incentives in other states and countries. Some as much as 40% off. Note that these figures are not reduction of taxes but actually funds applied directly or indirectly to a movie. That means it’s not a level completive field and any visual effects company in California has to drop their prices if they wish to compete with other companies on the basis of price. Many visual effects companies actually underbid the work when required to try to keep money coming in. This means in some cases it’s costing the studio less than it actually costs. Same thing with the tax incentives. In those cases the local tax payers are in fact helping to fund the movie and thereby lower the studios expense for the visual effects.

The amount of work for visual effects companies fluctuates widely so when it’s all said and done on a yearly basis most visual effects companies make razor thin profits or may be further in debt. Many visual effects companies have gone out of business. Some of the ones still operating do so because they have the backing of a large corporation or individual with deep pockets. Some companies are able to stay afloat because they are in a country or area with tax incentives. There’s good work done all around the world and price is not the only factor studios look at but the visual effects industry would be much different if there were no tax incentives.

If visual effects were really such a profit center the studios and investors would be breaking down the door to buy or create their own visual effects companies. Some studios have had their own visual effects departments in the past. Disney had Secret Lab and Image Movers at different times. They closed both. The only major studio currently with a visual effects component is Sony with their Sony Imageworks. Sony had moved a number of jobs to Albuquerque, New Mexico a few years ago in an attempt to get in on the New Mexico tax incentives. They’ve now closed that and are moving many jobs to Vancouver for the tax incentives there. There may be a future where all visual effects for Hollywood based movies are all done out of this country simply due to tax incentives. This is the outsourcing that isn’t talked about.

See the Digital Domain IPO documents showing one of the more successful California companies. They’re trying to make animated films, getting involved in for profit schools and getting money from Florida in an attempt to make money since visual effects is not cutting it for them.

[Update: 7/21/2012  New Digital Domain news item
Textor said investors are punishing Digital Domain because of the low-margin nature of its visual effects business.
...
In the first quarter of 2012, Digital Domain reported a loss of $14.8 million on revenue of $31.1 million.  

Digital Domain is one of the largest and more successful visual effects businesses in the U.S. And it lost $14.8 million in a single quarter. Hardly the idea of a big profit center. What if they had billed the full out of pocket costs? ]

[Update: 9/11/2012 Digital Domain has closed it's Florida facility and has spun off their visual effects group for a much smaller amount and filed for chapter 11 for the main holding company. This after going with a full IPO less than a year ago. No, visual effects is not a big profit center. ]


Most of the larger animation companies create their own content and receive the profits from how well the films do. Visual effects companies at times have a small percent of a film but most of the time the companies are doing work for hire.

Live Action Movies:
The blame for high priced movies seems to be placed directly on visual effects by some people but lets look at non-visual effects films.
Why are movies so expensive to make even without visual effects?

That’s a common question for laypeople. Why should it costs millions to make a movie let alone $100 million or more? How long can it possibly take to shoot a 2 hour movie? And isn’t it all done by a handful of movie people?

Time:
A movie typically takes a year to make from green light (approved funding) to theaters. That’s after possibly spending years in development. There’s a pre-production stage (building sets, casting, working on the script, finding locations, etc) that can be a few months, production (filming) can take 40 to 100+ shooting days. A 100 shooting day schedule is 20 weeks. The shoot days are the most expensive days of the film because there’s a large crew working so the more shoot days the more expensive the film, all things being equal. And time is the gold standard for shooting such that production is guiding the director to do it as quickly as possible and to remain on schedule. Once shooting is finished the film goes into post-production. This is when editing happens along with sound mixing, composing, recording of the music and this is when most of the visual effects work is done. This is also the most likely compressed stage since the movie has already been booked into theaters and that date cannot be changed. So studios tend to provide shorter post-production schedules than in the past. sometimes this requires the director to edit and lock sequences even before the filming is finished just so there is enough time to do the visual effects.


Costs:
Even non-visual effects films can be very expensive. Ignore high actor salaries and marketing costs. Even looking at just below the line costs (crew) on a large production. Camera crew, grips, electrics, wardrobe, makeup,  audio crew, etc. And consider all the people not on the set that are required to make a movie - art dept. with carpenters, painters and others making sets, orchestra, sound mixers, sound effects, etc. Watch the credits of a large non visual effects film and see how many people are listed. Those are skilled, experienced, talented people being paid a reasonable rate for the services they provide in this freelance world. That’s hundreds of people. It’s as if you started a company, staffed it up and ran it for a year before any revenues appear. That’s expensive.

Labor:
Few people actually sit down and consider the enormous cost of labor.  I’ll be making up some numbers so don’t take these as actuals or even averages. Let’s suppose we had 300 people working on a project at an average rate of $80,000 a year. The average income in the US in theory is around $30,000-$55,000 a year but most in the film industry don’t work full time over the year and this also has to take into account overtime which most jobs don’t have so don’t assume even someone making $80,000 rate ends up with $80,000 at the end of the year.  That’s 300 people at $320 a day = $96,000 a day in expenses just for labor. Now you can adjust the number of people or the income either way but the fact is labor is more expensive than most people realize. Even in this example that’s $480,000 a week (almost 1/2 million dollars) A 6 month project with these people will rack up $12 million just in labor costs. It would be $15 million if the average rate was $100,000. Add all the extras costs (building, supplies, support, rentals, etc) and adjust for overtime and work required. In the case of a movie add in the high priced talent, directors and other above the line costs along with transportation, locations, equipment and other costs as well as marketing.

Take a look at the company you work for. How many people work there and what’s their average pay? This will help you calculate how much money needs to be coming in just to break even on labor alone. Don’t forget most workers get benefits so the cost to the company per person is above and beyond just their rates.

The visual effects work:
Each visual effects shot (5-8 seconds typically) is unique. It's the same as having to do a complete new setup in live action which requires changing the camera position and the lighting. In many cases even shots can be equivalent to the doing a full 'company' move where the live action crew has to travel and setup at a new location. a time consuming and expensive process.

In a previous article I noted that shots are like snowflakes since no two are identical.  I compared them to oil paintings. Another analogy is to consider the visual effects crew is constructing a building based on nothing but a sketch on the back of a napkin and the specifics of where walls go and what carpet is selected for each different room is in constant flux. Creativity combined with technically challenging work, fixed bids, changes and deadlines does not make the process easy. I don't think there is another industry like visual effects that has this same type of business model.

Visual Effects expenses:
Beyond what most businesses have as basic expenses (buildings, desks, basic services, etc) visual effects companies have to provide at least one high end computer per worker. These tend to be top of the line fast computers with a lot of memory, large hard disks and advanced graphics cards and graphics tablets. Each needs  to be loaded with high end graphics software. In addition there’s typically at least one large room with racks and racks of computers for rendering the images you see. These are the render farms and usually have special power and air conditioning needs that make them very expensive. All of these computer systems require advanced and costly wiring along with an IT department to support it all. Hundreds or thousands of shots at 24 high resolution images a second times dozens of elements equates to a lot of storage space that needs to be maintained and archived to avoid losing critical work.

Visual effects companies have screening rooms and editors with editing systems so they can be synced to the production. They have special security systems as mandated by the MPAA to keep all studio materials under safeguards. Many companies have at least a small insert stage to shoot elements and odds and ends as required. A visual effects company may also have a motion capture stage, 3D scanners and model shop or have to sub-contract this type of work when required.

Hidden crew:
But even with these expenses the labor is the largest cost for doing visual effects. This labor is the hidden crew. Most directors and producers seldom see much of the visual effects crew because these crews work away from the studio. Even for those directors that visit the  companies they likely only see part of the crew and since most visual effects these days are spread out over multiple companies (shorter time schedules) the work is being done all over the world (thanks in part to tax incentives).

On a live action set most of the crew is standing there, ready to work. It’s obvious that decisions need to be made to keep these people productive and there is a schedule for what needs to be accomplished everyday. Since most of the visual effects crew is out of sight, they’re out of mind. There’s a disconnect to making decisions and changes and what that relates to in terms of costs and time. On a set if the director turns the other way and wants an elaborate set built the cost and crew time is evident to all. In many cases the notion may be dismissed as not being worth it. A director is constantly making decisions and makes a number of compromises at all steps of production. Should the director shoot another 10 takes of the actor to try to get a better take? Should they try a totally different angle or camera move? Make adjustments to the set? At some point the director has to move on if they want to make their day and complete the film in the number of shooting days budgeted.

Disconnect:
But none of that is evident when working with visual effects. Adding a 100 shots or doing 100 takes of an animation only takes a request. What impact it has on the schedule and costs is seldom considered. Because of the competitive nature of visual effects and the fact that the number of potential clients (the studios) is less than a dozen, companies forego many change orders, thereby lowering the price of doing the actual visual effects work even more.

And this is another disconnect. Just about everyone else working on a production is  employed by the production company directly. Should they be required to work more days or hours they tend to be paid. And crew members are union members except for visual effects workers. Visual effects people are primarily employed by a visual effects companies and not the production companies. Most are freelance and have to switch projects to keep working. Artists may or may not be paid overtime. They also may or may not get benefits. All depends on the company and location.

Color correcting, making film prints and other services are done at a lab. Sound mixing is done at a sound company. These types of companies typically work on a time and material basis. If the director wishes to spend another day making adjustments then that cost is obvious and billed accordingly. A visual effects company bids on a fixed bid even with partial information. Even with accepted change orders this tends to whittle down any planned profits.

Visual effects crew size:
The number of people working on visual effects varies with the scope of the work. It's not unusual for a major visual effects film to require hundreds or more people. On larger projects the size of the visual effects can easily eclipse the size of the rest of the crew. Next time you're at a large visual effects film sit through the entire credit list. There are a lot of people listed under visual effects. And usually there will be quite a few companies listed. Note that you won't actually see the full list of all visual effects people who worked on a film. The end credits is usually a partial list since the studios only allow so many credits to each company depending on the contracts. There may be people from visual effects not listed that worked for a year but it's likely someone who was an assistant to an assistant getting coffee for a week on the set is in the credits.

What do all of these people do:
(I'll have a post in the future listing the various positions. The Visual Effects Society lists over 200 job titles) [Update: Here's that post that sits many, but not all, key positions. In some cases there may be a few dozen people in the same position, working on different shots. ]

A visual effects company is like a mini-studio with a wide range of artists and craftspeople.  In pre-production artists are drawing and designing, modelers are sculpting every set, actor, prop and location required to be rendered. Everything is built from scratch. Every computer graphics actor or creature needs a skeleton and skin that moves correctly. Texture painters are painting everything down to the finger nail or leaf. Specialists are focused on how the clothing moves.

If the project was shot on film a small team will be going through every frame and painting out the dirt. Yes, there are a number of jobs in visual effects that require working on every frame. By hand. Others are hand tracing actors in specific scenes so new backgrounds can be replaced. Anywhere something had to be removed (stunt rig, etc) needs someone to hand paint it out. Remember that film is running at 24 images a second. Visual effects on a 2 hour movie could be  a lot of frames (172,800 frames). Most movies probably around 2000 shots - visual effects films likely have at least 500 or more shots. In some cases every shot in the film is worked on.

Match movers have to create a computer camera that exactly matches what the real camera did. Animators have to animate any computer graphics actors or creatures. Lighters and others will have to light scenes just like a cinematographer. All of these images have to be calculated and rendered. And then compositors work to combine all of these images into the final shot to be used in the movie. In addition visual effects requires a supervisor, producer, editor and numerous other support people.

Visual effects time:
Visual effects is one of the first departments involved on a project in pre-production  (or should be ) and continues until the end of post-production. This can be a year or more on large projects. Remember that a large animation film probably averages 3 years. Post-production time is frequently being squeezed into 4 to 9 months. Because post-production time is less than it used to be most of the crews start working overtime from the start. The films release date is locked so it's not unusual for the last few weeks or even months require working 80+ hour work weeks. 24 hour work days are not unheard of.

Computers:
The question of computers comes up as well. Don't computers do all the work anyway? NO. A camera doesn't film a scene by itself and a word processor doesn't write a script. The computer is simply a tool. And no, there is no magic button on the keyboard to do the work.

Computers are getting faster so why aren't visual effects a lot cheaper and faster to make? Just as a faster computer doesn't allow a writer to now create a novel in a day it doesn't allow a visual artist to do a shot in a shorter time. Much of the time is spent thinking, planning and working. Any speed ups or reduction in costs due to improvements of techniques and people is eclipsed by the next project since it requires more shots that are even more complex in a shorter period of time. Many movies are about taking it up to the next notch for the audience. That next notch absorbs any gains. Newer requirements also demand extra time and labor - 3D, 4k resolution, and/or higher frame rates.

Base cost:
All of this totals up to a  large price to have a very large crew of very talented, skilled and experienced people working long hours over this length of time. Consider  that the companies had to provide a fixed bid based on how many people and how much time was required, simply based on storyboards and some previs. (rough animations) They also have to estimate how much a new and never before seen effect for the film will cost. Calculated in these budgets is the assumption of how many takes will be required by the director.

The reality:
If the company estimated 4 takes per shot it's just as likely the director will require 12 takes for every shot. And that's just the start of things that happen to the 'profits' of visual effects. On a location in a forest clearing the director may decide to replace the sky for the entire sequence. Seems a simple enough task given all the other work the companies are doing but now people have to try to separate the existing sky in every shot. Some of that may be possible using semi-automated techniques but more than likely it will require a team of rotoscopers (artists who trace to create mattes). Every leaf and branch needs to be traced and any actor (and their hair) have to be separated. Maybe a prop doesn't work. That's added to the list as well to be added or replaced later in visual effects. If a stunt or special effects action doesn't quite work or isn't as big as the director wanted, that's added to the visual effects shot list.

When projecting the dailies in a theater after the shoot is finished, the studio may notice they don't care for the makeup or may spot wig netting or other flaws in the footage. That's added to the list of visual effects shots. Boom mics and crew members in shots are also added to the list. Some shots that were originally planned to have no visual effects now require them based on the edit. Based on test screenings there may be a need for reshoots or new scenes to be shot. Since the sets may have been struck or the actors may not have time to go back the location, these are shot as greenscreens with the visual effects company responsible to add the backgrounds and match all the other footage.

There are times when visual effects are held to a different standard than live action. On a location if a stunt car rolls over for a couple of takes the director may accept it and move on. The shot works fine in the movie. It may not be exactly what the director had in mind but it tells the story. Since visual effects can control everything down to the pixel with precise adjustment the director and studio may wish to tweak and adjust down to the last minutia. If the stunt car was done as a visual effects shot it may be requested to make it roll 6 frames earlier and to roll 5 degrees more. On the 20th take it may be decided to hit a specific parking meter which needs to bend at a specific angle.

Most of this feedback and the changes requested are sent to the hidden visual effects crews. Since these people are seldom seen (with the exception of some people in Skype conferences) it's easy to lose sight of the labor costs being incurred daily. Also unseen is the amount of work and overtime being put in to making the visual effects.

The need to get all of the shots done by the finals deadline drives the pacing. It's easy to spend months on a small number of shots and tweak them and then end up rushing the last batch of shots. The quality of visual effects shots in most cases is directly proportional to the time allowed to finish and polish them. Rushed shots will have flaws and that's why companies sometimes push for a CBB (Could Be Better) shot status. It's not perfect but it can work in the movie. If time permits some or all of these will revisited but that's better than the alternative of tweaking shots the first batch of shots and having a week to finish half the movie.

Remember this all started with a fixed bid for a specific number of shots in a specific time frame. The deadline never changes so the companies now find themselves having to do many more visual effects shots with increased complexity all in the same time frame. In some cases there may be clear changes that enable submitting change orders. But in many cases such as increased noodling and takes, it's a gray area. That tends to erode much of the profits the company was able to build into the original bid.

The amount of work flowing into a visual effects company can also fluctuate a lot. They may finish a very large project and then have a few months with no or little work. Many of the crew are laid off but there's still a need to keep a core team along with paying for all the overhead costs. Any profits made on projects will have to pay for these lulls in production.

Extra costs:
But that's not the end of the story. Many decisions may be delayed. There's usually no one keeping the post-production moving in the manner that the live action was. Sequences aren't turned over while edits and re-edits are done. The studio and director may have conflicting ideas. Test screenings popup with little notice. Results of test screenings require more changes. Sequences are added. Shots and sequences that had been approved 2 months earlier need to be redone with major changes. In some cases the majority of the visual effects happen in the last month or two of the film. What had been scheduled for 6 months must now be done in a fraction of that time and it must be done well. The deadline remains fixed since it must be in theaters as scheduled. And it's not like they can just hire another 1000 people to help out at that stage and make up for the added work. No, the original team will have to put even more hours a day and work 7 days a week. The more hours worked the lower the productivity of the workers. Some work may be farmed out to other companies when possible as 911 emergency calls are made. But that comes with a price of time and money as those companies get up to speed.

To put this in perspective think how much it would cost a live action production to scrap a month of shooting and be required to build new sets and reshoot on new sets, all in the period of a week.

If the artists are compensated for the overtime then prices really start to skyrocket. And more than likely after this big push most of these artists will be laid off until the next rushed project comes through the door.

And that is why visual effects cost so much.



Related posts:
Visual Effects are inexpensive
The Miracle of Visual Effects
Visual Effects - The Big Picture
VFX Business Models
How VFX is perceived by at least one DP
People, not computers, create visual effects
Getting the most out of your VFX Budget

Visual effects service - The Big Picture

(Note: I’m on the board of the VES but all posting to this blog are mine own and do not represent the VES) (As always, I could be wrong about anything)

If we’re re-examining the current VFX situation we need to take a step back and look at the whole process.

I think most of us think of visual effects as a service. But is it?

History
Years ago the studios had their own visual effects department with people on staff and basic optical and animation equipment. When the studios closed those departments, labs and small optical companies took their place. I can remember looking over lab and vfx company price lists in the mid-70’s. A dissolve was so much per foot. A matte was so much per foot with minimum cost of X dollars. Some of the places had small insert stages they would rent for so much per day. If you needed something special they could give you a quote but the majority of the work was on a time and materials basis.

For larger vfx projects the productions themselves would set up a full department somewhere. This was the case with 2001, Logan’s Run, Close Encounters, Star Wars and other films. The production would lease a building and set it up from scratch with the people needed to run and operate it. For Close Encounters we were in an industrial building in Marina del Rey. They had to custom make the matte painting stands and other equipment as need be. They purchased or leased optical printers and an animation camera. Everyone working there was paid by the production. This is in fact how productions usually run. They become their own company that has a group of people that round up or build whatever is required for the film. In live action this would be the construction of the sets, special rigs, etc. Once those shows were done the one off facility they setup would usually be closed down and the crew laid off. In the case of ILM, since George Lucas had such success with Star Wars and thought he might like to do more things for himself and his friends he re-created it in Northern California. Apogee was formed by John Dykstra and others using the original ILM building and some of the same gear(?) used on Star Wars. Many of the Close Encounters people went to Universal Hartland where Universal setup up a facility to handle Buck Rogers and Battlestar Galactica.

When we formed Dream Quest most of the initial projects we did on a time and materials basis, especially if it was large. If someone wanted us to shoot motion control we charged for the stages and the crew for so much per week. This saved us in a number of cases where the director or the vfx supervisor they had hired made changes or threw out work because of a change.

Today if you’re working on a commercial the cost of a telecine is so much per hour and so much for tape, etc. The work on the Flame system or equivalent is on a per hour basis. They may provide a rough estimate but it’s always up to the client how much time is used. If the agency wants to tweak something all day that’s fine. They get billed for it and the video house doesn’t have to worry about making a budget.

Fixed Bids
Somewhere along the way the studios wanted a fixed bid on vfx work for feature films. Estimates were no longer good enough and the vfx companies would now have to stick to the budget. This changes a number of dynamics. In the eyes of some studios and directors the vfx people were no longer working directly for the production, they were working for the vfx company. The vfx crew became another step further from the film crew. Outsiders. Now there’s always the ping-pong of trying to please the director but not going over budget. Filing change orders and having discussions with the studios regarding the costs now became standard process. At times the vfx company can be pushed into a corner. The vfx supervisor was now that guy from the vfx company. The name of the vfx company became the main selling point. The vfx supervisor, not so much. If the client doesn’t like the first proposed company supe then another one at the company will be swapped out on a whim. The crew having to work overtime was now the vfx company’s problem. We still had crazy hours at times in the ‘old days’. On Star Trek: The Motion Picture I worked several weeks straight, 12 hrs a day, 7 days a week. (and a few 24hr days) But the studio knew it and actually visited the facility. Today, they’re now removed from those details.

So are there any other areas of film production that are completely farmed out to a 3rd party company besides VFX? All the other main leads tend to be hired directly. DP, Production Designer, Wardrobe, etc. Even though most Special Effects people have companies I believe most are hired as a team of people or at least paid on a time basis. Sound mixing or the DI? I assume these are on a time and material basis as well. The previs team is frequently brought in to work in the same office down the hall from the director. They typically bill by the man days or hours. Most of the set construction I see is done by a team of people working for production with special projects (cars, etc) farmed out.

When I think of a service I think of a dentist, a car shop where they work on your car or a plumber that comes to your house. In these cases they do work but don’t tend to produce anything. The costs are based on time and materials.

Custom manufacturing?
Should vfx be considered as custom manufacturing? We actually create something when we finish our work, whether it’s from scratch or a montage of material provided. That’s what the studios want, not the actual service part.

Here is where things get crazier. Each shot is unique like a snowflake. It’s own little world of issues, handwork and tweaks. You try like anything to make shots as consistent as possible and to be able to run them through the exact same process but it’s never full automated. For all the talk about computers in our business it’s still a very labor-intensive process. The number of people and the time required to do a shot from start to finish would astound most outsiders.

If you go to most manufactures and request custom work you will be required to make specific requirements in writing. (I.e. you want cabinet style 32 but in this specific color of blue. You want a custom cake that says Happy Birthday. It will be yellow cake with vanilla ice cream and chocolate frosting.) And that is what you will get. They seldom show you the work in progress or have your input at every single stage. The other thing is a custom manufacture will tell you when it will be done. They dictate the schedule. In the film business it’s the opposite of all of this. The studio specifies when the delivery will be. It’s almost always less than the time that would have been arrived at by a normal scheduling process for the facility.

On a VFX project you start with the script, which provides a wide-open interpretation of what the final visuals will look like. In pre-production the director hopefully approves concept art, does storyboards and ideally previs. While most previs lays a good foundation the number of nuances and changes required for the final shots can be enormous. The director usually wants something never seen before that will require a lot of R&D. Not just custom but a totally unknown look or process that needs to be invented. Just how much time and money will that take? The vfx companies have to provide a bid for all of this before the film is even shot. During shooting things will change. During post-production things will continue to change.

This is a creative process so there will be changes but think of it this way: The vfx company is making a 1000 custom oil paintings that technically have to be delivered on a hard date for a fixed price (at least initially). This process could costs in the 10’s of millions of dollars, make up half of the film budget and fill up half of the screen time. There are some rough thumbnails but not enough information to simply deliver the finished paintings. The director is involved at every step of the process for every single painting. In some cases, for every brush stroke. Some directors only want to see the final pieces. In these cases you can end up with ‘no, now that I see it I don’t’ want apples in the painting, I want pears’. So much for the time and effort to create the initial painting. If a director changes one painting that may change two dozen that are almost finished. Remember, the due date will not move, regardless of the changes. And of course shots are not paintings but moving images so time and motion presents another infinite number of possibilities.

How many other areas does the director really work in this much minutia? Normally when they’re working with Directors of Photography, Production Designers, etc they discuss and try to get in sync regarding the general look and style they want. The director may be asked about the color of the pillows on a set but at some point they pass on the taking care of the details to their key creatives. The director is unlikely to ask to change the 3rd brick from the right on the set or ask the DP to reduce a specific light by ½ stop. And yet at times it can be that way when working with visual effects.

With visual effects the director has unlimited control. Every pixel of every frame can be changed. If production has an on set stunt or action the director shoots what takes they feel are appropriate and will select one. The fact that the stuntman’s hand is raised a little doesn’t cause problems. The best take will be selected and production moves on. With the advent of digital visual effects that’s not the end of the story. What would have been fine previously in any movie is now something to be scrutinized and analyzed by the director, editor and studio. Now it may be an added shot for the vfx crew to fix that hand position. And while they’re working on the shot can they change that thing back there and that other thing over there? A shot with a jet may get a request to roll the jet another 3 degrees. Will the audience notice 3 degrees? Will it make it a better shot? Obviously if production paid and shot a real jet they would be unlikely to schedule another shoot day simply to get the jet to roll 3 degrees more.

On the set the director knows it will take a certain amount of time to make a change so they always have to balance that because time is their gold standard. They have so many days to shoot the show and have only 2 days scheduled for this set and need to shoot 20 setups a day. With vfx that time balance is thrown out the window. Most of the work is done after filming. The amount of time and effort to make the change is all hidden. It’s happening elsewhere by unseen people. It’s no longer the director or producers responsibility to complete this phase of production on time; it’s up to the vfx company. To add to this difficulty is the fact that the live action shoot can and does go over schedule. Problems during shooting may now require additional, unplanned work to be done by the vfx company. But the vfx company cannot go over schedule. They are the end of the road so every delay during shooting, every added fix, shot or change needs to happen by the deadline. That’s the finals date that was set before the vfx company even started bidding on the show. Not only does the vfx company have to do all the work they initially agreed to do in that time, they have to absorb most production issues that have accumulated and rippled down the pipeline since the pre-production began. Add into that mix the requirement by the studio to make last minute changes, possibly based on test screenings, possibly based on an idea of an executive.

Are there other non-film businesses setup like vfx companies in terms of the requirements and client involvement? That would be useful to look at and learn from. Unfortunately I can’t really think of anything on the scale or dealing with the same types of issues. Many construction projects are of course custom and involve a lot of money and people. However they have blueprints that have been signed off on. They have colors that were selected to paint the walls and the client has approved the carpet and the tiles. Sure there will be some changes but the majority of the work is usually very well specified. Any major changes will involve a change of completion date or will require client to pay a large fees to have it accelerated.

Summary
Visual effects is a very labor-intensive business. The labor is made up of dedicated and highly skilled and trained people. There’s the requirement to complete hundreds of works of unique, never before seen, art (shots), based on rudimentary starting points, that are constantly being scrutinized and changed. And this all has to be done for as much adherence to a fixed bid as possible and above all has to be finished on the deadline, - no ifs, ands or buts.

I do want to go on record that I support all the directors I work for and that I’m all for anything that can make a film better. All vfx artists want the best possible film. What I hope this posting will illustrate is just how complex this issue is. We have art, technology and commerce all colliding. The vfx companies are put in a tough situation and the vfx artists are put in a tough situation to try to balance this all out. The end result is any process or structure that will help balance this issue to create the best creative and to make it reasonable for the vfx artists will be a welcome relief.

Related posts:
Pass me a nail
Risk and subsidies
Oh, the mess we're in!

Previs thoughts

Just Misc Previs thoughts from a note I had sent someone months ago.

I think previs encompasses anything that is used to get a visual sense for the final product ahead of time. The majority today is 3D but video, stills, storyboards, film snippets, etc are all possible and useful.
For Star Wars and even some pictures today existing footage (from another film or stock) is seen as a placeholder and visual guide. For Phantom Menace and other projects temp actors have been shot, sometimes against greenscreen, to block in concepts and editing ideas. For some movies such as Amelia they used videos and stills from actual locations or sets to get a sense for how the scenes will work visually.

Misc thoughts

DON’T’T CHEAT!
When working in a 3D environment it is all too easy to make a change at the director’s wish. Scaling an object, moving the camera, placing objects, etc. Usually this is done to try to make the previs as exciting and interesting as possible but these add up to real production problems.

Examples:
Fantastic Four – A major sequence was to take place at the London Eye. The previs team decided to scale the London Eye to less than ½ of its actual size to make better compositions. For months we had been looking at these and all departments were basing their planning on these only to find out that it was all fictional. Since we were to shoot the real London Eye, create accurate 3D models for the final shots as well as potential miniatures that were the correct scale this was a huge problem. We now had to have the previs scramble and redo all of the shots. By now production was in a different country and another previs team had to do the work with a different software package.

Shots moving in the tunnel varied wildly in terms of camera speed, car speed and character speed. In a simple previs it looked right but impossible to make look right if it were to be shot like that.

For Van Helsing some of previs was done in a way that couldn’t be shot. In some cases the cameras were placed in areas they couldn’t possible be or doing something that defied physics. When the production crew is half way around the world 6 months looking at a previs on a laptop and the director discovers they can’t do the shot as planned, it’s very painful for everyone involved.

You don’t want to get to a location and find that you’ll need to jackhammer the road to place the camera where the previs was ‘shot’ from’

You also don’t want to find your lead actor is now somehow supposed to be 10 feet above the ground. On the location cheats are made (actor on apple box, shooting in a different direction, etc) but everyone there knows it’s a cheat and why it’s being done. When a cheat occurs in previs the previs artist may be the only one who knows it. It’s likely the director doesn’t even know that it’s taken place.

Purpose
Determining the purpose of the previs is critical. Make sure everyone, including the director is onboard.
We just need the previs to a certain point to understand how the sequence and shots work, the approximate action and timing, etc. Yet a director can easily spends weeks ‘directing’ the ‘actors’ in the previs. “No, he should smile here and then look toward the camera”.

Previs has a tendency to become like temp tracks to the sound track. Something that the director has been looking at so long that it is the only way the director sees the shots as being. On a non-prevised show the creative team may make full use of the here and now and compose the shots to their advantage. (some feature at the location, the light at that time of day, etc) On a prevised show the director may be unwilling to consider these alternates.


Speed
It’s difficult to get a full sense for speed in a previs. In early previs a car doing a drive by might have been just a colored rectangle moving against a simple background. The director of course wants it faster, faster. If the same scene were shot for real at the original speed it would have been fast enough. All the details of the car, backgrounds and motion blur would give the sense of speed. Even with today’s rendering there’s still some visual speed discrepancies.

Accuracy
When storyboards are done everyone understands these are the basic shot designs and placeholders. It is understood that the perspective and placement of the actual location will be different in real life. One problem with 3D previs, especially the more detailed they become, is everyone thinks this is the actual shot, even if the previs was done months before the actual location was chosen.

Design
A major potential problem is a lot of previs is started before any of the key creative team is hired. It may be just the director (or even just the producer or studio executive) and a team of previs artists. The amount of visual sense the director has can varied widely and the visual sense and experience of the particular previs artist can vary widely as well. On a non-previs show the director works closely with his DP, camera operator and others (production designer, VFX supv) to determine the best compositions. This is likely also based on blocking in the actors motions. Yet the previs may well lock the creative team into design decision that were made by the director and the previs artist in a vacuum. I’ve seen shows where the stunt coordinator was told that he’s to match the action in the previs. Imagine having someone like Jackie Chan being told that some previs person has already designed all the action and action shots and that all he needs to do is get his stunt team to do it that way.

A car stunt may require a special rig be placed and the stunt coordinator may know the best camera angles to capture it but the previs artist knows nothing of this and places the camera exactly where it’s going to be a problem. The director has now grown to love this and wants the stunt team to sort it out.

The same thing has started to happen with DPs but most of them have enough clout to stop it there and do it their way. So how worthwhile is the previs if it’s totally ignored? If the VFX has been budgeted based on the previs but the director, Dp, etc ignore it completely where does that put the budget and schedule?

If the previs is done ahead of time it’s difficult to talk the producers and studios into redoing the previs so the now hired DP, production designer, etc can be involved.

Moving Camera

Using a moving camera when filming live action for Visual Effects.
Locked off camera, Post Moves, Motion Control, 2D Motion Tracking, 3D Match Moving and Face Markers are all covered in this podcast relative to live action photography.
Approx 25 minutes.

Transcript
Today I'll be discussing the moving camera.

Locked Off
The simplest type of camera move is no move or the locked off shot. The camera is placed on a tripod or dolly and isn't moved. This makes it easier to add visual effects later or to do multiple elements of the same setup. Examples of this could include adding a matte painted house to the top of a hill. If you wanted to create a shot where part of the actor is removed later – such as a leg as in Forest Gump or most of the body as in the invisible man you would shoot the shot two times. Once with the actor and another time with no actors. This shot with out the actors is called a clean plate and is pretty common for many visual effects. Because the camera doesn't move you have identical images – one with the actor and one without. In post production if we remove part of the image of the actor the clean version gives the image without the actor. Special rigs are removed the same way. This same process is easily used for creating twins from one actor. Shoot the scene once with the actor on the left side and then shoot the same thing with the actor on the right side. Because these shots don't change position you can do a simple split down the middle of the scene.

Normally the camera operator is making slight pan and tilt adjustments while sitting on the dolly and the asst cameraman is making slight focus and exposure adjustments. To obtain the best quality of clean plate get everyone away from the camera, off the dolly and avoid changing the settings, even to do slates. The size and position in frame will be different if anything changes, including the exposure. You can spend time fixing this in post but it's better if you can avoid the problem.

Even though a locked off camera makes it easier to accomplish visual effects it may not fit with the look of the rest of the film or the requirements of the shot.

Post Move
Doing a move on an image in post is sometimes called Pan and Scan. This also can refer to transferring a widescreen film to full frame video.

The scene is photographed normally and then in the composite stage the image is enlarged and a synthetic move is added by moving the image digitally. It's also possible to do this type of move with some scanners for better quality.

The problem here is the loss of resolution from enlarging the image. If your end production is video then it's possible to scan the film at a higher resolution.

In the past another way around the resolution problem was to shoot on VistaVision or 65mm cameras. VistaVision a format which shoots with special cameras that run 35mm motion picture film sideways, much like a still film camera. This larger film size allowed for blowing up without as much quality loss in the days of optical printing. Large formats were also common when doing any effects heavy productions in the days of optical compositing. 2001 A Space Odyssey, Close Encounters and Blade Runner are some of the films that used 65mm. Star Wars and most ILM films used VistaVision for effects until the last few years

If the live action element only makes up a portion of the frame then a post move doesn't cause a quality loss. An example of this is starting on a scene which is all live action and then pulling back to reveal a large matte painting extension to the scene. In the old days a motion controlled camera might be used to film a physical matte painting on glass. A rear-projector would be one method of adding the live action with the same move. With the advent of digital matte painting the matte painter creates the painting at a higher resolution. If the original live action scan is 2000 pixels across, the matte painting might be done at 4000 pixels across if the final scene was going to show twice as wide.

I'll discuss 2D match-moving later in this podcast, which is a form of post move on an element.

One of the problems with post moves is the very limited 2D or two-dimensional appearance of the moves. There's no perspective change or feeling of depth. To obtain true 3D camera moves of the original live action there are a couple of processes.

Motion Control
One way to deal with this is using motion control. Motion control is used frequently with shooting miniatures and models but here I'll focus on live action use. Normally the camera is mounted on a pan and tilt mechanism known as a camera head. The operator has a hand wheel for pan and one for tilt. For motion control you have a special head and usually a special dolly where all of these motions are controlled by motors. In some cases the operator moves this just like a regular head and a computer records the positions by using position encoders. In other systems the operator uses a joystick or a remote camera head that is designed to just record his hand moves. In this case the remote head looks like a camera head but it's just a box with the hand wheels.

Note that non-recording remote camera heads are also with live action frequently when the operator can't be at the camera itself. The camera is on a boom arm or other remote system where the operator watches a video monitor and remotely controls the camera.

With true motion control once a camera move has been programmed the move may be repeated over and over again exactly in sync with the camera. The repeatability means you can shoot the same actor walking through the scene multiple times with a complex camera move. This would be done to create twins using one actor during a camera move. Motion control also allows you to film moving clean plates so you have many of the same benefits as a locked off camera such as rig or actor removal. Since the move is recorded you can also take this data and use it later in a motion control system for models or convert the data and use it in the computer for adding CG elements.

Motion control can also be used to shoot secondary elements such as bluescreens people or objects that exactly match a previous live action plate. In this case the data for the move comes from the original motion control shoot or from a process called match moving, which I'll be covering shortly.

The downside of motion control is the requirement for a special system that has to be setup and be programmed. Most directors hate it because of the extra time and process. I recommend it only when you really have to get repeatable motion on the set or location.

I'll cover more details of motion control in a future podcast.

MatchMoving
The other option when a moving camera is required is to do matchmoving in postproduction. This is now one of the most common techniques that came about with the advent of the computer. The simplest technique is shot with just a pan and tilt that requires another images to be added later in the composite. As an example if we pan and tilt an outdoor scene and wish to put the image of a flying saucer hovering we could matchmove a point near the horizon or infinity like a distant mountain or building.

This is done after the film images have been scanned into the computer. These days' most professional compositing programs allow you to specify an area in the moving footage that you want to track. Frame by frame the computer compares the image in this area and tries to find the best fit around where it was last. We call this 2D tracking since it's only calculated in 2 dimensions x and y.

Once this is done the image of the flying saucer is composited over the background and the motion of the background is applied to it. Now you have a scene with a pan and tilt of a cityscape and a flying saucer that hovers over a building no mater what move you make. This creates the illusion it was there the time of photography. If you have 2 distant points that you can track then it's possible for the computer to calculate the rotation and size so you can not only pan and tilt but roll the camera and even do some basic dolly or zoom moves. I suggest distant points to track so you can obtain a more accurate motion track without having to worry about parallax or perspective. You typically also want to track a point in the scene close to where you want to place the additional image. Make sure you always have at least one or 2 points in the scene to track during the entire scene. If you're doing a big pan move make sure there's another suitable point that you can switch the motion tracking to during the shot. Ideally these would be close to the same distance away from camera.

Even though this is a 2D track it's possible to obtain 3D pan and tilt information. By knowing the lens used to photograph the scene the computer can calculate and output a 3D camera file that can then be used in 3D program for the 3D camera to pan and tilt. This renders an object that matches the move exactly.

Another variation on the 2D tracker is a 4-corner track. If you have a billboard in a moving shot and want to replace it with a different image you could track each corner of billboard on the computer and tell it to apply the move to each corresponding corner of the new image. This will distort the new image to fit it in the billboard even if the original scene is moving.

3D matchmoving
The next level of match moving is 3D match moving.
When shooting a scene measurements are taken of any landmarks in the scene (windows, doors, tables, that sort of thing) and in some cases special markers are placed in the scene. Later after the film has been scanned into the computer a person called a matchmover will build a rough replica of the set or location in the computer. The virtual camera, which is a simulated camera in the computer graphics program, is now animated to match the move of the film camera that filmed the live action scene. Originally this was done as a manual process of lining up the CG set with the image from the film. Now much of it is done using special 3D motion tracking software.

Note that some measurements such as the focal length of the lens may be a bit different than marked. If it's labeled as a 50mm lens it could in fact be a 48mm or a 55mm lens. As such you may need to make some adjustments manually to get the final matchmove perfect. Also be sure to check any automated 3D matchmoves since it's possible to fool the computer.

Now that we have a CG camera that matches we can now place a CG object such as a box on the match move floor and it will look like it was on the floor in the original photography. It still has to be lit and composited into the scene but it stays locked with the correct position and perspective in the scene even if the camera operator is moving a hand held camera around the imaginary object.

If a CG creature will be walking on rough terrain hopefully the matchmover has been able to recreate that so the animator can always keep the creature on the ground. This terrain shape is likewise used to cast CG shadows onto. The matchmover has also included any large objects such as tables and trees so the animator can avoid those when moving his creature. If the actor in the scene is interacting with the creature then the matchmover moves a CG pawn that represents the actor. In some cases the matchmover has to do a very tight match of the arms and legs of the actor so the animator can time and match the creature to the actor. Even though I'm using the term creature this could be any 3D object or effect such as CG sparks.

So now that we have the basic of the matchmove process down lets take a look at how that works when we shoot. If you're going to be using 2D tracking then survey the scene and make sure you have some definable points. Any place of sharp contrast or a spot. The corner of a building against the sky provides a clearly defined area that is fixed. The edge of a tree blowing in the wind is not a good point since it's changing. The edge of a building isn't a good point because it needs to be one specific and unique point.

These same requirements apply to four-corner tracking. If you're replacing the image of an old TV you may find that the TV screen is rounded and almost blends into the frame of the TV. In this case you may want to put fluorescent dots from the office supply store on the corners of the area you want to track. You'll need to paint or composite out the dots in the final scene.

If tracking points don't exist in the scene and you need them then you have to create them. If you're shooting an actor in front of a blue screen to create a distant vista in post production then place markers on or in front of the bluescreen. These may be 2 to 6 inch plus marks made out of tape or plastic. If it's a cloth screen material you can put Velcro on the back of the plastic markers. If you can't touch the screen use c-stands to hold them but note that there will be more removal work to do in post. It's important for any markers to be solidly locked down so don't suspend with them with thin wires.

You want the markers to be visible on the final film scans but not too large or you'll be doing a lot of extra paint out. Same with the number of markers. You want to always have 2 or 3 markers visible even when you pan and tilt the camera. If you plan to pan the camera 180 degrees hen you'll need a number of markers around the camera. The actors or props may obscure some of the markers. If you're doing a full day of shooting in front of the bluescreen then it's best to just setup a grid pattern of the plus marks. Note that if the depth of field makes the markers way out of focus or if there are no markers visible in the shot it becomes a matter of experimenting in post to create a move on the background that seems to work with the foreground. This can be very time consuming and frustrating especially if your actor is jumping or moving.

If you're doing 3D motion tracking you always want to have at least 3 trackers visible. As always the matchmover will be recording the lens and tilt information from the camera, possibly with the help of the coordinator and certainly with the help of the camera assistant. If you're shooting on a set the matchmover should be getting measurements of anything with straight lines that are clearly defined such as the tabletop or windows. With this information they can build a rough CG version of the set. Set drawings are seldom used to build the CG matchmove version from since there may have been adjustments and changes made to the real set that aren't reflected in the blueprints. You want to make sure to build the CG world to what's actually captured on film, not on what was planned. To help with this work the matchmover usually shoot stills and especially Polaroid's where they can mark the actual dimensions on the image. These and the notes will be used months from now when the visual effects are being started. In some cases stereo images are taken or multiple cameras are used to help document the relationship of the objects in the set.

For organic sets such as caves or outdoors in natural landscape small ball markers are typically used. It's important for these markers show up so they are usually ping-pong balls or tennis balls painted a fluorescent color. Bright LEDs can also be placed in ping-pong balls. This is especially useful in darker sets or night shoots. At times a box frame of known size is filmed in the scene as well to provide a defined object and perspective check.

If the scene is supposed to be a close-up of a creature but there is no fixed object in frame then vertical rods may be used that hold a marker in place. As always there should be at least 3 markers.

If it's a large exterior scene on uneven ground a grid of markers may be set down or at least measured. Now the animator will be able to match the creature feet directly to the real ground. On DragonHeart and most of the shows afterward when filming in a large natural setting we used a surveyors transit system connected to a powerbook or a handheld computer that records the true position in 3D space of xy and z.

The matchmover back at the studio creates a 3D ball in the computer matching those coordinates. In the past the computer camera would be moved manually to match the position. Now there are special 3D tracking programs that can do this automatically.

Face Markers
Sometimes we're required to create CG prosthetics. This is done for shots that can't be done as normal makeup work since it's going to change during the shot or it requires removal of sections of the real actor. This is the case with the jaw extensions in Van Helsing and the facial electronics in something like the latest Terminator movie. Small colored dots from an office supply store can be applied to a face or arm. It's also possible for the makeup person to apply makeup marks. In these cases it's important to know where to place the dots to get the best match. Because of muscle changes you typically want some on the cheekbones or area that won't change as much in addition to the edge of the imaginary prosthetic. The animator, model maker and/or matchmover will then have to change to shape and position of the CG prosthetic to match frame by frame.

Note that matchmoving doesn't solve the problem of repeating live action camera moves such as required for twin shots and doesn't provide us with a clean plate. It is possible to take the match move data and convert it into a move for a motion control systems later when you're shooting live action or miniature elements. If you need a clean plate (for paint restoring) then the operator tries to repeat his same movement as accurately as he can without the actors and then the compositor will have to massage it in post to try to make it work.

Low Budget
Most beginning filmmakers want to make as complex scene as possible but it's important to keep it simple and to learn the basics of the craft before overwhelming yourself.

As mentioned before the simplest process is the locked off camera. Attach you camera to a solid tripod and film your different elements, including your clean plate as needed, without touching the camera or tripod. This allows you to focus on the art of compositing without getting tied up in the difficulties of match moving.

If you're creating CG elements measure the set or area and try to create a replica of this. Use real units of measure in your computer graphics program. Measure your camera as well. This would include the height from the ground and the distance from camera. Technically you want the nodal point of the lens but middle of lens or film plane is usually accurate enough to start with.

You also want to record the tilt of the camera. You can use an inclinometer to measure this tilt. You can find these on the internet or at larger hardware stores. You don't need the fancy, all digital versions. A simple type based on a bubble level is fine. These are 3 or 6 inch vertical disks with a flat bottom. A needle always points up to provide a readout the number of degrees of tilt, Place it on something level to the film plane, such as the tripod head plate.

If you can find one of these or they're too expensive then get a simple 6 inch protractor. Tie a knot in the end of a one-foot piece of string. Thread this through the hole in the protractor from the back. Tie a weight such as a large nut or bolt to the other end. Place the protractor upside down with the straight edge of the protractor resting against the straight edge of the tripod head. Read the degrees where the string passes the protractor. You may find in your CG program that you have to add or subtract 90 degrees from this number or you may have to negate the number. There's no absolute position for pan so we don't bother to measure it.

Place your CG camera at this point and use the set you built be your guide to where to place the CG creature or object. You also want to be able to cast shadows from your creature onto this ground plane and any large objects such as tables.

Once you've done these types of shots then you can experiment with 2D motion tracking and 3D matchmoving. Start the 3D matchmoving by using just pan and tilt when shooting.

Filming

Filming live action for visual effects on feature films. I discuss the process of shooting on locations and sets, using references, creating interactions and things to watch for. In addition I give some suggestions of to apply this to low budget filmmaking.
I'll probably discuss moving cameras and matchmoving in the next podcast. Filming of bluescreen, minatures and other elements will be covered in future podcasts.

Transcript
In today's podcast I'll be covering the filming process relative to visual effects. First I'll focus on how it's done on a feature film and at the end I'll provide additional suggestions for filmmakers. I've actually split up this podcast so moving cameras will be discussed in the next podcast.

There are infinite possibilities when shooting a film, which is one of the reasons why visual effects are interesting. I'll be covering the basics of shooting with actors or what we term as live action. Later podcasts will discuss shooting bluescreen and miniatures.

First a review of a few terms I'll be using.
CG stands for computer graphics.
An element is an image that will be part of a composite.
A plate is a live action element.
A clean plate is a version of the shot without actors that can be used to remove any unwanted items from the real shot.

The shooting of a feature film can take two to six months. Much of the shooting depends on the project. Some projects such as Dragonheart are shot almost all outdoors, regardless of weather. This makes it like a camping trip with 2 or 300 hundred other people. Other projects may be primarily on sound stages in front of bluescreens. Most projects are a balance of exterior and stage shooting.

Prepping for the shoot
Before shooting begins it's important that the cameras are checked and prepped. This is handled by the camera assistant but for visual effects we request a steady test. When film goes through a movie camera the camera movement may cause the film to shift a bit from frame to frame. This isn't visible in a typical shot projected on the movie screen but if you composite multiple images you may well see them moving against each other. In some older movies when you watch the titles you may see them shaking against the background. This was because the original camera wasn't completely steady.
To test the steadiness a grid of white lines is applied to a black backing. This can be tape lines on a 4 by 8 foot black. This is filmed with the camera fixed on a tripod or pedestal. Depending on the test the film can be rewound and re-exposed to the same grid offset halfway by a grid space. When this is processed and projected you can see if the camera is steady and repeatable to itself. The preferred steady test is to scan the grid from the camera with your film input scanner and confirm that it's not moving relative to your control system, the input scanner.

Film live action for visual effects

The visual effects crew directly involved with the shooting is fairly small. Normally the team consists of the visual effects supervisor or a plate supervisor, 1 to 4 matchmovers, a coordinator and possibly the effects producer as well. If the show is heavy with animated creatures the animation supervisor may also be part of the team. The remainder of the visual effects crew is back at a facility working on creating CG and real models along with preparing for the full render and composite processes. The full effects crew won't start until a sequence has been shot and edited since they need to work with the footage. Because of the deadlines most shows are editing simultaneously with the shooting so finished edited sequence will be done even before the entire movie is finished being shot.

The visual effects supervisor or plate supervisor is in charge of making sure the required footage is shot correctly to do the effects later. The plate supervisor is called that since live action pieces or elements are frequently referred to as plates. A background plate for a bluescreen would be called BG plate. The matchmover position was created primarily in the digital age. It's important to be able reproduce the camera and objects exactly and that means recording all the camera information such as lens, tilt and also to measure specific items in the scene. The coordinator helps to organize all of this and to facilitate passing of information. The effects producer is normally busy at the effects house overseeing that process but the production itself may have their own effects producer on the location to help the different departments and to make sure things are moving smoothly.

Before the photography begins there's usually some pre-production at the location. Part of this is a key meeting with all the department heads, including visual effects, so everyone is clear on the requirements of each sequences and who's doing what. It's also a chance to flag any problems. Typically the storyboards and animatics are shown to the department heads and the actors so they're aware of what the final shots will look like.

Each day there is a call sheet passed out to all crew members that lists the shots/sequences for the next day as well as when each crew member is required on set. A shooting day is usually 7am to 7pm and night shooting is 6 or 7 pm to 7 am. Shooting is 5 or 6 days a week.

The setup
First thing in the morning is brief huddle of key personnel with the first asst director and director. The director has his shot list, which is the list of all shots he plans to shoot that day. For shots that require visual effects the first task is to figure out the camera position and blocking. A director may run through with the actors first to get a handle on how he wants the scene covered. For camera placement it's important to consider anything that will be added later in postproduction. I typically have a set of storyboards that have been reduced to half size so it fits in my side bag or jacket pocket. I've gone through and made notes about every element or piece of film we have to shoot for each shot and have a technique in mind for each shot. We do a quick review of the storyboard. Since the storyboard is only a guide the director or circumstances may require a change. As a visual effects supervisor I have to be able to quickly determine any implications these changes might mean. Will other elements be required? Does this change the technique? Have we already shot something else that this needs to work with? Will there be any major cost differences?

For Dragonheart I wrote some software for a powerbook that would display the dragon in the correct size on a location still, but we found it faster to take the poseable dragon model in front of the camera. With the actor standing in the scene and the physical model of the dragon we can easily check the composition and framing. I'd have a matchmover measure the distance to the actor and would calculate the scale distance for the model. If the actor was 50 feet away it might mean the model should be 23 and ½ inches away. Now the director, DP and myself would review the framing. Once the initial framing is setup we review the specific storyboard and animatic with the camera operator and actors along with key people. The practical effects and stunt crews start any rigging necessary while the director of photography lights the scene.

To help the actors and the camera operator interact with a creature we try to provide a stand in. For Dragonheart we had a monster stick that was an expandable pole with 2 disks on each side representing the eyes. For something like Mr. Hyde on Van Helsing a 2d foam core cutout was attached to a helmet worn by an actor for reference. Roger Rabbit used 2d cutouts and full-size 3d practical model. If the creature is moving someone moves the stand-in around. The actors can use the eyes as a guide where to look and react and the camera operator can make sure they provide enough room in the film frame for the creature. At least one reference is shot with this stand-in moving around. This can be used as a temp in the edit stage since the action and framing is clearer with the reference. We try to do the majority of takes without the reference to avoid having to do paint out removal of it in post. In some cases though the complexity of the action may require the stand-in be in all the takes.

Interaction
Any time you're combining multiple images you want them to blend together to make them appear to be a real scene or location. To help with this illusion we try to provide interaction of the different elements such as a creature that lifts up the actor and then swings it's tale to knock over a bookcase. This is where the special effects crew gets involved. As mentioned before technically special effects in movie credits are practical or mechanical type of effects. Normally this will have already been discussed with the special effects supervisor who will have planned and constructed some type of rig to do this. If it involves moving the actor or a stunt person then the stunt coordinator is involved. New ideas from the director at the time of the shoot require everyone to scramble to make it happen. If it was preplanned and there is an animatic we use that as a starting point for the timing of the action. If there is no animatic the animation supervisor or the visual effects supervisor is involved in working out the timing with the director. We need enough time for the actor and the imaginary creature or object to react. The asst director will usually provide a verbal cue so the special effects crew can do their part at the right time. If the creature is supposed to speak then someone on the set will be a stand-in for the creature by speaking off camera or this may come from a pre-taped voice. An actual person on the set delivering the dialog is preferred since the timing and dialog can change easily.
Another form of interaction is the lighting from the director of photography. If colored objects are supposed to emerge from a glass then the DP has to create an interactive light on the faces of the actors so when the objects are added later by the visual effects crew they look like they're creating the light effect on the faces.

Rolling film
Each shot and take is noted on the camera report and slated. For visual effects shots there are usually specific shot names or ID codes that relate to the sequence. These names may have been defined months before in pre-production and relates to specific storyboards.
The first take is usually done with any interactive reference material as monster sticks or a stand-in. After that the shots may be done without these depending on the specific requirements.
During each take the visual effects supervisor is making sure everything is working as required. Is the actor's eye line correct? Is he looking for where the creature will be in that specific moment? Has he missed a mark and put his arm through where the creature is supposed to be? Are there additional items that may need to be removed? Will the special effects rigging work and will everything look correct once the shot is finished in post? Are any cables causing a bunching up on the costumes that give away the fact the actor is on wires? Is the camera move correct and timed right? There are a hundred items to keep an eye on.

The actor's eye line becomes even more difficult when there are multiple actors. If you have a lot of extras it becomes very difficult. If you have a small object flying and hovering the illusion won't work if everyone is looking in a different direction. So to handle this you do the run through with the reference and may need to shout out timings so to choreograph this eye motion. You may need to work with the special effects people to create a small target that can be moved around that people watch. For Dragonheart we used an ultralight plane for some shots where the dragon is flying. This provides everyone with a clear idea where to look and the plane itself is removed in post.

Multiple video cameras may be mounted on tripods to provide additional reference for the animators and match movers.
I try to list on my storyboard book any additional items that may be critical to keep an eye on in a specific shot. I've also listed the additional elements that may be required.
Most of this checking involves looking at a video monitor coming from the video camera mounted in the film camera. This is known as a tap camera and unfortunately the quality of the tap cameras is about the same as surveillance cameras. The video monitor itself is at what's known as the video village. There's an operator who runs this and records video of the different takes for reference. It's around this small monitor or two that the director, script supervisor and other key people may be grouped around during a take.

If there are changes to be made the effects supervisor discusses this with the director. Normally you let the director be the one giving directions to the actors to avoid confusion.
Once the director is satisfied with the takes for the main action any additional elements are filmed. This may be the clean plate as previously discussed. This could be some practical effects such as dirt hit or an additional actor. For things like dust hits a black flat may be placed behind the dust and this element would be screened or lumikey in the final composite. If it an additional actor this may be shot against a small portable bluescreen on the location. One of the advantages of shooting these elements on the location or set is the lighting will match and camera position will match exactly. Trying to re-create sunlight months later on a stage and have it match is very difficult. The downside is this will take up more time when shooting the live action.

Once any additional elements are shot references are shot. For 3D work normally a ½ of a gray sphere is shot. This will provide a controlled reference for the technical director who lights the CG scene later. It shows where the light is coming from and the basic balance of the lights. A ½ of a chrome sphere, which may be the other side of the gray sphere, is filmed. This will provide an image for the reflection map that is wrapped around the image. This provides some of the ambient illumination as well as image for the reflection. Sometimes stills are taken with a fish eye lens to provide this same map.
A color card or grayscale may be filmed to provide a color reference. I usually try to have a reference of the object that will be added later. If we're adding a CG version of a clock such as in The Mask I'll move that through the action so the technical director, also known as TDs, can use as a guide for how the material and lighting interact. If there's a creature the model shop may provide a reference material such as section of fur if this is a creature with fur. By trying to have as many real world references as possible the final results will be based on what it would really look like in that environment. Even with fantastical creatures the aim is to make them photo real. It's actually more important to make them photo real since the eye and mind knows it's not real it tries to find any discrepancies.

So finally we have one shot done. There may be another 20 more to do that day, each with their own concerns and rigging. On a large show they may be running 2 cameras all the time to provide more angles and coverage. For complex action scenes they may be running 4 or more cameras at the same time. Any of these shots that will require visual effects will need to be watched and measured. A large film usually has second unit crew shooting additional scenes or pickups and inserts at a different location. Each of these shots has to be dealt with the same way if there are any visual effects.

On a large effects film there comes a time when the crew along with the director and assistant director start assuming that you can fix everything and they may become a bit sloppy about removing things from the scene or making final adjustments or providing you what you need. There is always pressure to keep moving on a film shoot and in some cases it will be cheaper for you to do something in post rather than taking an hour at a cost of tens of thousands of dollars to do it on location. The visual effects supervisor has to weigh these two issues and choose his battles wisely. If the quality will suffer because of production shortcuts then it's critical to flag the director and discuss the issue.

Low Budget
So how can we apply all of this to a low budget filmmaker? As always planning and preproduction are vital for keeping production moving smoothly and keeping the costs down. Make sure the effects are there to tell the story.

Try to do a rough storyboard of all your effects shots. Don't get wrapped up in created elaborate animatics or 3D storyboards. Usually the added value of these, especially for simple projects, is minimal but the time required can be enormous. It's better to put your additional time and energy into the real shot.

Analyze the different elements required to make the final shots. Remember to keep it simple. If you can do it using 2 elements instead of 5 do it with 2. List those elements with the storyboards.

Keep the storyboards with you while shooting and check off the elements as you shoot them.

Do tests ahead of time to check your technique and to determine if there are other requirements when shooting.

If this is a really low budget film you may be directing and running the camera in addition to doing the effects. This makes it even more critical to have a checklist and make sure everything is shot correctly.

Communicate with your crew so they understand what you need done. Make sure the actors understand what's required of them. The storyboards will help here.

Don't rush the shoot. If you don't get what you need or get poor quality elements then you're likely to spend a lot of time in postproduction just to create something marginal.

Be prepared. Have your tools with you. Storyboards, notepad, pens, tape measure, etc. A fanny pack or other bag is useful to hold these items.

Slate and label everything. Try being organized. When you're doing effects work you may end up with several elements for each shot. It's very easy to get overwhelmed but al l the different variations. On a feature film we have people who's focus is to keep track of all these bits.

Double-check your camera settings. If you review the footage on your video camera be sure to cue it up to after the last take. This is to avoid recording over a shot and to make sure the timecode is correct if your camera supports timecode.

Just because you can do something in post doesn't mean you should. As an example if someone left a c-stand in the shot you could paint that out later but why bother when you can just take 5 minutes and move it out before shooting. Don't try to fix everything in post. Balance the time on the location with the amount of work required and the final quality.

Keep the shots simple, especially if this is one of your first projects. Even when on set make sure you're not making things too complicated. When we were filming the The Mask we had planned a wide shot where large props are being pulled out of Jim Carrey's pants. Since this included items such as a tuba and bazooka and we saw his entire body we were going to make CG pants and stretch them to show the objects being pulled out. On the night of the shoot the director changed it to a cowboy framing, which means the bottom of the frame, is between the knees and waist. Jim's costume was a very baggy zoot suite so the stretching pants gag became un-necessary with this framing and costume. I suggested we cut the pants to make them into shorts and cut holes in the pockets. We then had two people shoving real props up into the pant pockets.
We eliminated an effects shot and had something that was better for that shot.

If you're shooting on film, especially 16mm, do steady tests before shooting or your images may jump against each other. If you're shooting film but will be finishing on video make sure the telecine is done with a solid movement. In the past you had to request a special pin registered transfer but film movements have gotten better. Just check with the video house and make sure to steady test it.

If you're shooting video try to shoot progressive if you can with a camera like the Panasonic DVX100. You can shoot visual effects with a standard video camera but the interlacing makes the process a little more difficult. Likewise if you're planning to shoot video and doing a lot of greenscreen work consider shooting on a higher end system than miniDV. You can do greenscreen with miniDV but a higher end system will provide better quality easier.

Shoot your shots as locked down cameras if possible. This will simplify most effects a lot.

Try to shoot a clean plate when possible. To make sure there's a good match you can just keep the cameras rolling and have the actors leave the scene. This avoids the chance of the camera changing or the lighting changing as the sun moves.

If you're going to purchase a composting software try to get one with motion tracking. If you're in school check on educational pricing for all software. And certainly take advantage of cameras that may be available through your school.

High quality visual effects takes a lot of time and work. Even with all the tools available nothing is as simple and easy as you would think it would be. Accept that and keep moving forward.