VFX Interactions

Interactions
Most visual effects are designed to integrate with live action images. Whether you're creating a cg creature, cg background, physical model addition or even 2 live action elements, the intent is to match these elements together so they look as if they were photographed together. That means matching lighting, camera angles, color and other aspects.

But visual effects are more than still images. To help the illusion of integration even more you should look to create interactions whenever it's appropriate and possible.

Interaction and the special effects crew
Special Effects are the things done in front of camera such as fire, break away glass, etc. They're sometimes refered to as practical effects or mechanical effects. Interaction should be planned in pre-production as the shots are being designed (storyboards, previs).  The vfx supervisor will work with the special effects supervisor to determine how interaction can be done during filming. In some cases it may be as simple as a fishing line (monofilament) used to move a hat that a cg creature is supposed to pickup.  Or a gust of wind from a fan on set. In other cases special effects or props may have to build something elaborate which will require time.  This is why it's best to discuss in pre-production rather than waiting until you're on the set.  There will be times when you will have to throw something together on the set but preplanning provides the most options and most likelihood of success.

Any rig or setup should be tested ahead of time with enough time to make corrections if necessary. If it doesn’t work on the day of shooting for whatever reason you'll be stuck with it or will have to abandon it.

Keep in mind what you're trying to simulate and how that should look. Reviewing related references ahead of time is always good. On TIME MACHINE we had a morlock who was supposed to crash into a stand of bamboo trees.  The special effects team had rigged some of the trees to simulate it the way they thought it should work. The bamboo trees had been pre-scored (partially cut) and then fell over, cut in mid section. Not exactly what a large creature crashing into them would do to the trees. There was another test with cables to each tree but we worked with them and they build a simple metal plow that would be pulled by a winch that literally crashed into and through the bamboo.  Now we had something that would match what the animation would require. We would have to paint out the rig where the creature didn't cover it but that was much better for the overall look of the shot.

Always keep an eye on the interaction and imagine the final results. If the on set action is wrong (it's at the wrong height, it's moving too quickly, the rig is shaking, etc) then you need to correct it then and there.  Trying to force an animated character into contortions simply to match what was shot will not make a good shot. If there are major problems you may have to skip the interaction or may have to do it all as post effects.

For DRAGONHEART we had a number special effects rigs for interaction. Special effects rigged up multiple air canons to spray the water when the dragon bursts through the waterfall. This is when your prep work in pre-production comes in handy because you can provide them with the size and placement necessary of the creature or virtual set to be added. For the shot of the dragon tail going into waterfall we had special effects build a full size black version of the tail. This was pulled through and once the cg tail was added later the water traced the contour and provided the interaction to help place the dragon into the world on screen.

For the scene at the end where the dragon is in chains we had special effects build a pipe rig that acted like a teeter-totter. Chains were draped accordingly over one end of the rig, which was sized to match the dragon’s horns. A special effects crewmember used the other end to move it slightly while Dennis Quaid removed the chains. Other chains on the dragon were cg.  By doing this we not only got the correct motion and look of the foreground chain, we also gave the actor an opportunity to act with his missing cg actor.

For the end of the film we see a spirit flowing by a group of people. In this case wanted to get interactive light on the face of the villagers. The camera and lighting crews worked with the special effects crew to build a translucent box with colored changing lights suspended on wires and moved by the crew.  We removed the light box in post and added a particle spirit to complete the scene.  Now the glowing cg image seemed to be illuminating the actors and provided the illusion it was really there.

There are many other examples on that film including requesting the special effects crew lay in the tall grass and make snow angels (move their arms and legs while laying down) to move the grass when the dragon lands. Special effects welded 3- 55 gallon drums together and we had this dropped from a helicopter to simulate the dragon diving into the lake.

In all of these types of cases you end up with the correct interaction, the correct lighting, correct camera match since it's all happening in the shot. This also makes it easier for the director and the rest of the crew since they can see the action on set. In post this provides the editor with a better sense of the action.

As soon as multiple things start happening you need to work out the timing. First this happens and then 1 second later this happens and then 2 seconds later this other thing happens. If previs exists it will provide a good reference but there’s almost always adjustments to be made on set. If the action is related to something close to human size  the vfx supervisor may pantomime the action on set so the director, actors, camera operator and special effects crew get a sense for the timing. If there are a number of these types of shots then production may hire an actor to play the role on the set.  Ideally after the run through the shot would be done without the standin actor (to avoid a lot paint removal) but in many cases it's worth having them in the scene. This provides a moving reference for everyone.  The director and the asst director will try to get through these shots as quickly as possible but you should push to shoot 2-3 takes if possible to have different timings.  Invariably there are new ideas later or the creature or type of action is refined.

Note that in many cases it's not recommend to paint the rig or outfit the person in green or blue.  In many cases you won't be able to easily pull a matte anyway and more than likely you'll have colored bounce light that will need to be removed.

Post-production
While filming keep notes of the various bits and pieces of additional elements that will be required for the finished shots. Flames for torches, smoke from the house, water splashes, etc. In some productions you can shoot these elements with the special effects crew at the end of production.  For DRAGONHEART special effects built essentially a waterproof wood frame probably 20 x 20 feet and 1-2 foot deep. This was used to shoot additional splash elements.  At ILM we had special effects and camera crews available so we would plan an element shoot such as the flames to match the top of the coach in VAN HELSING or blowing leaves or dust hits to match the motion of creature foot steps.

Many of these types of elements (smoke, fire, dust, water splashes, rain, etc) can be shot against black.  Only use a colored screen if you're shooting something that is multiple colors. Use video and or stills images from the original footage to get basic framing. Shoot wider to make sure you don't clip the element off. You can blowup a bit in the comp but if the smoke or fire hits the edge you can't reduce it or reposition it.  Lighting for rain and splashes is usually best done as cross back lighting. Backlit smoke also works well. These days you can actually do a quick comp on your laptop depending on your camera system (if it's digitial of course).  Depending on the relative size of the element action compared to it's finished size it may need to be shot at a higher frame rate.  You can slow down in post but if you have the option to shoot at a faster frame rate, do it.

For those who only have experience with computer graphics they will likely try to solve all of these elements with computer graphics. The problem is cg simulations tend to take a long time to look correct. It’s can be very efficient to shoot elements if you're prepared and know what you're doing and the results have all the nuances of reality. You won’t be spending days debating whether it looks real or not.  It is.  The key advantage to cg is you can sometimes tweak it to a fine degree but that takes time.  In some cases the scale of the effects elements may make it impossible to shoot elements for.  You may also shoot elements and then modify, retime and distort as necessary to fit the particular needs of the shot. The vfx company should always try to keep the elements they shoot based on their contracts.  There are some elements that require very specific alignment but many are likely to be useful in an element library where you can simple re-use elements from previous productions.  These end up being used a lot and come in very handy when you need to stick in a small smoke element here or there.  In some cases a vfx company decide to spend a day shooting generic elements to have on hand.

In post-production you may have to help the director and editor determine which take works best for the animation to be added, especially if there's not a clear performance issue of the actors in the scene.  In some cases I’d take the video versions of the takes and do quick mockups to check for timing and position.

ArtBeats and I suspect a few other companies offer elements that could be used in smaller productions.

For more info check out the VES Handbook
which has articles on shooting elements and providing references for the actors eye lines.