Moving Camera

Using a moving camera when filming live action for Visual Effects.
Locked off camera, Post Moves, Motion Control, 2D Motion Tracking, 3D Match Moving and Face Markers are all covered in this podcast relative to live action photography.
Approx 25 minutes.

Transcript
Today I'll be discussing the moving camera.

Locked Off
The simplest type of camera move is no move or the locked off shot. The camera is placed on a tripod or dolly and isn't moved. This makes it easier to add visual effects later or to do multiple elements of the same setup. Examples of this could include adding a matte painted house to the top of a hill. If you wanted to create a shot where part of the actor is removed later – such as a leg as in Forest Gump or most of the body as in the invisible man you would shoot the shot two times. Once with the actor and another time with no actors. This shot with out the actors is called a clean plate and is pretty common for many visual effects. Because the camera doesn't move you have identical images – one with the actor and one without. In post production if we remove part of the image of the actor the clean version gives the image without the actor. Special rigs are removed the same way. This same process is easily used for creating twins from one actor. Shoot the scene once with the actor on the left side and then shoot the same thing with the actor on the right side. Because these shots don't change position you can do a simple split down the middle of the scene.

Normally the camera operator is making slight pan and tilt adjustments while sitting on the dolly and the asst cameraman is making slight focus and exposure adjustments. To obtain the best quality of clean plate get everyone away from the camera, off the dolly and avoid changing the settings, even to do slates. The size and position in frame will be different if anything changes, including the exposure. You can spend time fixing this in post but it's better if you can avoid the problem.

Even though a locked off camera makes it easier to accomplish visual effects it may not fit with the look of the rest of the film or the requirements of the shot.

Post Move
Doing a move on an image in post is sometimes called Pan and Scan. This also can refer to transferring a widescreen film to full frame video.

The scene is photographed normally and then in the composite stage the image is enlarged and a synthetic move is added by moving the image digitally. It's also possible to do this type of move with some scanners for better quality.

The problem here is the loss of resolution from enlarging the image. If your end production is video then it's possible to scan the film at a higher resolution.

In the past another way around the resolution problem was to shoot on VistaVision or 65mm cameras. VistaVision a format which shoots with special cameras that run 35mm motion picture film sideways, much like a still film camera. This larger film size allowed for blowing up without as much quality loss in the days of optical printing. Large formats were also common when doing any effects heavy productions in the days of optical compositing. 2001 A Space Odyssey, Close Encounters and Blade Runner are some of the films that used 65mm. Star Wars and most ILM films used VistaVision for effects until the last few years

If the live action element only makes up a portion of the frame then a post move doesn't cause a quality loss. An example of this is starting on a scene which is all live action and then pulling back to reveal a large matte painting extension to the scene. In the old days a motion controlled camera might be used to film a physical matte painting on glass. A rear-projector would be one method of adding the live action with the same move. With the advent of digital matte painting the matte painter creates the painting at a higher resolution. If the original live action scan is 2000 pixels across, the matte painting might be done at 4000 pixels across if the final scene was going to show twice as wide.

I'll discuss 2D match-moving later in this podcast, which is a form of post move on an element.

One of the problems with post moves is the very limited 2D or two-dimensional appearance of the moves. There's no perspective change or feeling of depth. To obtain true 3D camera moves of the original live action there are a couple of processes.

Motion Control
One way to deal with this is using motion control. Motion control is used frequently with shooting miniatures and models but here I'll focus on live action use. Normally the camera is mounted on a pan and tilt mechanism known as a camera head. The operator has a hand wheel for pan and one for tilt. For motion control you have a special head and usually a special dolly where all of these motions are controlled by motors. In some cases the operator moves this just like a regular head and a computer records the positions by using position encoders. In other systems the operator uses a joystick or a remote camera head that is designed to just record his hand moves. In this case the remote head looks like a camera head but it's just a box with the hand wheels.

Note that non-recording remote camera heads are also with live action frequently when the operator can't be at the camera itself. The camera is on a boom arm or other remote system where the operator watches a video monitor and remotely controls the camera.

With true motion control once a camera move has been programmed the move may be repeated over and over again exactly in sync with the camera. The repeatability means you can shoot the same actor walking through the scene multiple times with a complex camera move. This would be done to create twins using one actor during a camera move. Motion control also allows you to film moving clean plates so you have many of the same benefits as a locked off camera such as rig or actor removal. Since the move is recorded you can also take this data and use it later in a motion control system for models or convert the data and use it in the computer for adding CG elements.

Motion control can also be used to shoot secondary elements such as bluescreens people or objects that exactly match a previous live action plate. In this case the data for the move comes from the original motion control shoot or from a process called match moving, which I'll be covering shortly.

The downside of motion control is the requirement for a special system that has to be setup and be programmed. Most directors hate it because of the extra time and process. I recommend it only when you really have to get repeatable motion on the set or location.

I'll cover more details of motion control in a future podcast.

MatchMoving
The other option when a moving camera is required is to do matchmoving in postproduction. This is now one of the most common techniques that came about with the advent of the computer. The simplest technique is shot with just a pan and tilt that requires another images to be added later in the composite. As an example if we pan and tilt an outdoor scene and wish to put the image of a flying saucer hovering we could matchmove a point near the horizon or infinity like a distant mountain or building.

This is done after the film images have been scanned into the computer. These days' most professional compositing programs allow you to specify an area in the moving footage that you want to track. Frame by frame the computer compares the image in this area and tries to find the best fit around where it was last. We call this 2D tracking since it's only calculated in 2 dimensions x and y.

Once this is done the image of the flying saucer is composited over the background and the motion of the background is applied to it. Now you have a scene with a pan and tilt of a cityscape and a flying saucer that hovers over a building no mater what move you make. This creates the illusion it was there the time of photography. If you have 2 distant points that you can track then it's possible for the computer to calculate the rotation and size so you can not only pan and tilt but roll the camera and even do some basic dolly or zoom moves. I suggest distant points to track so you can obtain a more accurate motion track without having to worry about parallax or perspective. You typically also want to track a point in the scene close to where you want to place the additional image. Make sure you always have at least one or 2 points in the scene to track during the entire scene. If you're doing a big pan move make sure there's another suitable point that you can switch the motion tracking to during the shot. Ideally these would be close to the same distance away from camera.

Even though this is a 2D track it's possible to obtain 3D pan and tilt information. By knowing the lens used to photograph the scene the computer can calculate and output a 3D camera file that can then be used in 3D program for the 3D camera to pan and tilt. This renders an object that matches the move exactly.

Another variation on the 2D tracker is a 4-corner track. If you have a billboard in a moving shot and want to replace it with a different image you could track each corner of billboard on the computer and tell it to apply the move to each corresponding corner of the new image. This will distort the new image to fit it in the billboard even if the original scene is moving.

3D matchmoving
The next level of match moving is 3D match moving.
When shooting a scene measurements are taken of any landmarks in the scene (windows, doors, tables, that sort of thing) and in some cases special markers are placed in the scene. Later after the film has been scanned into the computer a person called a matchmover will build a rough replica of the set or location in the computer. The virtual camera, which is a simulated camera in the computer graphics program, is now animated to match the move of the film camera that filmed the live action scene. Originally this was done as a manual process of lining up the CG set with the image from the film. Now much of it is done using special 3D motion tracking software.

Note that some measurements such as the focal length of the lens may be a bit different than marked. If it's labeled as a 50mm lens it could in fact be a 48mm or a 55mm lens. As such you may need to make some adjustments manually to get the final matchmove perfect. Also be sure to check any automated 3D matchmoves since it's possible to fool the computer.

Now that we have a CG camera that matches we can now place a CG object such as a box on the match move floor and it will look like it was on the floor in the original photography. It still has to be lit and composited into the scene but it stays locked with the correct position and perspective in the scene even if the camera operator is moving a hand held camera around the imaginary object.

If a CG creature will be walking on rough terrain hopefully the matchmover has been able to recreate that so the animator can always keep the creature on the ground. This terrain shape is likewise used to cast CG shadows onto. The matchmover has also included any large objects such as tables and trees so the animator can avoid those when moving his creature. If the actor in the scene is interacting with the creature then the matchmover moves a CG pawn that represents the actor. In some cases the matchmover has to do a very tight match of the arms and legs of the actor so the animator can time and match the creature to the actor. Even though I'm using the term creature this could be any 3D object or effect such as CG sparks.

So now that we have the basic of the matchmove process down lets take a look at how that works when we shoot. If you're going to be using 2D tracking then survey the scene and make sure you have some definable points. Any place of sharp contrast or a spot. The corner of a building against the sky provides a clearly defined area that is fixed. The edge of a tree blowing in the wind is not a good point since it's changing. The edge of a building isn't a good point because it needs to be one specific and unique point.

These same requirements apply to four-corner tracking. If you're replacing the image of an old TV you may find that the TV screen is rounded and almost blends into the frame of the TV. In this case you may want to put fluorescent dots from the office supply store on the corners of the area you want to track. You'll need to paint or composite out the dots in the final scene.

If tracking points don't exist in the scene and you need them then you have to create them. If you're shooting an actor in front of a blue screen to create a distant vista in post production then place markers on or in front of the bluescreen. These may be 2 to 6 inch plus marks made out of tape or plastic. If it's a cloth screen material you can put Velcro on the back of the plastic markers. If you can't touch the screen use c-stands to hold them but note that there will be more removal work to do in post. It's important for any markers to be solidly locked down so don't suspend with them with thin wires.

You want the markers to be visible on the final film scans but not too large or you'll be doing a lot of extra paint out. Same with the number of markers. You want to always have 2 or 3 markers visible even when you pan and tilt the camera. If you plan to pan the camera 180 degrees hen you'll need a number of markers around the camera. The actors or props may obscure some of the markers. If you're doing a full day of shooting in front of the bluescreen then it's best to just setup a grid pattern of the plus marks. Note that if the depth of field makes the markers way out of focus or if there are no markers visible in the shot it becomes a matter of experimenting in post to create a move on the background that seems to work with the foreground. This can be very time consuming and frustrating especially if your actor is jumping or moving.

If you're doing 3D motion tracking you always want to have at least 3 trackers visible. As always the matchmover will be recording the lens and tilt information from the camera, possibly with the help of the coordinator and certainly with the help of the camera assistant. If you're shooting on a set the matchmover should be getting measurements of anything with straight lines that are clearly defined such as the tabletop or windows. With this information they can build a rough CG version of the set. Set drawings are seldom used to build the CG matchmove version from since there may have been adjustments and changes made to the real set that aren't reflected in the blueprints. You want to make sure to build the CG world to what's actually captured on film, not on what was planned. To help with this work the matchmover usually shoot stills and especially Polaroid's where they can mark the actual dimensions on the image. These and the notes will be used months from now when the visual effects are being started. In some cases stereo images are taken or multiple cameras are used to help document the relationship of the objects in the set.

For organic sets such as caves or outdoors in natural landscape small ball markers are typically used. It's important for these markers show up so they are usually ping-pong balls or tennis balls painted a fluorescent color. Bright LEDs can also be placed in ping-pong balls. This is especially useful in darker sets or night shoots. At times a box frame of known size is filmed in the scene as well to provide a defined object and perspective check.

If the scene is supposed to be a close-up of a creature but there is no fixed object in frame then vertical rods may be used that hold a marker in place. As always there should be at least 3 markers.

If it's a large exterior scene on uneven ground a grid of markers may be set down or at least measured. Now the animator will be able to match the creature feet directly to the real ground. On DragonHeart and most of the shows afterward when filming in a large natural setting we used a surveyors transit system connected to a powerbook or a handheld computer that records the true position in 3D space of xy and z.

The matchmover back at the studio creates a 3D ball in the computer matching those coordinates. In the past the computer camera would be moved manually to match the position. Now there are special 3D tracking programs that can do this automatically.

Face Markers
Sometimes we're required to create CG prosthetics. This is done for shots that can't be done as normal makeup work since it's going to change during the shot or it requires removal of sections of the real actor. This is the case with the jaw extensions in Van Helsing and the facial electronics in something like the latest Terminator movie. Small colored dots from an office supply store can be applied to a face or arm. It's also possible for the makeup person to apply makeup marks. In these cases it's important to know where to place the dots to get the best match. Because of muscle changes you typically want some on the cheekbones or area that won't change as much in addition to the edge of the imaginary prosthetic. The animator, model maker and/or matchmover will then have to change to shape and position of the CG prosthetic to match frame by frame.

Note that matchmoving doesn't solve the problem of repeating live action camera moves such as required for twin shots and doesn't provide us with a clean plate. It is possible to take the match move data and convert it into a move for a motion control systems later when you're shooting live action or miniature elements. If you need a clean plate (for paint restoring) then the operator tries to repeat his same movement as accurately as he can without the actors and then the compositor will have to massage it in post to try to make it work.

Low Budget
Most beginning filmmakers want to make as complex scene as possible but it's important to keep it simple and to learn the basics of the craft before overwhelming yourself.

As mentioned before the simplest process is the locked off camera. Attach you camera to a solid tripod and film your different elements, including your clean plate as needed, without touching the camera or tripod. This allows you to focus on the art of compositing without getting tied up in the difficulties of match moving.

If you're creating CG elements measure the set or area and try to create a replica of this. Use real units of measure in your computer graphics program. Measure your camera as well. This would include the height from the ground and the distance from camera. Technically you want the nodal point of the lens but middle of lens or film plane is usually accurate enough to start with.

You also want to record the tilt of the camera. You can use an inclinometer to measure this tilt. You can find these on the internet or at larger hardware stores. You don't need the fancy, all digital versions. A simple type based on a bubble level is fine. These are 3 or 6 inch vertical disks with a flat bottom. A needle always points up to provide a readout the number of degrees of tilt, Place it on something level to the film plane, such as the tripod head plate.

If you can find one of these or they're too expensive then get a simple 6 inch protractor. Tie a knot in the end of a one-foot piece of string. Thread this through the hole in the protractor from the back. Tie a weight such as a large nut or bolt to the other end. Place the protractor upside down with the straight edge of the protractor resting against the straight edge of the tripod head. Read the degrees where the string passes the protractor. You may find in your CG program that you have to add or subtract 90 degrees from this number or you may have to negate the number. There's no absolute position for pan so we don't bother to measure it.

Place your CG camera at this point and use the set you built be your guide to where to place the CG creature or object. You also want to be able to cast shadows from your creature onto this ground plane and any large objects such as tables.

Once you've done these types of shots then you can experiment with 2D motion tracking and 3D matchmoving. Start the 3D matchmoving by using just pan and tilt when shooting.