Beyond 24fps

Peter Jackson has been shooting The Hobbit in 3D at 48fps*. Jim Cameron is looking at doing Avatar sequels at 60fps. The preview of some of The Hobbit footage this last  week caused some concern and confusion. I didn't see the footage but will describe some of the reasoning and history of why frame rates are being explored.

Doug Trumbull developed the Showscan process (60fps) years ago and I worked on some projects in it as well as projected the demos for awhile. It was actually 72fps when I started but we conducted tests with people wired up and there was a definite response to the higher frame rates. Most dramatic was 24 to 48 and then it started tapering down by 60 fps. Dogs and most animals see at a higher frame rate than people. Dogs would go crazy in the showscan theater in a way they don't in a regular theater.

Keep in mind when you're seeing something projected at 24fps the projector is showing the same image 2 or 3 times (48 or 72fps) to avoid you seeing flicker because the human eye can easily see 24fps changes. On film projectors they had multi-blade shutters to accomplish this. Also note that before sound fps was 16-18 and 24fps was determined as the slowest frame rate they could use with sound. It wasn't based on research for the best quality but the least expensive to make do. Just like the layout of the qwerty keyboard (standard keyboard used around the world) was simply base on the limits of the mechanics of the day (avoiding jammed keys) and not on best usage.


Doug showed a Showscan demo that had a roller coaster ride (like the This is Cinemarama demo decades before) and a log ride. The demo would start in 35mm 24fps and then would switch to 70mm 72fps. At that point the entire audience would feel the movement of the coaster and the sloshing of the log. It was in fact a very real and visceral experience. You were there. In the log ride there was a shot of a waterfall and of a duck flying up with water drops flipping off it's wings. At 24fps that scene has a blurred waterfall and very blurred wings which we expect for film because we're used to it.  A similar scene at 72fps you could see the full waterfall effect as if it were live and could see the wings of the bird just like you can in real life.


This is NOT the Saving Private Ryan look. This doesn't strobe and this isn't artificial. 


One of the demos he later showed was a projectionist behind the screen leaning on the screen and talking to the audience. Most people thought it was real.


This works great for simulator rides where you have action going on. I directed the Space Race simulator ride at ILM. We shot it on VistaVision but it was created for 60fps Showscan. If you saw the results converted to 24fps compared to true 60fps there was no comparison. In one you're watching a movie and in the other you're experiencing the movie. This is why most special venue films and simulator rides run at higher frame rates since they're not locked into a standard and since they want to provide a real experience.


For film it produced a grain free experience and with 70mm it showed everything so makeup and closeups could be problematic. Just as 4k resolution, higher frame rates give more of a sense of details for flaws on set are more readily apparent. As with 3D stereo, some of the old filmmaking cheats are less tolerated.


With 3D stereo you're relying on eye disparity. And with 24fps you actually get a lot of strobing for any image that moves sideways or during a pan. And that's one of the problems Cameron and Jackson are trying to solve by going to a higher frame rate.


So with 48 or 60fps you have a much more real and visceral experience. You tend to move or jump in your seat more during action sequences or POV shots because you're mind is no longer looking at an illusion projected up on the screen. Your subconscious mind is responding because it now believes much of what it's seeing is real. The difference of going 2k to 4k is much less than going from 24fps to 48 or 60fps. You can see the difference on any device, any size. Those with newer TV sets can get a pseudo feeling for this by switching on the 120fps or 240fps cinema mode. But realize this is a simulation, not real original images, and realize it's on a small screen compared to being 40 feet or more across.


Part of the problem is video was developed to run at higher frame rates. For years of course it was inferior in terms of quality and live events have a certain look. This includes soap operas. One of the main reasons for that look was the 60 fields per second. And that's why 24fps video, when it finally came, was overwhelming chosen by filmmakers because it had a look like 'film'. It wasn't because of lack of depth of field or film grain that caused the obvious difference of video to film. It was primarily the frame rate.


Had video been at 24fps it's much more likely audiences would be more welcoming to higher frame rates. As it is anything starting to resemble live video has a stigma attached. 


Because 24fps has a bit of an illusion to it and because we've been so ingrained by seeing so much of it all our lives, it provides a slightly dream like quality to this form of story telling. At higher frame rates action sequences will seem more real but likewise so will the people, acting and sets.  Will this make them appear more as soap operas for standard dialog scenes?  I suspect young people use to video games will have no difficulty adjusting but for most of the audience it may be a bigger jump.  


24fps may be like the qwerty keyboard that continues years beyond it was required simply because it formed a standard that everyone learned and is comfortable with, despite not being the most efficient.


Doug Trumbull is doing on going tests including up to 120 fps and hopes to show comparisons likely in the fall of 2012. The VES and other groups are working with Doug when he is ready for presentations. His new method also includes a much more effective means to convert down to other frame rates. His website provides more details on the history and reasoning of frame rates and the other issues he's been pushing (brighter projection, etc)

Here's a recent video interview with Doug,



As far as visual effects is concerned all of these newer processes (4k resolution, 3D stereo, higher frame rate) add more work and require more time. That's why the compression of post-production schedules combined with the desire from  the studios for more advanced effects work and the additional workload of these new technologies results in massive overtime and frustration of the visual effects crews.


3D stereo requires twice as much rendering and requires some tasks to be done twice. Some things that visual effects relied on, such as standard 2D matte paintings, can no longer be done in the same way.
Moving from 2k to 4k resolution results in 4 times the size of image data. That takes more time to render, composite, move files, etc and takes 4 times the amount of disk space.
Higher frame rates increase many of these tasks in proportion to the increase in frames. Many frame by frame hand work (and yes, believe or not people work by hand on individual movie frames) such as rotoscoping will obviously require more care. Hopefully splining and other other systems will minimize the additional frame specific work but there will always be some.

*FPS stands for Frames Per Second. These are the number of still images displayed in sequence in a second of time to create the illusion of motion.

[Update 12/2/2012  Just saw THE HOBBIT at 48fps. It  definitely reduced the stereo horizontal strobing that's common in stereo films and smoothed some of the fast action. There were a few odd shots in regard to motion. Some seemed to have been sped up in post.  A little sense of video in some of the earlier scenes but once the movie gets going it doesn't draw attention to itself but certainly added a bit more realism with the 3D and higher frame rate. I hope to see it again soon to review more fully at a close position to the screen. ]


[Update 12/13/2012  I do recommend people try to see it at 48fps with an open mind so you can decide yourself. This could be one of the only times you get the opportunity to see a full feature film at 48fps (if the results don't sit well with the studio they will not do it again)  
As I've covered elsewhere this of course is the first attempt to do a full film at this rate so there are bound to be issues that will have to be sorted out.  

Also in the future we may see variable fps films.  24fps for 'normal' scenes and then a shift to higher fps for action scenes,POV scenes or as a creative choice to different viewpoints or sequences just as directors choose at times with color grading or other manipulation.  Doug's film BRAINSTORM was supposed to show the brainstorm experiences at 60fps so they were different. Wizard of Oz has black and white until the Oz sequence. ]

VES Annual Meeting

The Visual Effects Society had it's annual meeting last Saturday. There was a live feed to Vancouver and San Francisco.
It was also webcast throughout the world including the other sections of London and Australia.

The video of the meeting should be online in the next couple of weeks or so for all VES members.

As noted at the meeting the VES website is being totally re-written and is scheduled to be online in October.
Lots  of great stuff like forums, videos and user areas.  Headed up by Carl Rosendahl

Mike Fink gave a talk called 2000 years of visual effects in 30 minutes.
Discussed the history and art-form of imagery and how even early paintings are making an impact on cinema and visual effects.
Helped to illustrate the artistry and advances in visual effects.


Bill Taylor was awarded the VES Board of Director's Founder's award.
Bill talked about many things including:
1. Keeping production local
2. Having a visual effects trade organization for visual effects companies.
3. Having a visual effects union.

This was well recieved. These are some of the issues that the VES board has been discussing the last couple of years as we look to the future.
As Bill mentioned Jim Danforth had brought up some of these same issues years ago.

The VES is an honorary society for Visual Effects Artists.  It's been discussed that that these 3 things: Society, Union, Trade organization, make a fully rounded industry and each of those had different purposes.

The VES determined it was best to remain as it was designed to be - an honorary society, instead of changing into one of the other forms.

I think many members and VFX companies are interested in the future and want to avoid some of the issues that we currently have to deal with.  A union and trade organization are possible ways to deal with those issues.

Doug Trumbull was the main speaker at the event.

Doug is always interested in getting the greatest experience and felt that many of the limitations have been based on
decisions made years ago that aren't relevant now.

Doug felt that film will disappear (no time frame given) and that digital can equal the quality of film, even now.

FPS
Doug had said that in early motion picture development approx 50 fps was determined to be needed to show continuous movement and avoid flicker.  Soon there after multi-bladed shutters to minimize flicker issues. 16/18 fps were chosen for silent movies to use as little as film as possible.  Sound went to 24fps not for the picture but to make it possible to do sync sound.
He developed Showscan years ago which originally ran at 72fps and later at 60fps.
24fps results in a lot of blur.  It requires the blur for your eye to read it as continuous motion instead of flicker still images. He and Kubrick found this out when trying to shoot stars without blur on an animation camera. Faster frame rates don't need as much blur and tend to produce sharper images.
Even though 24fps is thought of as movies and 60fps is thought of now as video, the faster frame rates provide a more life like experience.

Projection
The current standard for footlamberts on a movie screen is 16.  This is much darker than your flat-screen at home.
With stereo 3D projection now a polarizer is needed over the projector lens which results in 1 stop loss of light.
Then the glasses the audience wears has polarizers which result in another stop loss.  End result is the audience is watching an image at 4 footlamberts which reduces the color and quality of the image, especially compared to real life.

The current digital projection systems use a chip size that limits the amount of light that can be projected through it so there still remains a limit to how bright and real the screen is.  Doug would like to avoid these limits and take the opportunity now that we're transitioning into digital projection to think in terms of the future and not just come up with standards that are barely good enough.

Worth checking out the video.

There were also reports from the various committees with a lot of progress being made in the last year.