Sam Fisher and Research Assistant Marissa Ranali at NSCAD's Cineflux facility. Photo by Krista Kirby.

For NSCAD University film prof and researcher Sam Fisher, there is no technology without art – technology emerges from artistic drive. His Andra Motion Focus System nicely illustrates the point.

Showcased at NAB 2014 in Las Vegas, one of the world’s largest electronic media shows, the automated focus pulling technology was inspired by a need to make it easier for low-budget filmmakers to create high quality productions.

The system combines motion capture tech with remote focus – a way to control lens focus wirelessly – to automatically track subjects and deliver near perfect camera focus with terrific accuracy. Andra’s integrated iPad app and touchscreen controller also make the difficult processes of focus pulling considerably simpler.

Traditional focus pulling is a highly technical and entirely manual operation. Becoming a professional focus puller (or First Assistant Camera) takes years of training and practice requiring extensive knowledge of optical and cinematographic theory. By removing the technological barriers to filmmaking, Fisher aims to give artists more time to pursue film's purely creative aspects.

“I’ve always been interested in the ability for anybody to make a movie that really looks professional. There’s something interesting about the way the image communicates to people, the quality of the image, the quality of the product,” he explains.

As an instructor in NSCAD’s Media Arts program, he saw his students grapple with the technical challenges of focus pulling. Professional focus pulling is something beyond the scope and budget of most film students and independent artists yet crucial to the polished look of big budget cinema and video production – even if it goes largely unnoticed by moviegoers.

“I’d get these really talented students, but they couldn’t create work that can compete in a big arena simply because they didn’t have access to the funding. One of the key aspects to that is focus pulling. Without it, much of what they shoot is unusable. I wanted to solve this real problem and be able to give it to my students to test out.”

A graduate of the prestigious London Film School with over thirty years of experience in the film business, Fisher also understood the limits of manual focus pulling within the industry.

With the shift to shooting digital film, artists have pushed for larger chip sizes, low light filming, and very shallow depths of field. Difficult shots such as improvised performances and quick-paced movements are virtually unmanageable using traditional manual focus. Depending on the parameters of a given shot, there is simply too little room for error. It inspired Fisher to rethink decades-old film technology.

“There’s always an artistic impetus to do something, and that wanting to do something is pretty much always artistic. If you want to do this creative thing, then you want to find a technological solution.”

Value of pure research

Fisher began working on an initial prototype system in 2009, shortly before joining GRAND. His research assistant was former NASA programmer Mike MacDonald, an undergraduate HQP in GRAND. Now finishing a BFA at NSCAD, Macdonald was the architect behind the iPad interface for Andra, as well as other aspects to the system’s hardware and software.

Ruling out in-camera solutions, which lacked the precision needed for close range focusing, Fisher realized he could automate focus pulling by triangulating the distance between the subject and the camera. To get the exact positional data needed to make the calculations, the researchers turned to motion capture technology.

Thanks to a GRAND equipment grant, the team procured an advanced wireless magnetic motion tracker called Polhemus G4. The technology uses discreet sensors placed under the performer’s clothing, similar to lavalier microphones. Focal points could then be set relative to the sensor’s location allowing for focal lengths to adjust  to the performer’s movement. The motion tracker proved ideal in feeding the system precise real-time motion measurements.

The researchers faced a series of technical hurdles. During the proof of concept, they discovered the required depth of field was an infinitesimal 5mm, even for wide lenses because they are so often used for extreme close up work. A more troubling problem – a half second “data lag” in the system itself – meant that moving subjects were always just out of focus.

Using custom hardware built by Dartmouth-based electronics manufacturer Sunsel Systems, Fisher’s team was able to reduce the lag to fewer than 75 milliseconds. Further reduction was achieved using predictive filtering – a solution first proposed by McGill University computer science professor Dr. Paul Kry following a demonstration of the prototype at the GRAND 2012 annual conference.

Dr. Kry provided the perceptual data that informed Andra’s specs on how long it takes an observer to become aware that something is out of focus. Through the collaboration, Fisher was able to bring down the lag to a virtually imperceptible 50 milliseconds.

“Having Paul Kry’s research in the background meant I could fall back on it. You could answer a question like ‘how fast or accurate does this system really have to be?’ It is really valuable knowledge to have when designing a system.”

As a collaborator in GRAND’s VIRTPRES (Enhanced Virtual Presence and Performance) and MOVITA (New Directions in Moving Image Technology and Aesthetics) projects, Fisher came to appreciate the importance of research in film innovation.

“The film technology companies are all very small companies, for the most part. There’s a lot of money in the film industry, but not a lot of research dollars. GRAND has been on board with this thing the whole way through. Even though the product that I’m making is very industrialized – a straight line towards solving a problem – I’m a huge supporter of pure research.”

In 2013, Fisher took out a patent on his Andra System, and incorporated his startup Cinema Control Laboratories, where he is CIO. In partnership with Sunsel Systems, he plans to make Andra commercially available in 2015. His students will get to try it out even sooner.

“I’m finally going to introduce the technology next semester, so that’s really exciting!”

Future applications

Fisher is now moving on from his focus pulling research to new applications for the technology as part of GRAND’s Phase 2 projects.

Having refined technologies that capture positional data of objects in a scene, his research may prove useful for converting 2-D film into 3-D. Stereographers (the 3D equivalent of a cinematographer) deal with a number of parameters other than just focus and depth of field to obtain the right "look" for a 3D film. Fisher’s research offers the ability to pre-plan these parameters and then invoke them with a touchscreen controller, something likely to find applications in S3D (Stereo 3D) films.

Fisher’s research also intersects with a much broader inter-university research between NSCAD, Dalhousie and McGill universities to develop new and advanced uses of motion capture data to streamline production workflows and enhance creative precision.

“When you’re doing industry research, there’s always the tendency to not do the research; there’s always a tendency to say: ‘let’s just get it out there and figure it out.’ There’s a lot of technology out there like that. But you need to ask those pure research questions because so much grows out of it. There’s just not enough of that in any industry.”

-30-

 

Media Contact

Spencer Rose
Communications Officer