Photo: Wikipedia.
A new developer-friendly open-source library released by UBC’s Human Communication Technologies Laboratory helps pave the way for new applications in computer vision.
(Vancouver) For GRAND researcher Dr. Gregor Miller, computer vision is at the forefront of many of the cool technological shifts we see today. His research at UBC’s Human Communication Technologies Laboratory aims to make the domain’s techniques accessible to mainstream software developers – suggesting the biggest shift in computer vision tech is yet to come.
Computer vision generally involves extracting and processing visual information from images and video, often in real-time. In addition to numerous applications in industry and consumers - in cameras to detect faces in photos or, in manufacturing, defective products on a conveyer belt - computer vision techniques have been integral to many of the exciting new digital technologies developed in the past decade.
“Computer vision is very quickly becoming the basis of the most fun applications available to consumers. Things like Kinect on Xbox, or virtual and augmented reality, Google Glass – all these things are providing perspectives that were not available ten years ago,” said Miller, whose postdoctoral research is supported through GRAND.
With the advent of cheap digital cameras, as well as games and applications that rely on video input, interest has also grown in using computer vision to organize and process the torrent of video and image data being generated.
“Just on YouTube alone, 100 hours of video are uploaded every minute," noted UBC professor and GRAND investigator Dr. Sidney Fels (Miller’s postdoctoral supervisor). "Yet, there are virtually no tools available to help process all this data – essentially, we have more cameras than displays in the world,”
For Fels, a new wave of applications is needed to process visual data and take advantage of the latest improvements in computer vision and hardware acceleration research. The trouble is that computer vision algorithms are notoriously complex; few mainstream developers know how to apply them to real-world problems. Implementation of even simple applications, such as face detection, requires expertise and training that go far beyond the scope of general developers.
With the first public release of OpenVL (Open Vision Library), Miller and Fels hope to change all of that.
Making computer vision open and accessible
Fels launched the OpenVL project in 2004. He and fellow GRAND researcher Dr. Jeremy Cooperstock had found undergrads struggling to develop working applications with existing computer libraries such as Intel’s complicated OpenCV (Open Source Computer Vision Library).
“OpenVL started from frustration that there isn't a computer vision abstraction for regular application developers, such as normal fourth year undergraduate students, who want to make kick-ass applications that use cameras,” said Fels. “The applications didn't work so well as they weren't using the latest techniques or didn't know how to tune them.”
He also noticed that the same students working with OpenGL (Open Source Graphics Library) were able to create amazing applications within a week. This lead him to consider creating an abstraction layer for computer vision that would allow developers to specify what they wanted done, rather than how they wanted it done – something analogous to OpenGL.
OpenVL provides developers with access to sophisticated computer vision techniques while hiding their minute details. Using a rudimentary but powerful processing algorithm called “image segmentation,” images and video can be decomposed into simpler, analyzable parts. Software can then be built to process this input information and make decisions for a wide array of applications.
“We’ve spent a lot of work on trying to find the correct way of basically providing the vision methods to developers so they can understand it,” said Miller. “My main contribution has been to develop the interface for developers to use, which took about three years to get it right. It’s really exciting, and I’ve got it to the point now where it’s going to take off.”
Miller joined the project in 2008. At the time, the student team was looking at developing hardware design to make vision methods available to developers. He offered a different approach: to write an abstraction level in software to help the developer find what it is she or he wants to do. Fels let Miller run with ideas, and put him in charge of the project.
By providing an accessible interface to developers, OpenVL takes computer vision methods outside the confines of academia and experts, and minimizes the steep learning curve. It also supports hardware acceleration for a new generation of applications on mobile, cloud, and desktop platforms. If adopted as an industry standard, OpenVL will make computer vision methods available to many more developers.
“The most exciting thing is that, if this works, computer vision will get into the hands of millions of developers,” said Miller.
“It is a completely different way of looking at computer vision,” said Fels. “OpenVL gives mainstream developers access to the latest and greatest algorithms and acceleration without having to know specifics. By separating the application developer from the need to know specifics of computer vision algorithms, we will open the door to widespread adoption of computer vision in an enormous range of applications.”
“Developers are the ones who make technologies accessible,” said Miller. “They’re the ones who write the applications for the consumers to use. So if you can’t make the technologies accessible to the developers, then they will never get in the hands of the consumers.”
Goal to become an industry standard
Through the release of OpenVL, Miller anticipates developers will begin to experimenting with new applications. Future software may allow users to scan 3D objects with a camera and send a copy of it to anyone. Face recognition, face detection, and full body games are other areas that will benefit from the technology.
Miller hopes that in a few years every camera is equipped with an OpenVL chip, making a new range of real time image processing available to consumers. Using hardware that plugs into laptops or mobile phones, people will be able to use apps that use sophisticated vision techniques. The problem is this hardware does not exist yet.
“One of the arguments for why it doesn’t exist yet is because we don’t have OpenVL,” said Miller. “I firmly believe that this is a stepping stone for which we’ll end up with a huge amount of new hardware and computer vision expertise coming out. And developers coming up with applications they can’t think of yet because they haven’t had access to these methods.”
The OpenVL project has had support from industry and government, including organizations such as GRAND, NSERC, Bell Canada Inc., Avigilon Inc., NVidia Inc, and Vidigami Inc. OpenVL’s current industry partner is global microprocessor manufacturer AMD (Advanced Micro Devices). Miller has been working with an engineer from AMD’s research department to look at the feasibility and use of OpenVL. This pioneering project will determine whether AMD will support OpenVL as a future open standard for the computer vision community.
"We are interested in providing a computer vision platform to enable general programmers to write computer vision applications with high performance and energy efficiency", said Dong Ping Zhang, an engineer from AMD's Research Group. "OpenVL increases the development productivity significantly compared with other computer vision libraries."
Ultimately, Miller envisions OpenVL becoming an industry standard comparable to OpenGL and OpenCL – a vision he sees involving continued industry collaboration with his research lab.
“Industry involvement is absolutely essential to the project. It needs to be a complete fusion between academia and industry to get enough force behind it. It needs researchers to develop the interfaces and the algorithms underneath, [and] it needs companies to get on board to make them robust to provide hardware that supports it, and integrate it into their pipeline.”
-30-
Contact
Spencer Rose
Communications Officer
GRAND NCE