Facebook unveiled its artificial intelligence research, and the most remarkable accomplishment is a new system that allows people to ask questions about photos on the platform and the get correct answers in response.
The mobile app, which uses a system known as Visual Q&A, demonstrates how Facebook can use multiple approaches to a type of artificial intelligence called “deep learning” to convert existing data into something which can be utilized by an audience that would not normally be able to access it.
The new app means that the more than 285 million people globally who have hearing problems and another 40 million who are blind can now access and interpret information on Facebook that would ordinarily be beyond their reach. Through the app, they can now become part of the “Facebook experience” by sharing photo content.
Facebook has not released the app to the public as yet, but has allowed some people to sample it, and video documenting the experience is quite moving. In one of the clips, a cat is seen sniffing a large bunch of unripe bananas. The app, when queried about the photo, correctly answered that there was a cat in the image, what it was doing and the color of the banana and the cat (the bananas were green, and the cat was black and white). In other clip, a dog holds a toy in its mouth. When asked what the game the dog was playing, the app answered correctly “Frisbee.”
According to Yann LeCun, director of Facebook’s artificial intelligence research group, an app that accurately describes an image could prove very useful to the visually impaired.
The app is a perfect example of how artificial intelligence researchers at Facebook are trying to create systems that bring together the understanding of images and the ability to understand images. In the past, the two areas have been developed separately, and the new app will help Facebook users understand the world and manage it better according to LeCun.
LeCun and his team mainly focus on using the deep learning technique that involves creating software designed to learn from data. The software is modeled the same way brain cells connect and work together.
Google and Baidu are also moving toward developing apps that will smarten their products using deep learning.
When the new app by Facebook is finally released, it could as well be one of the most innovative apps to have hit the mobile world in the recent years, judging by the video documentation of its functionality.