NewsTechnology

Actions

Facebook programs computers to describe photos for the blind

<p>Facebook has always touted photos as a form of expression. And on Monday, the <a href="http://newsroom.fb.com/news/2016/04/using-artificial-intelligence-to-help-blind-people-see-facebook/" target="_blank">social media giant unveiled</a> artificial intelligence that allows blind users to join in on the conversation. </p><p>A voice from Facebook's new technology reads: "What's on your mind? Shaomei Wu. March 6 at 10:14 p.m. With my college buddies in my favorite place — ready for a great weekend! Photo. Image may contain: tree, sky, outdoor."</p><p>"One or more people. Jewelry. Smiling. Nineteen likes. Three comments –– like," a voice from the new technology reads. </p><p>"Now I can see the picture in my head, like, 'Yeah you shouldn't have been that close up,'" a blind Facebook user said. </p><p>Facebook is calling this new feature automatic alternative text. Before this, blind users would only hear the name of the person who posted the image and the word "photo" when they scrolled past images in their news feed. </p><p>The company's object recognition technology reportedly holds billions of parameters, which allows the new automatic alternative text tool to say what's in each photo. </p><p>Right now, the alternative text is only available in English and accessible on iOS devices. But Facebook says it plans to add other languages and platforms soon. </p><p>"Even having three words just helps flesh out all the details that I can't see. That makes me feel included, and like I'm a part of it too," a blind Facebook user said. </p><p><i>This video includes clips from </i><a href="https://www.youtube.com/channel/UCcr9tciZbuvJrEVAgIXCp8Q" target="_blank"><i>Facebook</i></a><i>. </i></p>
Posted
and last updated

MENLO PARK, Calif. (AP) — Facebook is training its computers to become seeing-eye guides for blind and visually impaired people as they scroll through the pictures posted on the world's largest online social network.

The feature rolling out Tuesday on Facebook's iPhone app interprets what's in a picture using a form of artificial intelligence that recognizes faces and objects. The iPhone's built-in screen reader, VoiceOver, must be turned on for Facebook's photo descriptions to be read. For now, the feature will only be available in English.

The descriptions initially will be confined to a vocabulary of 100 words in a restriction that will prevent the computer from providing a lot of details. For instance, the automated voice may only tell a user that a photo features three people smiling outdoors without adding that the trio also has drinks in their hands. Or it may say the photo is of pizza without adding that there's pepperoni and olives on top of it.

Facebook is being careful with the technology, called "automatic alternative text," in an attempt to avoid making a mistake that offends its audience. Google learned the risks of technology last year when an image recognition feature in its Photos app labeled a black couple as gorillas, prompting the company to issue an apology.

Eventually, though, Facebook hopes to refine the technology so it provides more precise descriptions and even answers questions that a user might pose about a picture.

Facebook also plans to turn on the technology for its Android app and make it available through Web browsers visiting its site.

The Menlo Park, California, company is trying to ensure the world's nearly 300 million blind and visually impaired people remain interested in its social network as a steadily increasing number of photos appear on its service. On an average day, Facebook says more than 2 billion photos are posted on its social network and other apps that it owns, a list that includes Messenger, Instagram and WhatsApp.

Until now, people relying on screen readers on Facebook would only hear that a person had shared a photo without any elaboration.

The vocabulary of Facebook's photo-recognition program includes "car," ''sky," ''dessert," ''baby," ''shoes," and, of course, "selfie."