Facebook’s foundation is built upon users uploading, sharing, and interacting with photos. But for many visually impaired individuals, this feature has never been available to them. Most sites have alt text that can be read by a screen reader, but the vast majority of the Internet is still fairly inaccessible to those with visual impairments. Facebook is changing the game using automatic voiceover and alt text to make Facebook, and hopefully the vast majority of the web, a more inclusive place.
Facebook’s advanced piece of technology recognizes objects within photos and identities them as that specific object automatically. This artificial intelligence has been conditioned by being shown millions of photos of common objects like dogs, smiles, pizzas, oceans, trees, and so on, and can now predict with high certainty what’s in a photo. The automatic alt text uses iPhone’s VoiceOver feature to read the descriptions aloud to users. The technology can give descriptions such as “two people, smiling, sunglasses, sky, outdoor, water” along with the caption associated with the photo.
Matt King, a Facebook engineer who is blind, spoke with The Verge on how inaccessible photos on the Internet can be for a blind person.
“People talk in pictures,” said King. “And talking in pictures is inherently out of reach for me.”
King noted that it would probably be too much to ask users to alt tag their own photos, so this advanced technology is a step in the right direction.
“We need a solution to that problem if people who cannot see photos and understand what’s in them are going to be part of the community and get the same enjoyment and benefit out of the platform as the people who can,” King said.
Not surprisingly, there are still some kinks to be worked out, and the tech has a lot of room for advancement. Right now, Facebook will only suggest the automatic alt tag and voice over for a photo if it is at least 80% certain that it knows what it’s looking at. And, there are still quite a few details that the algorithm can’t identify just yet. That probably won’t be far behind, and the Facebook team has already started working on automatic alt text and voice over in video, too. Facebook is taking some big strides to make the web more open and accessible for everyone, and it’s exciting to see it unfold.
“Inclusion is really powerful and exclusion is really painful,” King told The Verge. “The impact of doing something like this is really telling people who are blind, your ability to participate in the social conversation that’s going on around the world is really important to us. It’s saying as a person, you matter, and we care about you. We want to include everybody — and we’ll do what it takes to include everybody.”