Using plaintext to post status updates and spread hate messages can be easily picked up by systems designed to scan and filter text. However when this text is put into memes, images, and videos, it becomes a bit tricky, especially since one can easily swap out a video or image for a new one with existing text.
This is one of the challenges that Facebook faced and one that they’re taking head on with a new AI that they’re calling Rosetta. According to Facebook, “It extracts text from more than a billion public Facebook and Instagram images and video frames (in a wide variety of languages), daily and in real time, and inputs it into a text recognition model that has been trained on classifiers to understand the context of the text and the image together,” which you can see in the image above that shows how it is broken down.
As to what Facebook plans on using Rosetta for, the company points at how it can be used to detect hate-speech, especially when applied in different uses and different languages. They also point at how it can be used to improve the accuracy of classification in photos in the News Feed to help improve on personalized content.
Facebook also mentions how there are still areas that can be improved upon, such as in use for video.
Filed in . Read more about AI (Artificial Intelligence) and Facebook.