https://www.instagram.com/p/BV7ai1pAp2v/?hl=en&taken-by=kevin
In a profile of Instagram CEO Kevin Systrom for Wired, he revealed that in testing the DeepText system to figure out how to filter derogatory and offensive comments from the service, the team in charge put several bars of Kanye West’s “Famous” through to see if the machine could learn about context in filtering possible service violations.
DeepText is just one of the tools that is being used to cut down on trolling on Instagram, many of which were announced at the end of last year, but it might be the most sophisticated. According to Wired:
DeepText is based on a machine learning concept called word embeddings. When the system encounters a new word, it tries to deduce meaning from the other words around it. If a watermelon emoji is always surrounded by right-wing memes, that means something. The more data the classification engine analyzes, the smarter it gets. Like us, it learns over time; unlike us, it doesn’t get exhausted or depressed reading the word cuck 72 times in a row.
While the machine is showing positive results so far, testing shows that it still has trouble with certain concepts, especially when some cultural barrier changes the meaning. For instance, the system blocked even British uses of the word “fag,” which is slang for cigarette, and had issues when Kanye lyrics were quoted. The infamous “Famous” line “For my Southside n*ggas that know me best / I feel like me and Taylor still might have sex / Why, I made that b*tch famous,” got blocked every time but “You left your fridge open / somebody just took a sandwich,” was perfectly fine according to DeepText.
Instagram hopes that eventually the system will be able to automatically filter mean comments without also deleting critical, thoughtful discussion on the platform. In the meantime, users can still manually filter certain language or emojis, which is why you see significantly fewer snake emojis under Taylor Swift’s posts. Maybe one day, we’ll see a kinder, gentler internet.
