As anybody who’s dealt with a jerk can tell you, being a jerk is often not just in what you say, but how you say it. That presents a problem for social media, not least because trolls are constantly trying to find the line and scootch juuuuuust to the right side of it to be jerks while not getting banned. So Instagram is deploying an artificial intelligence that learns not just what people say, but how they say it.
On its official blog, Instagram discusses a machine learning tool designed to spot hate speech and spam. What’s interesting is that it can learn the context of hateful comments and the euphemisms people use to express bigoted thoughts. Basically, most internet trolls rely on plausible deniability; if you corner one screaming about “those people,” you’ll quickly discover that “those people” is this magical banner that somehow encompasses all groups in theory, and that said troll only applies it to groups of people they’re scared of in practice is just this weird coinkydink.
DeepText is driven by humans. It starts with a person looking at a comment and reading the subtext. Then, they mark that subtext out for the AI and it goes off into the wilds of Instagram, looking for other examples. As it brings them back, the humans teach it what’s offensive and what isn’t. Cleverly, though, the trolls don’t know they’re being muted; the comment is only visible to them, but it isn’t marked or otherwise declared invisible.
Some stuff is going to slip through, and some stuff that’s fine is going to be blocked. There’s not really going to be a perfect system, short of self-filtering. But at least Instagram is making it easier to filter out the jerks.
(via The Next Web)