Here's something you probably didn't know: Facebook has a team of employees who read your private messages if they have been flagged by an automated tool. The tool searches for content that appears to violate their terms of service, namely malicious (infected) URLs or child pornography. It's imperfect, of course that's where humans come in.
If a private message is flagged, actual people will jump in and read it. If there is something that could be illegal particularly regarding child exploitation those people contact law enforcement. The intent here is clear and defensible, yet the fact remains: All that stands between your "private" messages and the eyes of a stranger is the snap judgment of an algorithm.
Even more troublesome, OkCupid's readers are sometimes regular users, not employees. These users are deputized to help hand the large volume of "flagged" messages and can view ongoing conversations that may contain private information, whether or not the flagging users' claims are legitimate.
A private message on OkCupid read by a stranger and reposted online. Some moderators also share them publicly on blogs such as ThatsNotOkCupid.com and NotSoNiceGuys.tumblr.com While some aspects of content moderation can be easily automated, many can't. Google notoriously uses freelancers to comb through inappropriate public content, as does Facebook. But the issue of moderating privately shared content is a more complicated one; users should know that a Gmail message could easily be exposed by subpoena and that it is scanned for keywords to serve advertising, but would be mortified if Google employees were known to read "flagged" images.
