Can Facebook and Twitter stop social media surveillance?

Yesterday, social media surveillance became a very real problem for Facebook and Twitter. An ACLU report revealed a CIA-funded tool called Geofeedia was being used by police to track data from Facebook, Twitter, and Instagram to aid in investigations. Documents show Baltimore police used the tool, called Geofeedia, during protests after the death of Freddie Gray, even feeding Instagram posts through a facial recognition system to find protestors with outstanding warrants.

Facebook and Twitter were quick to revoke Geofeedia’s access to social feeds — effectively shutting down the current version of the tool — but its broader implications are harder to dismiss. Facebook and Twitter can control direct access to their data, but they have much less control where the information goes. Now that police departments are looking to tweets and Instagrams for clues, stopping them may be harder than shutting down a single app.

The center of the issue is how Geofeedia was getting the Facebook, Twitter, and Instagram posts it supplied to police. In each case, the company was drawing information directly from feeds supplied by the platforms: Facebook’s Topic Feed API, Instagram’s full API, and a feed supplied through Twitter’s social-data subsidiary GNIP. The feeds are meant to give developers direct, machine-friendly access to posts, making it easier to build software on top of the social networks. Typically, networks want developers building that software — it’s the same system that allows for third-party add-ons like VSCOCam and Tweetbot — but they control access with a Terms of Service and an API key that’s required to access the feeds.

After the ACLU report, Facebook and Twitter revoked those keys — but it’s worth considering how much that will set back similar tools in the future. Facebook and Twitter have complete control over their API keys, but both platforms have made it fairly easy to get one. Developers need to give a general description of the software they’re building and promise to abide by the Terms of Service, but there’s little enforcement and low-level terms of service violations are commonplace. Typically, that’s a good thing. The web was built on permissionless innovation, and heavy-handed enforcement efforts are often seen as bullying or arbitrary. But it also makes it hard for companies to ensure their data isn’t being used for anything controversial.

In theory, the Terms of Service would provide a clear line between what’s allowed and what’s not, but that mechanism is also more complicated than it seems. The document works more like a political party platform than a constitution, and can be subject to abrupt changes as company strategy shifts and certain products get deprecated or sherlocked. Facebook’s developer policy, for instance, includes strict language warning third-party data brokers and ad networks against exporting data. It’s a clear warning: if we catch you using the API to bypass our ad business, there’ll be trouble.

Geofeedia’s infractions are more subtle. Nominally, the company was violating Facebook’s provisions against reselling data and Twitter’s provisions against investigating and surveilling users. But the rise of big data has created countless startups devoted to mining insights from social media streams. People use that data for all sorts of things — trading stocks, spotting trends, or identifying influencers. When people start to get arrested because of that data, there’s an obvious chilling effect, but the distinction between selling data to police rather than a hedge fund is hard to pin down. The problem is with the clients rather than the behavior itself — and clients are easy to keep secret.

That ambiguity is a big part of why Geofeedia was able to stay on the platforms for so long. The company serves a wide range of clients — including “educational companies, cities, schools, sports teams, and the aviation sector,” in the CEO’s words. Absent a public shaming, there was no reason to think law enforcement clients would be any different. A similar case played out on Twitter earlier this year, when a company called Dataminr got in trouble for a contract selling Twitter analytics to the Department of Homeland Security. The company ultimately had to cancel the contract, faced with the prospect of losing access to Twitter’s data stream.

If surveillance companies do lose official access, there’s always the possibility of scraping the data without Facebook or Twitter’s permission. It’s a hacky solution, slower and less stable than getting data directly, and companies routinely get sued for it. LinkedIn is currently suing a team of still-anonymous scrapers after detecting some strange patterns on its network, and Facebook has waged a number of those battles over the years. That makes the tactic less appealing to contractors like Geofeedia, but if police departments or intelligence agencies conduct those scans directly, they might have a much easier time in court.

None of this will sound particularly encouraging to Geofeedia. Losing API access is still a huge blow, and it’s very hard to succeed as a social analytics business if Facebook and Twitter don’t want you to. Facebook and Twitter’s stance against law enforcement tracking really does make a difference, and will make it harder to build this kind of tool in the future. But it won’t make it impossible. There are plenty of less conspicuous ways to get at the same data, and as more of our lives move on to social media, it’s hard to believe police and intelligence agencies won’t exploit them.

Social media has been a powerful tool for protest movements from Black Lives Matter to Occupy, but it’s also a powerful tool for tracking and containing them. Facebook and Twitter take measures to protect users from blowback and clearly see themselves on the empowering side of the equation — but their businesses are built on using this data for targeting. If Pepsi can use a tool to find disgruntled Coke fans, why can’t police use the same tool to look for rioters and fugitives? It’s an uncomfortable question for Facebook and Twitter, and the answer may not be up to them.