Keeping users safe in the age of Trump
Engineers and founders need to adapt to the world in 2017.
The internet has become embedded in all of our lives. We use it to communicate, to share, to learn, and to organize. In a peaceful, democratic society, this power allows us to enjoy new experiences, be more effective at work, and be exposed to people and ideas we wouldn’t otherwise have discovered.
In a world where democratic ideals are being systematically dismantled, it allows us to be tracked like never before. For some people, this will make little practical difference — for now. For others, it’s a matter of life and death. And it will get worse before it gets better.
Tracking dissidents, tracking immigrants
Earlier this month, the web hosting service DreamHost revealed that it was fighting a Department of Justice request for data related to an anti-Trump website that included the IP addresses of every visitor to that site.
While DreamHost fought the request and managed to narrow its scope, a judge ruled that they are compelled to hand over data that includes authentication details and mailing list content — more than enough to identify active dissidents against the current administration. Whereas IP addresses are an imprecise way to determine identity (a large group of smartphone users on LTE might share an IP address, for example), login details and messages are not.
DreamHost also reported attempts to make the dragnet more palatable:
The court has asked the DOJ to present it with a “minimization plan.” This plan is to include the names of all government investigators who will have access to this data and a list of all methods that will be used to comb through it in search of evidence.
Shrinking the number of investigators who can directly access the data doesn’t make the investigation any less invasive; the results can still be shared more widely, and the potential effects on the people involved are no less severe.
Here in the US, Immigration and Customs Enforcement is analyzing public web data like social media posts in order to keep track of the activities of “visitors to the United States”, from the point of visa application to when they leave the country. Ostensibly, this is to detect when people overstay the length of their visa, but they’re also checking for “derogatory” postings.
In the UK, the Home Office used a dataset created by a charity to track the nationalities of homeless people sleeping on the streets to seize and deport them. The charity had compiled the data to provide better humanitarian services, but its effect was unforeseen.
Similarly, most people who work on social media tools don’t build them for their tracking potential. Nonetheless: if personal data exists for any person, it will eventually be used to track them.
Nobody is safe from surveillance. Let’s say that, for some reason, you don’t care what happens to immigrants. As the Atlantic points out, “surveillance of immigrants has long paved the way for surveillance of everyone.”
Biometrics are no exception. For years, the State Department let the FBI use face recognition to compare suspected criminals’ faces to those of visa applicants. In 2015, State and the FBI announced a pilot program to run these searches against the faces of Americans in passport photos. For years, Congress pressed DHS to use biometrics to track foreign nationals leaving the country. This year, DHS launched face scans through Delta and JetBlue — and both systems scan the faces of foreign nationals and citizens alike.
84 years ago, we saw racist policies extend to all Germans, where concentration camps held political opponents of the regime.
From Facebook posts to concentration camps
Immigration-related surveillance is even more troubling in the wake of President Trump’s unprecedented pardon of Joe Arpaio. Senator John McCain succinctly captured the problem in his statement:
Mr. Arpaio was found guilty of criminal contempt for continuing to illegally profile Latinos living in Arizona based on their perceived immigration status in violation of a judge’s orders. The President has the authority to make this pardon, but doing so at this time undermines his claim for the respect of rule of law as Mr. Arpaio has shown no remorse for his actions.
Arpaio singled out drivers based on their ethnicity and detained them without charge. This in itself is shocking; however, it pales in comparison to where he took them.
Tent City was an outdoor jail complex in Phoenix, Arizona, that held up to 1,700 inmates at its peak in the late nineties. Even recently, it could hold over 200 inmates. And even Sheriff Arpaio described it as a concentration camp.
The journalist Andrea Pitzer investigated Tent City as part of her book One Long Night: A Global History of Concentration Camps. In the Huffington Post, she described how Trump’s pardon has legitimized this activity:
“Once Arpaio began neighborhood sweeps and traffic stops deliberately targeting Latinos, and then detaining them without charges, his whole enterprise tilted further toward being a concentration camp for that set of detainees,” she wrote. “And even for those who had been convicted of crimes, it was a harrowing, often deadly experience.”
[…] “What happened yesterday is that the President of the United States put his position behind it and used executive power to bless these tactics,” she wrote. “Historically, when this kind of thing has happened, it’s encouraged other people to take up the same tactics. I think we need to hear from the Department of Justice whether official guidance is forthcoming about the use of these strategies by law enforcement.”
Nobody has counted how many people died in Tent City, but the temperature inside the tents could reach 145 degrees Fahrenheit. At a rally in Phoenix on Tuesday, just before he issued a full pardon, the President of the United States said that Arpaio had been convicted for “doing his job”.
The original big data
It’s hard to read these accounts without experiencing a knee-jerk response: this is against American values! It would never happen here! Nobody I know in technology would willingly work on this!
While understandable, this reaction is ignorant of the history of the tech industry. As the journalist Edwin Black recounts:
From the first moments of the Hitler regime in 1933, IBM used its exclusive punch card technology and its global monopoly on information technology to organize, systematize, and accelerate Hitler’s anti-Jewish program, step by step facilitating the tightening noose. The punch cards, machinery, training, servicing, and special project work, such as population census and identification, was managed directly by IBM headquarters in New York, and later through its subsidiaries in Germany, known as Deutsche Hollerith-Maschinen Gesellschaft (DEHOMAG), Poland, Holland, France, Switzerland, and other European countries.
The American tech industry was so integral to enabling the Holocaust, Hitler gave IBM’s President an award:
In 1937, with war looming and the world shocked at the increasingly merciless Nazi persecution of the Jews, Hitler bestowed upon Watson a special award — created specifically for the occasion — to honor extraordinary service by a foreigner to the Third Reich. The medal, the Order of the German Eagle with Star, bedecked with swastikas, was to be worn on a sash over the heart.
If you’re wondering, yes: Watson, IBM’s machine learning technology that works with enormous corpuses of data, is named after Thomas J. Watson, the CEO of IBM that sold Jew-tracking databases to Hitler. Let that put to bed any assumption that the tech industry is above assisting in these matters. (Palantir, of course, is directly involved in tracking immigrants today.)
Minimizing harm
In May, The Economist declared that data was the new oil:
What has changed? Smartphones and the internet have made data abundant, ubiquitous and far more valuable. Whether you are going for a run, watching TV or even just sitting in traffic, virtually every activity creates a digital trace — more raw material for the data distilleries. As devices from watches to cars connect to the internet, the volume is increasing: some estimate that a self-driving car will generate 100 gigabytes per second. Meanwhile, artificial-intelligence (AI) techniques such as machine learning extract more value from data. Algorithms can predict when a customer is ready to buy, a jet-engine needs servicing or a person is at risk of a disease. Industrial giants such as GE and Siemens now sell themselves as data firms.
Machine learning is an algorithmic technique that makes increasingly-good predictions based on trends discovered in enormous datasets. The bigger and better the dataset, the better the predictions are — and the more valuable the underlying technology.
The incentives, then, are to gather more and more data. However, as discussed, this same data has the potential to track human beings, potentially putting profit at odds with human wellbeing (perhaps not a completely shocking conclusion).
Yet not all data is useful for predictive outcomes — or any outcome. Companies tend to gather data for data’s sake. For example, none of the information logged in the DreamHost case was to be used for this purpose; the web server logs, mailing lists and account databases simply existed because they were the easiest way to provide certain functionality.
Ideally, in fact, those web server logs shouldn’t have existed at all.
In a hostile environment, we have to reconsider our data collection practices in order to keep our users safe. Most importantly, if you don’t have the data to begin with, you can’t be compelled to supply it to aid with a project you fundamentally don’t believe in.
One partial solution is to minimize data collection. Ask yourself: do you need this data? How long do you really need logs for? Can you allow users to authenticate in a way that doesn’t reveal their identities? I don’t believe that uncontrolled data collection is a moral activity in 2017.
Another partial solution is end-to-end strong encryption, which ensures nobody can examine data as it travels from one user to another via a service: something governments regularly and repeatedly decry. Amber Rudd, the UK’s Home Secretary, claimed in July that “real people” don’t need it. Chris Wray, President Trump’s new Director of the FBI, wants a “balance” that is impossible without rendering encryption hopelessly insecure. Leaving user communications out in the open for anyone to intercept is also not a moral act.
Even end-to-end encryption isn’t quite enough, as it only secures data in transit; the device running the app or displaying data also needs to be protected. In Scientific American, the computer scientist Megan Squire used a fictional app called “ClashBirds” as an example, pointing out scenarios we need to watch out for:
EvilRegime isn’t masquerading as the company that makes “ClashBirds” when we install our software.
No one has tampered with our “ClashBirds” app before or after installation.
The app doesn’t have any backdoors or security holes that could be exploited by EvilRegime after we install it.
As founders, engineers and investors, these situations are our responsibility to watch out for. In a world where law enforcement and other entities may be actively trying to harm our users, it is our responsibility to ensure that our software and services can’t have adverse, unintentional results for the people who use them.
Security is no longer about malware or identity theft, if it ever was. It’s about human lives, our democratic ideals, and the safety of real people in our communities. Even if Arpaio’s pardoning doesn’t turn out to foreshadow further actions along similar lines, it’s undeniable that government is using data to track human beings, sometimes splitting up families and putting lives at risk.
Moreover, if the tech industry is serious about being a force that changes the world for good, it must actively resist an administration that increasingly stands for divisive, violent hatred. Complacency — or inaction on changing common practices in light of the new environment — could have very real effects. There’s so much more at stake than raising money and growing our businesses. And there may not be much time left.
