Skip to main contentSkip to navigationSkip to navigation
Number crunching Chinese citizens will be digitally quantified by 2020
Number crunching Chinese citizens will be digitally quantified by 2020. Photograph: Guardian Design Team
Number crunching Chinese citizens will be digitally quantified by 2020. Photograph: Guardian Design Team

Algorithms are taking over – and woe betide anyone they class as a 'deadbeat'

This article is more than 5 years old
Zoe Williams

When big data decides if you deserve a loan, or classifies you according to bigoted stereotypes, we lose our freedom

The radical geographer and equality evangelist Danny Dorling tried to explain to me once why an algorithm could be bad for social justice.

Imagine if email inboxes became intelligent: your messages would be prioritised on arrival, so if the recipient knew you and often replied to you, you’d go to the top; I said that was fine. That’s how it works already. If they knew you and never replied, you’d go to the bottom, he continued. I said that was fair – it would teach me to stop annoying that person.

If you were a stranger, but typically other people replied to you very quickly – let’s say you were Barack Obama – you’d sail right to the top. That seemed reasonable. And if you were a stranger who others usually ignored, you’d fall off the face of the earth.

“Well, maybe they should get an allotment and stop emailing people,” I said.

“Imagine how angry those people would be,” Dorling said. “They already feel invisible and they [would] become invisible by design.”

The capacity of tech to outstrip the worst imaginings of its detractors is truly incredible. Prioritising emails turned out to be small fry for big data, which turned its attentions instead to simply ranking people, not for the interest they might hold in an inbox, but for their value as customers, employees, tenants – for all practical purposes, their value as human beings.

The Chinese government is working towards assigning its citizens a “social score”: by 2020, an algorithm will rate citizens as a “desirable employee, reliable tenant, valuable customer – or a deadbeat, shirker, menace and waste of time”, in the words of two US academics. “Waste of time”, it strikes me, is a more searing criticism than “deadbeat”, which sounds quite rakish and rebellious. Algorithms don’t understand nuance, because it saves time not to. But the erasure of small degrees of human difference is the least bad thing about it. The scored society, as the New Economics Foundation calls it in its report, What’s Your Score?, is everywhere: it is just more pronounced in China because the government is not embarrassed about it.

All our debates about the use of big data have centred on privacy, and all seem a bit distant: I care, in principle, whether or not Ocado knows what I bought on Amazon. But in my truest heart, I don’t really care whether or not my Frube vendor knows that I also like dystopian fiction of the 1970s.

I do, however, care that a program exists that will determine my eligibility for a loan by how often I call my mother. I care if landlords are using tools to rank their tenants by compliant behaviour, to create a giant, shared platform of desirable tenants, who never complain about black mould and greet each rent increase with a basket of muffins. I care if the police in Durham are using Experian credit scores to influence their custodial decisions, an example – as you may have guessed by its specificity – that is already real. I care that the same credit-rating company has devised a Mosaic score, which splits households into comically bigoted stereotypes: if your name is Liam and you are an “avid texter”, that puts you in “disconnected youth”, while if you’re Asha you’re in “crowded kaleidoscope”. It’s not a privacy issue so much as a profiling one, although, as anyone who has ever been the repeated victim of police stop-and-search could have told me years ago, these are frequently the same thing.

Privacy isn’t the right to keep secrets: it’s the right to be an individual, not a type; the right to make a choice that’s entirely your own; the right to be private. The answer is structural. I’m as sick now of being told to delete my Facebook account as I was 20 years ago of being told that turning plugs off would halt climate change. We need better laws, fast, or we’ll all be deadbeats in the end.

Dominic Raab – the Tories’ new ultra

Give Theresa May the benefit of all possible doubt: understand the position in which she finds herself, shorn of allies, her enemies emboldened, permanent private secretaries resigning in droves over matters of conscience that five days ago they seemed OK about.

Dominic Raab. Photograph: Mark Thomas/REX/Shutterstock

Also imagine, as is the vogue, that she is more intelligent than she looks. The appointment of Dominic Raab as Brexit secretary, successor to the departing David thick-as-mince Davis, could, in this light, be because she can’t stand Raab either and has given him the task as you might throw a stick off a cliff for an unwanted dog.

He remains a controversial choice, holding – per Labour’s pithy, timely video – views about as unpleasant as you could imagine in a Tory, past or present. Food bank users have a “cashflow problem”, properly funded universal healthcare is a “childish wishlist”, social and economic justice are for the birds, feminists are “obnoxious bigots” , getting rid of employment rights would be good for prosperity – his utterances are like a greatest hits of all the incurious, contrarian, lazy, Singapore-in-the-Channel banality that has become the Tories’ party piece since they uncoupled from the Lib Dems and stopped having to look at evidence, listen to people, consider the success of a nation as the sum of its peoples’ happiness, all that social crap.

Once, you would have known a Conservative ultra by the way he dressed and sounded: cummerbunds, monocles, flutey Rees-Mogg voices, fob watches – they had a whole range of flagging devices so that you knew when to stop listening. Now, they walk among one another as though they are all the same, extremists dressing like the garden-variety defenders of the propertied class, making it impossible to know whom you should engage in argument and whom you just ward off with garlic and a crucifix.

“It’s not my brief,” Raab said, when I met him a few weeks ago on Radio 5 Live, and asked whether the famous white paper on the Brexit plan had actually been written. Ha. It is now, sucker.

The end of the world? I’d know about it – wouldn’t I?

My kids saw the RAF flypast this week and reached, in concert with their school peers, the conclusion that it was the start of a nuclear war, which Donald Trump had started to coincide with his arrival. I didn’t want to be a buzzkill, but I thought I should point out some basic principles about the onset of the apocalypse. It wouldn’t start with synchronised planes flying in the shape of a 100, unless someone had a really dark sense of humour. And it would probably commence on the continent you were furthest away from. After that, I was out. “It won’t happen without me knowing,” I tailed off weakly, implausibly.

Most viewed

Most viewed