“I solemnly swear that I am up to no good”
-J. K. Rowling, Harry Potter and the Prisoner of Azkaban
Yesterday morning, I slept in and unpacked from a trip I’d made over the long weekend. I met some classmates in Rutherford for a group project and had tea in SUB. On the way home, I found a seat on the LRT and reminded myself to replace the passport I’d lost months ago, hating myself a bit for failing that hallmark of adulthood.
It was a pretty uneventful day, which is precisely why it’s so noteworthy. But that’s not to say it was perfect– I was late to my meeting, keeping my classmates waiting. I forgot to sort my tea bag properly into “Organics.” I still didn’t find my passport. What if there were consequences for these mistakes? What if I was kicked out of the project or not allowed to travel anymore?
These were the questions that entered my mind as I scanned the headlines about China’s proposed Social Credit System, which ranks, rewards, and punishes citizens based on their behaviour.
The fact that I can move about my day so easily, without consequence, is largely thanks to the Canadian constitution and my status as a white citizen. I can still think back to the couple years of my childhood spent in Fiji during the military coup, when my commute to school was sure to be interrupted by at least one tank.
I cherish being free to go wherever I want, talk to whoever I want, and even manage a bit of mischief like Harry Potter and his friends, within respectful limits. Think of a mistake you’ve gotten away with, or a “deviant” person you’ve associated with, and imagine if that one thing could defined the rest of your life.
What is the Social Credit System?
The Chinese government is developing a Social Credit System (SCS), scheduled for a full launch in 2020. The SCS monitors the behaviour of their citizens and businesses, ranking them according to “social credit,” or reputation. By rewarding desirable behaviours and punishing bad ones, the government hopes to foster “trustworthiness” in its people.
As of now, the SCS consists of a blacklist of people sanctioned for fraud, tax evasion, court orders, or safety violations. People on the blacklist can be banned from flying or working in state firms, or even have their internet speed lowered. The blacklists are public and employers are discouraged from hiring those on the list.
In addition, the government has authorized local and private social credit initiatives. This is where things get really wacky. Based on big data gathered from social media, administrative records, transaction records and consumer profiles, initiatives like Sesame Credit judge people on everything from late payments to leisure activities.
“Someone who plays video games for ten hours a day, for example, would be considered an idle person”
-Li Yingyun, Sesame’s Technology Director.
Though Sesame Credit is optional, millions have already signed up to receive the system’s benefits. People with high ratings can get loans easier, rent bikes or hotels without a deposit, and get their visas fast-tracked. High scorers get their online dating profile boosted on Baihe, a partner of Sesame which happens to be the largest online dating service.
It is unclear whether these initiatives will be centralised and run by the government upon 2020, or operated by a third party.
How does it relate to sustainability?
At SustainSU, we talk about the three pillars of sustainability: environmental, social and economic. Social sustainability refers to the humanitarian and justice aspects often overlooked in conversations about money and the planet. For example, when looking at factories, we should consider not just pollution and profits but also the need for a living wage.
Credit rating systems originated in America and exist around the world for loans and insurance. However, the Chinese SCS is special because it explicitly rates people according to social norms.
Below, I present 6 cons and 3 pros of China’s social credit system. I hope these points will exemplify how I see the SCS as a threat to social sustainability, but also open a discussion around its potential benefits.
Cons of the Social Credit System
1. Rates businesses and individuals
According to a 2014 State Council report, the goal of the SCS is to promote “trustworthiness” through four goals: honesty in government affairs, commercial integrity, societal integrity, and judicial credibility. Three of these four goals (excluding societal integrity) could easily be accomplished by focusing on businesses and state institutions. It’s one thing to hold producers and government accountable, but a whole other ball game to hold individuals to such rigid standards.
2. Algorithms are biased too
Since algorithms are made by humans, they inherently contain human bias. Thus, the decision of what is “good” or “bad” behaviour has serious implications. For example, Sesame’s understanding of video gaming as a negative trait is clearly debatable, but holds serious consequences for gamers. Just as racial profiling often surfaces in crime prediction algorithms, I’d be worried about discrimination in the SCS.
3. Algorithms are “black boxes”
Academics have described algorithms as a “black box,” highlighting how little we know about their inner workings. While some blacklists and credit scores may be published, we have no way of knowing how these are combined with consumer and personality attributes to result in an individual rating. Without transparency, there are concerns that the government could misuse this program for its own purposes.
4. Ignores social and personal context
Algorithms are extremely reductive, yet have the power to change people’s lives. For example, there is clearly a difference between one person who misses a payment while in hospital and another who is simply a freeloader. Further, social norms privilege the elite, rather than those in need.
5. Labelling and stigma
As I learned in an introductory criminology course, labelling acts of “deviance” often results in stigma, decreased life chances, and a higher rate of repeated offending. Stigma requires an audience, and what better way to shame someone than defaming them through a nation-wide reputation system? This may harm an individual’s chances of rehabilitation after an offense or mistake.
6. Human rights violations
Some aspects of SCS and its pilot initiatives may violate basic human rights. For example, Sesame users with a high score can skip the long lineups at healthcare clinics. This policy gives priority access based on a judgement that is not even medical. Further, SCS adjusts your score based on who your friends and associations are. A friend whose “bad” actions lower their credit score may drag yours down as well, punishing innocent people.
Pros of the Social Credit System
1. Solving the “trust deficit”
According to officials, the SCS will address China’s “trust deficit,” an epidemic of low-quality goods and fraud and financial scams that are harming the population. For example, the contaminated milk powder scandal in 2008 sickened almost 300, 000 children. By increasing trust and monitoring businesses, the government can better hold people accountable.
2. Subprime credit
Many Chinese citizens don’t have a credit history, making it hard to obtain loans or mortgages. By using non-economic measures to evaluate credit, these citizens could gain access to resources for economic and social mobility.
3. Punished for actions, not credit
In an article, “No, China isn’t Black Mirror,” Ed Jefferson points out that the current government SCS does not punish citizens simply for having low credit (those pilots like Sesame may). Only people who commit fraud or civil offenses are punished. While some petty actions, like walking your dog off-leash, may lower your score, they won’t get you blacklisted or banned from flights.
“Automated decision-making systems act as empathy overrides, outsourcing inhuman choices about who survives and thrives, and who doesn’t. We empower machines to make these decisions because they are too difficult for us.”
What do you think of the Social Credit System? Do you see it as a threat or an opportunity for social sustainability?