The following is a series of notes,
quotes and reflections left over from my two recent articles for the
Battleground on China's social credit model and their tech landscape more
generally.
Ꙛ. In the West technologies which would enable
the expansion of algorithmic governance are predominantly held by the tech monopolies,
Google, Facebook, Apple and Amazon, as well as smaller developers working at
the intersections of big data and surveillance infrastructure, in addition to
more traditional commercial incentivising, marketing and analysis tools. So far
there is little in the way of a centralised state led approach to joining up
the different strands of what would be a totalising system utilising large
amount of citizen data. This is not surprising given the transnational nature
of these corporations and their well established antipathy towards state
regulation or co-option. This is not,
however, to deny the links between state surveillance and the private
technology firms which are sequestered to enable it. These technologies
are – despite the scope of data collecting undertaken by the likes of GCHQ and
the US NSA being an open secret - still largely viewed by the ruling class and
the professional middle classes as economic tools with differing degrees of
deleterious collateral effects.
The situation in China, however, is
marked by a mixed tech economy that currently supports a high degree of market
competition, uneven deployment within civilian life, as well as an increasing
degree of centralised governance encompassing legal regulation and military
contracting. The political will towards centralisation of digital technologies
for the purpose of what the CCP call 'Data Driven Governance' is certainly
present, however the current state of play reveals a diverse picture which is
as much about regulatory containment and limit setting as it is on the
wholesale construction of a nationwide digital panopticon. This latter remains
a fantasy born of Western anxieties over loss of competitiveness and
geopolitical maneuvering.
The overarching name given to this
fantasy is The Chinese Social Credit System (shehui xinyong tixi).
“If her risk factor fluctuated upward—whether due to some suspicious pattern in her movements, her social associations, her insufficient attention to a propaganda-consumption app, or some correlation known only to the AI—a purely automated system could limit her movement. It could prevent her from purchasing plane or train tickets. It could disallow passage through checkpoints. It could remotely commandeer “smart locks” in public or private spaces, to confine her until security forces arrived”. - The Panopticon is Already Here - The Atlantic September 2020
֎. Generalising somewhat, we might describe
two differing central domains of politics out of which the movement towards
algorithmic governance is occurring in China and the West. On both sides the present
tendency involves an inmixing of three generic elements within that central
domain, Order, Control and Economic Growth; each of which impart their own
rationality upon the whole.
In the West the weight falls decisively on the side of growth which - with the passing of the consumerist model of capitalism - we should no longer identify with traditional measures such as GDP, but more with the profitability of the main tech and data monopolies, as well as the nexus of finance and eCommerce that they support. The ordering potential of algorithmic technologies and machine learning is - according to the historical movement of rationalisation - a desirable and ideologically neutral extension of the organisational and utilitarian principles of secular Western societies. We should understand this at its most basic metaphysical level as the dispelling of darkness, the bringing into light of all that is otherwise concealed and previously impervious to calculation within the complexity of human affairs. In this regard we should closely associate calculability with profitability derived from the opportunities afforded by digitalisation for primitive accumulation. This would obviously not be the kind of primitive accumulation associated with colonialist exploitation, but, following the insight of Hardt and Negri is the turning inwards onto Western populations of forces expropriating the communicative resources of human beings, just as the former European imperial powers exploited their colonial subjects.
"In today’s largely data-centric environment, truth-claims can be specifically manufactured in a way that maximizes the pleasure experienced by target audiences by affirming their desires, hopes, and fears, or suggesting aspirational visions derived from preexisting opinion. As striving for pleasure is a natural feature of human life, and individuals are capable of learning to maximize their satisfaction by repeating and emulating previously successful behaviors (e.g. siding with opinion-congruent fake news), it is only logical that positive affective impact replaces truthfulness as the main criterion for news selection". Ignas Kalpokas - Affective Encounters of the Algorithmic Kind: Post-Truth and Posthuman Pleasure
ϡ. The social control potential of digitalisation
is largely disavowed in Western politics, something to be mitigated and
regulated by democratic oversight and market mechanisms. That Liberal Democracy
is utterly incapable of controlling the forces it has engendered is the tragic
element in this story. Tragic in that it seals its own fate, but also the fate
of all those theories of political-economy (Neoliberalism chiefly) which place
the autonomy of the market at their core. The canonical distinction between
Market and State - a nominal gap within which the history of modernity
has been written - is drawing to a close. The ultimate harmonising power of
algorithmic governance is to do away with this great historical antagonism. A gestalt
switch is occurring as the pandemic opens up greater demands for curtailing
uncertainty (economic, epidemiological, political) and managing risk in all
areas of society.
“In this digital era, new data and technology are shaping the credit reporting sector. Big data and machine learning, for example, allow for faster, more sophisticated, and more cost-effective data mining and processing while lowering the risks associated with human interventions. These technologies hold tremendous potential for credit information sharing”. - Leveraging big data and machine learning in credit reporting - World Bank Blogs August 2021
Ꙛ. The demand for social media companies to
‘tackle online hate’ is the clearest example of a disavowal of the true nature
of these technologies as they are deployed within the unbounded rationale of
the attention market (competition for available brain power). The presumption
that Statute may be able to contain the furies of utterly disaffected and
spiritually impoverished populations is as tragic as it is laughable. Following
Bernard Stiegler let us call 'online hate' the basic form of activity which
negative desublimation takes in societies increasingly dominated by an attention
economy of extreme affect; a combination of instant gratification, infantilised
attention seeking, and monetised cruelty. The YouTube Influencer is the
poster-subjectivity emanating from such an economy.
“Social integrity is the foundation of the construction of a social credit system. Only by treating each other with sincerity and trust will we form harmonious and friendly interpersonal relationships, promote the progress of socialist civilization, and achieve social harmony and long-term stability". - Guidelines of Social Credit SystemConstruction (2014-2020)
֎. From the perspective of the desire to
‘tackle online hate’ Western politicians and the central committee of the
Chinese Communist Party are in complete agreement; here the harmonising
potential of an extended social credit system reveals its cross-ideological
appeal. Nonetheless without the authoritarian core and will to reign in the
"all powerful subjects" that Silicone Valley's finest represent, all
these committee hearings with whistleblowers and soft regulatory bluster amount
to nothing. They fail to register the basic point, which is that the
corruption, amplification and monitization of our worst instincts IS the
productive core of the social media industry. Meanwhile, China can afford to make
both the influencers and their richest tech entrepreneurs disappear.
ϡ. From the side of ORDER, the socially
divisive consequences of an economy of extreme affect are negative upon the
whole, leading to “culture war” and general disunity. Conversely from the side
of GROWTH, the rigidity of the ordering principle – an extension of the
bureaucratic principle – leaves insufficient free play within the system to
enable production of new profitable modes of sociality, including the capture
of libidinal economy. It is the task of the third element, CONTROL, to find a
mode of binding the other two elements. As yet it is only China which has taken
this as an explicit task, with the social credit system and their most recent
plan for Rule of Law with Chinese characteristics touted as its vanguard policy.
One should also consider the significance in China of the Rule of Virtue and
the tendency to use centralising and legitimising functions of the ruling party
as limiting factors on the depth to which algorithmic governance could reach.
There are no such limitations West of Ankara.
“Historian Yuval Harari argues “free market capitalism and state-controlled communism aren’t competing ideologies, ethical creeds, or political institutions. At bottom, they are competing data-processing systems. Capitalism uses distributed processing, whereas communism relies on centralized processing.”218 While this characterization may be reductive, it does neatly encapsulate the divergent approaches to algorithmic regulation taken by China and the United States. Employing AI and big data to inform regulatory decision making is not unique to China or the CSCS. In the report ‘Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies’ released in February 2020 by a team at Stanford University, researchers found that 45 percent of U.S. federal agencies have experimented with AI in capacities quite similar to CSCS data-based initiatives such as Internet Plus”. – Trivium China Project report onthe Social Credit System
Ꙛ. In the Anglophone world, the disruptive
force of digital technologies in combination with rising inequality and
political dissatisfaction, has resulted in societies which seem increasingly
ungovernable by Liberal Democratic means. The advent of the security state in
the wake of the attacks of 9/11 was the clearest expression yet of the
permanent state of emergency which has not ceased to restructure the public
sphere around the management of risk and uncertainty. Now the pandemic has
accelerated these tendencies and forced crumbling Western democracies onto
foothills already trodden by Beijing.
“In the biosecurity context, as in any other, intelligence is a ‘service industry’ whose sole aim ought to be to interpret the threat and risk environment for a decision-maker in order that they can either disrupt threats or put in place mitigation strategies. Put another way, national security intelligence and law enforcement agencies play a role both in reducing uncertainty around the biosecurity environment, and the impact of threats and risks associated with it. The degree to which intelligence can interpret emerging bio-threats will depend on the issue, and the extent to which intelligence agencies are able to provide ‘value added’ information in a timely manner for decision-makers”. Patrick F. Walsh - Intelligence,Biosecurity and Bioterrorism (2018)
֎. Data plays a fundamental ideological
role in the field of risk management. A quantitative assumption is made
regarding the relationship of data to the evaluation of risk. Despite overtures
being made to the ‘quality’ of intelligence, in practice one never ceases to
demand more data, and the bureaucratic systems within which risk analysis takes
place ensure that the will to collect ever more quantities of data will
coincide with the blind expansion of the bureaucratic/rationalisation process
itself. In other words, bureaucracy and big data share deep seated metaphysical
characteristics. This should be of no surprise as the literal meaning of
bureaucracy – rule by the files – is immediately applicable to a society
algorithmically governed via the mass collection of citizen information. But,
with decision making having escaped human cognitive limitations into artificial
intelligence, will there be a place in the future for the bureaucrat?
“System 206 is the codename for the Shanghai High People’s Court intelligent assistive case-handling system for criminal cases, which aims to improve quality and reduce ‘false, unjust, or wrong’ charges and sentences. The system has two main components. The first is a cross-referencing system that uses speech recognition and natural language processing to compare different types of evidence and alert the judge to any contradictions in the fact patterns. The second is a sentencing reference tool based on machine learning algorithms that use the defendant’s basic information (criminal record, age, ‘social harm’ incurred, etc) to search a large database of past court records for similar cases and make sentencing recommendations”. - From Traffic Management to Smart Courts: China’s approach to smartcities - NESTA May 2020
ϡ. The signature characteristic of the Panopticon
is taken to be the architectural device by which a person located at the centre
of the building is able, without being seen themselves, to observe the totality
of the inmates. But in addition to this scopic
feature there is also the basic fact of confinement
without which the device could not function. It follows that in order to apply
the panoptic paradigm to modern surveillance society we should also recognise
the character of late modernity as a kind of ever-expanding prison. Here,
Foucault’s observations on visibility and discipline converge with Giorgio
Agamben’s bleak assessment that the concentration camp is the paradigm of
modern government. The pandemic has rendered this convergence more explicit
than ever, as executive orders enforcing the confinement of whole populations
have been combined with the broadest and most invasive of digital surveillance
technologies. How is it that under such conditions of expanding and
diversifying boundaries can the concept of fluid or liquid modernity still hold
true, still be the driver for ever more disruptive indeterminacy?
Ꙛ. The disruptive consequences of platform capitalism, and digital technologies more generally, on the national and international scale have been – claims the late French philosopher Bernard Stiegler – a massive degree of desocialisation, where digital technical systems have overtaken social systems. The results being balkanisation, anger, disaffection and political and social disintegration. The situation in the Western world is, according to his description, a state of fact that awaits its state of law. This perceptive point is perhaps key to understanding what China is doing with its recent drive towards containing development of digital culture within a legal framework, and the fate they are hoping to avoid.
"The broad range of policy goals projected on the system explains why what is generally translated as “social credit” is not a clearly and legally defined concept. Documents and discussions of the system contain a set of terms that range from financial creditworthiness (征信) to broader trustworthiness, law-abiding behaviour, or even moral values such as honesty and integrity (诚信/守信). [ ... ] The vagueness of “social credit” has enabled China’s local authorities to use the catchphrase to enforce (often unrelated) policy priorities through highly localised initiatives”. - Merics China Monitor - Report on Social Credit system 2021
֎. There is seemingly an ambiguity at the cultural level between common actuarial procedures for assessing levels of risk, say for the purpose of granting loans, and decision making regarding the political or moral trustworthiness of individuals. The same can be said for other key terms involved in China's tech/market/social control nexus, especially Fǔbài (腐败), which isn't just corruption in the legal or political context but has a special emphasis on the sense of something rotten in the character of the person or institution; a definition that would deserve to be revived in the West if corruption were not already the waters in which we swim. Then there is Fǎzhì (法治), which the CCP like to translate as Rule of Law both for their documents and increasingly at the UN, but in fact is better rendered as rule BY law, law as an instrument with value insofar as it achieves an effect, but without any inherent virtue, and certainly without the great historical baggage of Law in the West emanating (literally) from the confluence of Christian theology and NeoPlatonic metaphysics.
“Accelerate the advancement of social credit legislation and improve the punishment mechanism for dishonesty. Standardize the list of targets for punishment for dishonesty. Clearly formulate the basis, scope of application, punishment standards and relief mechanisms in accordance with laws and regulations, and protect the legitimate rights and interests of citizens and enterprises while strengthening punishment for dishonesty”. - Plan for the Construction of Rule of Law in China (2020-2025)
ϡ. The fragmentary nature of the social
credit system and the strategic ambiguity which exists around the system’s key
concepts allows traditional personal authority in the form of local and party
state officials to pursue their own agenda, and, as such, undermine any general
claim as to the system's strengthening of the rule of law in either the
conventional sense, or the extended and highly centralised version pursued by
the CCP. Additional harmonisation and standardisation is severely needed if the
social credit idea is not to end up as a patchwork of local loyalty schemes
with varying degrees of efficiency and participation.
“It is important to note that the SoCS remains an extension of the existing legal and
administrative system under party-state control. The official limitation to “enforcement of
laws” does not prevent overreach and human rights violations:
First, the SoCS enforces all laws and regulations. This includes repressive ones such as censorship regulations, or those that lead to discriminatory treatment of companies. It also enforces the growing list of regulations aimed at tackling anti-social conduct, standardising and restricting civic behaviour in line with “socialist core values”, as proclaimed by the leadership. There is therefore an acute risk of “credit” being randomly applied with disproportionate punishments, undermining the party state’s claim that the SoCS would establish fairness and transparency”.
- Merics China Monitor - Report on Social Credit system 2021
Ꙛ. In other parts of the world the aims of
the SoCS could fall under the more bureaucratic categories of industry
standardisation and regulatory alignment. However, the broader moral and social
connotations of trustworthiness and integrity within the Chinese system of
socialist core values, and the official nature of the SoCS means that the
boundary separating economic activity and personal lifestyle is extremely
porous, allowing the CCP to promulgate social credit as a general concept
supporting their version of the Rule of Law in concert with the Rule of Virtue,
and overall social cohesion.
“It is also a necessary condition for promoting the establishment of an objective, fair, reasonable and balanced international credit rating system, To adapt to the new situation of globalization and control the urgent requirements of the new pattern of globalization”. - Guidelinesof Social Credit System Construction (2014-2020)
“Chinese economists have proposed that a multi-dimensional approach under which lenders consider regulatory compliance metrics in addition to financial metrics could form the basis of a new Chinese-led financial credit rating model. Thus, in the realm of global finance, it is possible that over the long-term, the Chinese Social Credit System could present a challenge to U.S.-led sovereign debt and corporate bond rating models developed by S&P, Moody’s, and Fitch Ratings”.- China’s Corporate Social Credit System - Trivium China Project report Novermber 2020
֎. The dream of a totally harmonised society has at its apex the great historical sovereign of world history, now reduced to the status of a Fisher King. Wounded, impotent, unable to control the forces swirling all around him, he retreats to the lake where he fishes day after day, awaiting the one who can heal him by asking the right question. Who do we imagine is more likely to end up on such an interminable fishing trip, Biden or Xi Jinping?
