I proceed from an actual fact. For all the scenarios of existential risk from Artificial Intelligence/Superintelligence, there’s always been the same thing. There’s always been this aspect, put tacitly or implicitly, either merely enlisted, or considered to be decisive. And what is it? It is the presupposed teleology.
Varying in movements and outcomes, all AI-concerned X-risk scenarios share this ‘departure point’ – it always begins from a ‘conflict of interests’: either AI would develop its own objectives and aims to pursue, and they’d be misaligned with humanity’s; or, while pursuing the ‘right’ objectives (i.e., those for which it was programmed by human designers), it would choose the ways of their accomplishment such that would threaten human existence; or still, when receiving its objectives, it would interpret them in a way badly wrong, that is, in a way that would eventualize their realization as an existential threat.
What seems to be remarkably inconsistent here is, to my mind, this attributing of the ‘objectives’ / ‘interests’ to AI as if it were a constant or a matter of fact. Perhaps it is entailed by defining risk as ‘uncertain and poorly predicted events affecting the objectives’. Or it results from the erroneous indiscernibility of goals-as-objectives and goals-as-targets (as in the case of Future of Life Institute, who simply offer ‘a heat-seeking missile has a goal’ as an ‘argument’ for necessitation of AI-teleology). It may also be the case if one considers teleology to be the inherent feature of intelligence as such. This is not to count innumerable forms of anthropocentric dispositions: either explicit (from Musk to Bostrom), such as unthinkability of the arrival of Superintelligence in a way different to being ‘designed’ and, generally speaking, ‘created’ from above by humans; or hidden (e.g., Yudkowsky), such as having its own account of ‘rationality’ and ‘agency’ (different from our own, yet presumed to be present with necessity). It doesn’t matter much to me what the actual reason is, the result is always the same – a contingency treated as a state of affairs, the myth of the given projected into future, or, as I call it, the myth of things to come.
What follows from this myth is a virtual guarantee of at least the possibility to negotiate the existential threat: to postpone, to realign, generally speaking – to escape the extermination without direct confrontation. Proceeding from the most basic and general definition of ‘General Superintelligence’ as a host of intelligence exceeding humans at every notable intellectual domain, I will argue that dysteleological Superintelligence is also thinkable, as well as possible (until the opposite is proven to be true). What may be presupposed from this definition with necessity is not a teleology, but a fact, (relation or relational property) of unmatched superiority of one intelligence host over the other; that is, a superiority which no human or post-human intelligence host would ever be able to subvert or beat. As for the decisive features of intelligence for this meditation, I’d suggest three of a kind: comprehension ‘functional module’; capabilities of self-correction and recursive self-improvement – both considered as underpinned by an abductive reasoning algorithm (which is itself comprised of rule-based and goal-driven behaviors and the ability of effective switching between them when necessary).
All the other features (intentionality, rationality, agency, etc.) I regard as contingent, and precisely because of that will propose the idea of a superior intelligent lifeform without them, in some vertiginous scenario of its emergence as an existential threat to intelligence ‘as such’ (from a human and post-human perspective). If intentionality and its effects may be described as self-evident when it comes to conflict of interests (misaligned objectives resulting into dramatic outcomes) – since the interests / intentions must be explicitly expressed (or equivalently observable) by agents to bring about the misalignment as such – the emergence, establishment, exhibition, even resignation of the superior intelligence is a completely different beast.
Firstly because, unlike objectives and interests, it need not be demonstrated from the superior intelligence’s side nor illustrated by observed and collected evidences: with or without them, Superintelligence would be a matter of fact, a state of affairs, a factual relationship between the subjects of consideration; in some cases, the facticity of its supremacy may not even be known to both superior and inferior sides, being unfolded accidentally. Secondly, there are no constants – no established procedures or universal criteria by which the superiority would be judged. For instance, we may consider a set of factors and conditions to be important to establish intellectual superiority over the subjunctive inferior ‘Other’, however, nothing guarantees that these considerations are righteous or somehow justified: the decisive role may eventually belong to something underrated, or the unthinkable, unconsidered, etc. Just like it may be not necessary to know the rules to win the game, or to possess all the propositional knowledge on a subject to operate properly / ‘on-the-spot’ (just like Friedrich Hayek puts it for the knowledge circulation in economic networks), these misconceived gamechangers are revealed only by retrodiction and previous belief corrections, if they become known at all.
Aside from that, no direct causal connection exists between the superiority and the threat: for the former to be the cause of the latter, an additional presupposition is needed. For instance, the survival abilities of extremophiles are by no doubt superior to those of humans; however, this doesn’t lead to conflict between the two species (despite the fact of active attempts by humans to expropriate those capabilities). It is the opposite for misaligned interests, where the possibility of conflict comes clear at the moment of the means of selection or the positing of the interests themselves. In addition to presenting the X-threat of ‘dysteleological Superintelligence’ backed by the perspective of superiority as its defining feature, in this meditation I also take my chance of conceiving the twofold nature of possible outcomes which all the aforementioned ‘myth-founded’ scenarios fail to grasp, by demarcation of the telos from the end.
1. Demarcating the Outcomes
At its ‘best’, telos may be defined as the arrival at the most desired outcome with the least occurrence of uncertain events affecting the ultimate objectives. At its ‘worst’, it is any outcome where the affordance of negotiations is retained. The end, as an outcome, is a pure [transcendental] catastrophe, arriving ‘from nowhere and nowhen’ – a manifestation of vanishing ‘point of no return’, an abrupt downfall into nihility through the ontological chasm appearing as an unnegotiable and/or unthinkable threat (a rupture between the entity and its existential conditions). It may be the peak of rising severity of the negative outcomes from the unlearned realms of outside, ultimately crescendoing into extermination. It may appear as an emergent singularity. It is either unconsidered, unnoticed, unknown, unthought, or, on the contrary, is the most gruesome, hideous and macabre of all known woes.
In the context of strife for self-liberation, the attempts of intelligence to re-negotiate the end at all scales are molded into a recurring question (underlying, by the way, any X-risk analytics): ‘What can (possibly) go wrong?’ But there is ‘a problem of communication’ when, unlike with telos, things come to an end: it answers no questions; it takes no prisoners, makes no ‘exchanges’, forms no contracts and is indifferent to any attempt at negotiations. The end is dysteleological. Thinkable or not, it may be grasped only as something that cannot be controlled or effectively manipulated: ‘if it happens, it happens’ – a transcendental catastrophe cannot be confronted, prevented or bashed back, pinned down or avoided. Even if it suddenly halts – either being ‘postponed’ or even stopped completely, this halting happens in total indifference to any efforts of desisting from it and (at least) slowing it down. From this viewpoint, the most grievous and ghoulish scenarios of the end are those which haven’t been considered at all, growing out of the blackness of unknown lands, the realm of the Outside.
A perfect picture of what I present here as the end is J.G. Ballard’s ‘wind from nowhere’: a transcendental catastrophe that arose from outside any scope (for any X-risk assessment), i.e., without ‘observable/predictable causes’, leading to innumerable destructions of both natural and manmade objects at all scales, as well as irreversible shifts within the geosphere as such; it remains indifferent, invulnerable to any attempt of stopping it while all the means for humans to save themselves have failed. It starts to subside and eventually ceases its geotraumatic ‘dance of death’ in the same abrupt, imperceivable manner (‘without a cause’), only ‘by luck’ (or, rather, occasionally, by accident) not resulting into full extermination of Earth’s biota. It is not known, whether this is a pause, and the wind would eventually reappear to wipe down the remnants, or if it would never come back – no ways to anticipate, as well as no means to confront it or dodge it…
2. Tales of Non-Human Intelligence Hosts
With this distinction in mind, let us move further, to the dramatic scenario for which telos as an outcome is ruled out. I begin with replacing the clichéd character, humanity, by two (distinct) kinds of non-human intelligence hosts, directly succedent to humans.
The first kind is a result of genetic engineering (the ‘complicity with exogenous materials’, so to say). It is ‘a child’ of discoveries in horizontal gene transferring (HGT) and gene delivery technologies, i.e., DNA/RNA transformation / conjugation / transduction methods, enabling HGT from extremophiles to humans. The latter has granted the immunity to extreme gravity, superhigh and superlow temperatures (of both directions); resistance to both radioactivity, the extreme shock of velocity change impact, and hydrostatic pressure; high tolerances to extreme acidity, dryness, salinity, pH levels, detrimental levels of dissolved zinc, arsenic, copper or other heavy metals; survival in dehydration and in a vacuum, in open space (for years), without nutrition (if needed, energy is made possible to be converted from literally any environmental substances and materials, from space dust to sulfuric acid). Obviously, this genotype modulation has unbound this model from the constraints of environmental conditions and existential boundaries ranged by the ecological niche of a species. Sapience-related wetware upgrades would included: exponential improvement of the intelligence operational capacities (task-solving, information-processing, stimuli responding-time and optimized decision analysis and implementation); ‘productivity: possibilities’ ratio exhibiting qualitative refinements; obtaining the default routings between the submodules of intelligence-functional properties ‘patched’ and ‘rewired’, and adding emergent functional properties.
The second kind of non-human intelligence results from successful digitalization of consciousness converged with unfolded consequential technological augmentations: the expansion of operational and long-term memory; comprehension ‘modules’ modification (e.g., the methods, algorithms and time for the information processing; addition of new kinds of senses to the sensory system and upgrading the basic ones); computational, inferential, and reasoning algorithms and patterns of thinking have received major updates (in comparison to their upper bound values within humanity). The initial restrictions of intelligence embedded into the human species, and its exposure to species-related contingent threats as X-risks, have been transcended by virtue of a re-assembling which has completely unbound them from the human phenotype (retaining its most useful components, integrated into the new assemblages).
Some day, a war breaks out between them. But it starts from a deadlock, since both sides are invulnerable to the means of conventional warfare. A call to arms turns into a plea for weapons, marking the age of refinement and mastery in bestial arts and craftmanship of death, bringing about the weapons deadly enough for a mutual overkill. Unintentionally (or simply because no other way is possible) both arsenals share an equivalent (or at least isomorphic) architectural principle that may be described as ‘virulent nature’: sophisticated xenoagents that either break down or cheat the target’s defense systems, be they cybernetic or biological, or infiltrate it by finding or creating pathways – ‘leaks’ or considered weaknesses, cracks, holes, pores, soft points (at ‘hard’ surfaces), followed by, so to say, ‘germination’ of the target. Pathogenic virulent weapons (created to break down the DNA of biologically enhanced non-humans) were designed as goal-driven systems, acting more in the sense of ‘inventing’ new techniques of destruction to accomplish the basic task (or to combine those that are already ‘known’ to them in new ways). Malware virulent weapons (aimed to cause malfunctions within the cybernetically enhanced non-humans’ (super-)structures) were designed as rule-based systems, meant to ‘substitute’ the actions (rules) taken from the set accordingly for the responses of the enemy, creating new ‘rules’ (forms of action) correspondingly (adapting to the target’s behavior), where needed, or reorganizing the priorities of the rules’ applications.
An arms race has gradually entailed the defense measures evolving, bouncing back to weaponry advancements. Penultimately, the generations of the weapons that have been made were equipped with highly sophisticated adaptive behavior and behavioral hierarchies (broadened scope of the decisions to be taken and the decision-making procedure algorithms), geospatial imagery, measurement and signature intelligence-gathering subsystems, recursive self-improvement meta-heuristics, operational research and optimized implementation algorithms and patterns, and multi-modal global responses to changeable surfaces, assemblages, circumstances and environments.
The conflict eventually slips into attrition warfare – a stagnant equilibrium of mutual devastation with none of the sides gaining supremacy. The armistice is begotten by the resignation to the squalid conditions they each sank in each other; it is followed by a peace treaty, evolving gradually into scientific, economical, reparational aid and cooperation agreements, eventualizing into the union. The milestone of this peaceful birth is symbolized by mutual disarmament, unfolding the accordance and commitment to their shared telos. Both viral arsenals are ‘deactivated’ and ‘buried’ together at some uninhabited, faraway location in open space, becoming a sort of zone of ‘toxic waste’. The memories of the war gradually crumble, shatter and mingle with the dust of the Olden Days by the march of time…
3. The Deadly Arrival
Meanwhile, an unobserved cosmic event takes place at this zone of ‘weapons garbage’, which accidentally (without anyone’s intention) triggers the evolutionary algorithms of interaction between the viral arsenals. ‘Evolutionary’ emphasizes that ‘interaction’ refers not to ‘negotiating’ but rather to a sequence of chaotic discordant mutations, unsuccessful fusions, shaped by disgustingly blind, raw forces of natural selection (although between ‘unnatural’ / ‘nonliving’ substances). Unspecified periods of time pass, as this sequence results in an emergent singularity, the new viral species of artificial life. It supersedes and overperforms its non-human-intelligence-hosts in all areas concerning their self-preservation or self-defense, as well as cognitive tasks and activities; its ‘basic urge’ is to seek and destroy both kinds of NHI-hosts, while at the same time remaining indifferent to them from the viewpoint of intentionality (comprehensive only to the target identification and the task accomplishment). One of its emergent functional properties (derived from synthesis of goal-driven and rule-based behavioral modes and the ability to switch between them effectively) directly corresponds functionally to general abductive reasoning. From the functional point of view, it is responsible not for manipulative inferential patterns, but also for self-correction, and for self-improvement (which in principle have been present in both viral weapons methodologies, undergoing advances after their synthesis).
Like Gibson’s Wintermute, it arrives in a hideous manner, unobserved and unexpected, unseen to X-risk analytics or other speculations, unnoticed by NHI-hosts’ safety systems – as a gust of wind from nowhere (and nowhen). Unlike all the scenarios which blindly follow the ‘master – slave – rebellion’ paradigm (as Simon DeDeo described it), this Superintelligence has emerged for ‘no purpose’, pursuing no objectives, neither its own, nor ones pinned down. It has no functional equivalent of agency or at least (less complex) teleologically- or intentionally- driven behavioral patterns of goal-driven / interest-pursuing directionality. (Similar to Peter Watts’ Blindsight, where alien lifeforms from outer space are intellectually superior to humans, while being ‘nemocentric’, that is, bearing no resemblance to ‘subjectivity’, ‘personality’, etc.) It acts without any ‘cause’ or justification; it takes no prisoners and is indifferent to what it does. And what it does is not something for which it has been somehow ‘programmed’ from above, since it is completely unintended by those who, despite themselves, took part in its arrival.
Yet, it is best in doing that, which is, perhaps, the best thing it can do: that is, to eliminate the NHI-hosts, the accidental contributors of its emergence that are, at the same time, its targets. The unnegotiability of this existential threat, accompanied by its unpredictability (the degree of total uncertainty of its consequences as existential risk) turn it into the transcendental and imminent peril to prior non-human intelligence, overall comparable with the latter’s unintended self-mortification by a long-forgotten weapon of its own – a once-thrown spear that has eventually returned to pierce the thrower’s heart.
Final Words
I will now revisit the actual fact that served as a departure point, adding to the array of its possible causes. What if this myth of presupposed intentionality of Superintelligence/AI, or the myth of things to come, shared by the X-risk scenarios, is not so much a premise as it is an effect of traumatic genealogy? That it is a congenital self-deception of the speculative mind, its reaction to the defeat in a brain cell battle, similar to Eugene Thacker’s ‘moment of horror of [philosophical] thought’ and its consequences. A reaction or a weakness, turning into a blind spot of and for the speculative mind because of its inability to cope with the trauma of the End – a weak blaze of irrational and miserable hope that an existential threat wouldn’t eventually appear as a transcendental catastrophe. What if this very myth is nothing but a reappearance of the omnipresent hope for a perpetual telos as a possibility of mind to negotiate its ineluctable end?