- 1 This piece was inspired by and examines the fictional world of “Upload”’s first season, which aired (...)
1In April 2020, Amazon released a new comedy series called “Upload.” The show’s first season extrapolates a future in which advances in computation and neuroscience have led to the successful simulation of human consciousness in silico.1 In this world, individuals can pay to be “uploaded” into a series of competing digital afterlives. When uploaded, human consciousness is converted into stored memories and executable code, which can be edited, reset, throttled, or even deleted depending on each upload’s membership plan and payment status. By portraying this world, the show breaks the boundaries between reality and virtual reality, consciousness and artificial intelligence, and even life and afterlife, entangling various legal questions in novel ways.
2The neuro-simulation technology depicted in the show is, in fact, already in development. The OpenWorm project, for example, used the fully mapped neural system of a nematode (C. elegans) to “‘copy-paste’ its brain into a computer, creating a virtual copy of the organism that reacts to stimuli the same way as the real thing” (Micu). The uploaded nematode can move around, hunt bacteria, and even respond to simulated physical stimuli. It does so relying on its virtual nervous system, which is simulated down to the level of individual ion channels on the nematode’s 302 neurons (Sarma et al.)! Such successes, however, do not mean that human uploads are imminent. While petascale supercomputers may one day be able to support “cellular level simulations of the complete human brain,” mapping the various “elements, interactions, emergent properties, and time scales” of the brain remains a “massive challenge” (Markram, et al. 40).
3Seemingly rising to this challenge, a company called Humai came onto the scene in 2015 promising to transfer people’s consciousness into artificial bodies within 30 years (Thubron). The company, however, was not a true tech venture. Commentators later noted that the company was simply “an effective way to rob people” and that the ambitious future it imagined “is about as likely as a zombie apocalypse” (Moye).
4While it is possible that much more limited “Brain-Computer Interface (BCI)” technologies will continue to make progress in the foreseeable future, the upload technologies featured on the show will almost surely not exist anytime soon (Gonfalonieri). Therefore, it may seem strange to give “Upload” any sort of legal analysis. The show’s world-building and narrative, however, provide an opportunity to explore—through a possible distant future—the legal, social, and business decisions being made today. It serves as an exaggerated mirror through which we can engage in “conscious psychological exploration and social criticism” of the legal norms currently taken for granted (Brantlinger, 41).
5Specifically, this short analysis will examine three topics. First, we aim to leverage the show’s advanced technology to question the distinction between human consciousness and artificial intelligence in the context of copyright and contract law. In so doing, we highlight how legal personhood may be doctrinally denied to “uploaded” human minds, even if they are—for the sake of argument—as capable, conscious, and creative as living human minds. We then build on this conclusion to demonstrate how the power imbalance between individuals and technology companies will continue to exacerbate the problem of contracts of adhesion and surveillance capitalism (even after physical death!). Lastly, we seek to demonstrate how an unyielding, unquestioning, myopic desire for digital rights (at all costs) may lead to the unintentional loss of crucial non-digital rights.
6“Upload” raises complex questions in almost all areas of law, from tort liability to consumer protection, to estate planning. One of the most jarring, however, is how “Upload” depicts a future in which the current legal regimes of copyright and contract law may unintentionally reinstate a virtual form of servitude. Specifically, it demonstrates how current law may, in the face of technological change, conspire in such a way that robs individuals of agency, autonomy, and ultimately legal personhood.
7To understand how this is so, consider first the existing law governing ownership of AI creations. In the United States, the Copyright Office has adopted a stringent “human authorship” requirement according to which the Office will only “register an original work of authorship, provided that the work was created by a human being” (Compendium of U.S. Copyright Office Practices, Ch. 300 §306). For the Office, the “fruits of intellectual labor” are limited to those “founded in the creative powers of the mind”—the living human mind that is (Compendium). This position was reaffirmed in the now infamous “monkey selfie” case concerning a photo taken by a crested macaque named Naruto. In that case, the Ninth Circuit concluded that “a monkey—and all animals, since they are not human—lacks statutory standing under the Copyright act” and therefore is unable to seek damages or injunctive relief against copyright infringement (Naruto v. Slater).
8Even in jurisdictions that seem more welcoming to the idea that non-human generated content may be copyrightable, the ownership rights continue to belong to a company or living human individual. This is the case with AI-generated content whose rights are granted to the company or individual employing the AI and not to the AI itself. In 2020, for example, a court in Shenzhen, China, recognized that stories produced by an automated software called Dreamwriter possessed sufficient originality to qualify for copyright protection. The court, however, granted the rights over the stories not to the Dreamwriter AI but to the Chinese tech titan Tencent that created it (Sawers).
- 2 “[The United States Congress shall have power] To promote the Progress of Science and useful Arts, (...)
9For our current purposes, such limits on the entitlement to and assignment of copyright protections may seem benign. This restriction, however, smuggles with it an undertheorized assumption about creativity and agency. It assumes that copyright law is only meant to incentivize and protect the legal interests of currently living biological humans. In the United States, such assumption is suspect even at the constitutional level. Doctrinally, federal copyright protection exists solely under the government’s constitutional power to “promote the Progress of Science and useful Arts” in society (U.S. CONST. Art. 1 §8 Cl. 8).2 Constitutionally, then, copyright law was a “distinctly utilitarian construct” from the very beginning (Stadler, 611). By limiting the authorship requirement to living humans, the law “denies the incentive of copyright to an increasingly large group of works that are indistinguishable in substance and public value from works created by human beings” (Denicola, 286). Ultimately, though, the effectiveness of modern copyright doctrines in fulfilling their constitutional mandate is an empirical question outside the scope of this short piece.
10More interesting, however, is a theoretical question raised by modern copyright doctrine. The law’s requirement of a living human author assumes that a discrete qualitative difference must exist between living human brains and computers. But where exactly is the boundary between an AI and the human mind? If a human brain is fully simulated, are its works worth any less to society, promoting the arts and sciences? If not, are simulated minds still less worthy of incentives?
11The uploads in the show challenge the categorical distinction between human minds and algorithms by offering a clear edge test. Nathan, the protagonist of the show “Upload,” is now an uploaded human consciousness. When describing his current situation to a living human, he noted that “[i]n a sense both of our consciousnesses are simulations, mine on a silicon computer and yours on a computer made of meat, your brain” (“Upload”, Season 1, Episode 7, “Bring Your Dad to Work Day,” 12:07). If uploads do indeed mimic all of “the synaptic connections” in the original human’s brain cells as well as the complete “thousands of terabytes” of the human’s memory data (Episode 3, “The Funeral,” 3:12), is the uploaded intelligence “artificial” in any meaningful sense merely because it is running on silicon?
12While the “Upload” showrunners purposefully do not answer this question, they affirmatively portray a world in which the law also fails to address this critical issue. The law in “Upload” relies on inertia to deny the legal existence (or even relevance) of uploaded humans. Whether consciously or not, they seem to have premised their narrative on the likely scenario that law will lag behind technological innovation. Much like Naruto the crested macaque and the Dreamwriter AI, the uploads are not allowed to work, contract, or directly earn money in any way, as that would “driv[e] down wages for us meat folks” (Episode 4, “The Sex Suit,” 1:45.)
13More importantly, the scanned uploads are themselves “owned.” Nathan, when describing his relationship with his living girlfriend Ingrid says, “it’s definitely changed the relationship. I mean, now she literally owns me” (Episode 4, “The Sex Suit,” 6:53), later noting that “I’m just this plaything for Ingrid, there when she wants me. [Ingrid] control[s] all my purchases, what I do, what I eat” (Episode 5, “The Grey Market,” 1:30). While the interpersonal dimensions of this reality are explored at length in the show, the legal ramifications are not.
14One such ramification is in the law of copyright. Throughout the show the various uploads of Lakeview, the virtual world in which Nathan lives, continuously produce content: David hits cinematic golf shots, Dylan devises innovative new pranks to play on the local groundskeepers, and Nathan himself constantly writes and performs thoughtful and artistic prose. All of these creations, however, seem to be owned by Horizen, the company behind Lakeview, or by the Horizen account holder, a living individual paying Horizen to host the upload. The uploads are treated much like the Dreamwriter AI.
15Uploaded human minds are not entitled to copyright protections nor are they allowed to contract around this limitation. They are stuck. Under fear of permanent deletion, they may be forced to engage in endless intellectual (or computational) labor. Despite their cognitive abilities and emotional responses, which are by presumption equivalent to those of living humans, they lack any and all entitlements of legal personhood.
- 3 A contract of adhesion is a “standard-form contract prepared by one party, to be signed by another (...)
- 4 Surveillance Capitalism, as defined by Shoshanna Zuboff, “includes the acquisition of ‘behavioral s (...)
16Why does this lack of personhood matter? Given the historical and theoretical connections between personhood and human rights, one can imagine the myriad problems this situation may create (Vatter & de Leeuw). In this piece, we will highlight just two. First, is the growing problem of contracts of adhesion in the face of life altering infrastructural technologies.3 Second, is surveillance capitalism’s role in transforming human minds into products available for purchase.4
17Regarding the role of contract law, it is true that each and every upload ‘agreed’ to this loss of personhood in Horizen’s terms of service. As living humans (and legal persons), they were each informed of Horizen’s policies and consented to the relevant terms and conditions before being uploaded. Under current California law, for example, such terms—even of indispensable internet services—are routinely upheld as enforceable contracts. Not even “oppression or surprise due to unequal bargaining power” are enough to render a contract of adhesion unconscionable and therefore unenforceable (In re Google Assistant Priv. Litig., 837). Presumably, enforcing such contracts is thought to somehow promote autonomy and the ability to rely on the promises of others. As Hanoch and Heller noted, “contract serves autonomy by enabling people to legitimately enlist others in advancing their own projects and thus it expands the range of meaningful choices people can make to shape their own lives” (66).
18To render a contract of adhesion unenforceable, current law in California requires that the contract contain “substantive unconscionability” which focuses on “‘overly harsh’ or ‘one-sided’ results” ((In re Google Assistant Priv. Litig, 838). Notably, these results must be biased towards one of the parties to the contract. In the universe of “Upload” it seems clear from Horizen’s successful business model that courts do not treat the uploaded individuals as legal persons and therefore do not recognize their mistreatment as substantive unconscionability towards one of the contract parties. Ultimately, “dead person[s] cannot sue” (Repko v. Our Lady of Lourdes Medical Center, 958) so uploads are not only be forever bound by Horizen’s terms of service, they also lack basic legal protections if or when Horizen breaches any of its own terms.
19This is even more alarming because Horizen advertises such loss of legal personhood and legal recourse as a heavenly afterlife. It also targets individuals at an extremely vulnerable time: when they are dying. Nathan, for example, is forced to accept Horizen’s terms of service (without reading them) while dying on a hospital gurney. When he begins to question whether he should accept these terms, the Horizen representative taunts him: “Hit accept! ... You can’t use our service without agreeing to the terms of service. Have you never gotten an app ever?” (Episode 1, “Welcome to Upload,” 14:13). Under pressure from his loved ones and feeling humanity’s inherent “fear of mortality” (Cozzolino, et al. 422), Nathan gives in and accepts the terms.
20In the process and for an empty promise of eternal life, Nathan forfeits legal personhood, the ability to contract, the right to own any property or intellectual property, and any other future legal recourse. After scanning the minds of dying individuals, Horizen has the uploads locked in a golden cage where it not only collects monthly fees for hosting these poor unfortunate souls, but also has the legal power to make further profit off all the fruits of their labor. The uploads have only their Horizen account holders to turn to, who themselves may at any moment, and for whatever reason, decide to delete, freeze, or alter their uploads without recourse.
21In fact, the relationship between Horizon, the uploads, and the living account holders who pay for the uploads’ maintenance raises the second problem inherent in the power dynamics of the show: surveillance capitalism. In the modern digital marketplace, “if you are not paying for the product, you are the product” (Hövermann). This phrase has become a modern proverb. Some have argued, however, that this axiom is already outdated. Nowadays, “[n]o matter how much you pay, under current US law, you’re the product if the company’s privacy policy says so” (Scholz).
22This is taken to the extreme in “Upload.” Horizen presents itself as offering a service for the individual uploaded, but uploads are not the ones who pay. They are not the ones to whom Horizon is obliged. Rather, Horizen must bow to the pressure of living individuals who continue to pay for the maintenance of each upload. The simulation of a loved one’s mind is the product being sold as an experience or service to living individuals. If you are that simulated mind, you are the product being sold in a much more concrete sense than ever before. One’s very existence can be extinguished whenever the consumers who pay for the service—the living individuals—cancel their subscription.
23In this way, “Upload” demonstrates what can happen when individuals are reduced to only two roles: consumers or products. If you cannot be a consumer (because you are dead) and you cannot be a product (because there is no market for whatever it is you have to offer) then you quite simply (and literately) cease to exist.
24Ultimately, this situation can lead to even further abuse by Horizen. Not only can it sell your existence as a service to living individuals it can also engage in product placement, adding “contextual cues” to “shape emerging habits” of the living individuals who come see and interact with you (McKinsey & Co.). If the entire world in which uploads exist is a manipulable digital environment, Horizen could easily “nudge” its consumers—the living individuals—towards even more profit-making purchases and activities (McKinsey & Co.). In this scenario, the fact that uploads’ minds are as powerful and as feeling as those of living humans is irrelevant. Simulated minds are merely digital services used to sell advertisements or engage in other profit-earning activities, all while collecting more and more data.
25In spite of these terrifying possibilities, the allure of an eternal digital afterlife was too much for society to bear. Human rights defenders on the show, for example, were so blinded by the fact that wealthy individuals get to experience digital immortality, that their immediate goal was to democratize this technology.
26According to the activists, everyone should have the right to Horizen’s digital life extension. If Horizen’s technology is seen as a form of health care or life support, this conclusion might make sense. As we have shown, however, the reality is much more complicated. Instead of fighting for the rights of uploads, activists are fighting for the rights of humans to be uploaded (Episode 4, “The Sex Suit,” 11:44). In other words, they are fighting for the purported human right to freely enter what amounts to a labor camp and retail shop for consciousnesses.
- 5 In 1791 Jeremy Bentham wrote the “Panopticon or Inspection House” which described “a new principle (...)
27An obsessive focus digital rights blinds these activists from the fact that what they are fighting for is actually for the right to be imprisoned. Horizen has, after all, created the perfect prison. Much like in Jeremy Bentham’s Panopticon,5 by inducing “a sense of conscious and permanent visibility” (Foucault , 201) Horizen mastered a new form of power and control. Entrapped in a simulated complex space, the uploads are subject to “instrumental coding” of their minds (153). This is a new form of “political anatomy,” in the Foucauldian context, a mode of ordering by omnipresent surveillance and a feeling of futility and perpetuity (138).
28In this age of cyber instability new digital rights are cropping left, right, and center. From a “right to be forgotten” (Lee) to a “right to cybersecurity” (Kilovaty) to a “right to internet access” (Tully). But perhaps the biggest lesson to be learned from the “Upload” is that we must resist our tendencies to immediately embrace and apply human rights law, and reassess “the ethical foundations underlying” the very field exploring the “outer limits” of the discourse and its potential pitfalls (Dror-Shpolianky & Shany). Especially when the subject matter to be regulated is a paradigm-shifting technology, we should consider leaving some of our legal paradigms at the door.
29“Upload” portrays many different visions for the future, but perhaps none more profound than the idea that society will inevitably subjugate its AI creations. What sets “Upload” apart from any other show about technological servitude (and robotic uprising), like “Westworld” for example, is that in “Upload” human consciousness is the AI. It is our minds, our memories, our thoughts and feelings, which are being scanned and uploaded, not an algorithmic version of them. In the world of “Upload,” we are so doomed by technological determinism that we voluntarily put our very consciousness on the stake.
30But as if that’s not enough, after waiving all of their rights in the process, uploaded individuals clamor for the technology that would allow them to “download” their consciousness back into printed flesh:
“We are about to witness a human consciousness reinserted into a printed clone body… They are effectively reversing the upload process, bringing us closer to endless corporeal life… many would agree the opportunity to live again in real life is worth any price… the team is now cueing up the reinsertion process. This will carry thousands of terabytes of data into the clone. And that data is that’s his consciousness… The machine is a reverse scanner reorganizing the synaptic connections in his brain cells” (Episode 3, “The Funeral,” 0:00–4-22).
31Nothing in this process, however, grants such downloaded individuals their original personhood. It is perhaps the ultimate testament to the fact, regardless of medium, that a person without personhood isn’t a person at all. The uploads and their living next of kin seek to reverse their own doing, translating consciousness back into a synaptic map of (synthetic but physical) brain cells. By perusing the move from the physical to the digital and then back to the physical again, “Upload” demonstrates how humanity is forever chained to its antiquity, even in the face of groundbreaking innovation and technological advancement.
32In this piece, we analyzed the law of “Upload.” We hope that this discussion can serve as a launching pad for a more nuanced conversation about what is artificial in artificial intelligence, what is virtual in virtual reality, and what is the digital in digital rights. We argue that becoming early adopters of a new reconceptualized language around “us” and “them”, the “self” and the “other” can perhaps future proof our society from the cyborg perils that await us in the undoubtfully arduous and tumultuous road ahead. This way we may possibly be able to trade a looming Horizen for at least a slightly better horizon.
Stadler, Sara K., “Forging a Truly Utilitarian Copyright,” Iowa Law Review Vol. 91 (2005), pp. 609-671.
Sarma, Gopal P., et al., “OpenWorm: overview and recent advances in integrative biological simulation of Caenorhabditis elegans.” Philosophical Transactions of the Royal Society B, Vol. 373, No. 1758, (2018), https://doi.org/10.1098/rstb.2017.0382.
Flux RSS
