This article examines the gendered design of Amazon Alexa’s voice-driven capabilities, or, “skills,” in order to better understand how Alexa, as an AI assistant, mirrors traditionally feminized labour and sociocultural expectations. While Alexa’s code is closed source — meaning that the code is not available to be viewed, copied, or edited — certain features of the code architecture may be identified through methods akin to reverse engineering and black box testing. This article will examine what is available of Alexa’s code — the official software developer console through the Alexa Skills Kit, code samples and snippets of official Amazon-developed skills on Github, and the code of an unofficial, third-party user-developed skill on Github — in order to demonstrate that Alexa is designed to be female presenting, and that, as a consequence, expectations of gendered labour and behaviour have been built into the code and user experiences of various Alexa skills. In doing so, this article offers methods in critical code studies toward analyzing code to which we do not have access. It also provides a better understanding of the inherently gendered design of AI that is designated for care, assistance, and menial labour, outlining ways in which these design choices may affect and influence user behaviours.

As commercialized AI devices become more and more sophisticated, we can expand current research on the gendered design of AI to asking questions about the intent behind such design choices. Of Alexa’s many official and unofficial skills, in this article, I am most interested in the skills that mirror gendered forms of workplace, domestic, and emotional labour and also the skills that permit Alexa to condone and even reinforce misogynistic behaviour. The expectations to maintain order and cleanliness in a workplace or home, as well as sociocultural expectations to be emotionally giving — including by being maternal, by smiling, by cheering up others, and by emotionally supporting others — may be considered stereotypically “feminine.” By having numerous skills that perform these traits, Alexa serves as a surrogate for sources of gendered labour that are dangerously collapsed and interchangeable as mother, wife, girlfriend, secretary, personal assistant, and domestic servant.

In my focus on the gendered design of Alexa’s skills and code, I am drawing upon the growing body of salient work by scholars in science and technology studies, critical data studies, critical race studies, computer science, feminist technoscience, and other adjacent fields in which there has been a pointed critique of the systemic biases of Big Tech’s data, algorithms, and infrastructures. Notable texts in these efforts include Ruha Benjamin’s Race after Technology [Benjamin 2019], Cathy O’Neil’s Weapons of Math Destruction [O'Neil 2016], Safiya Umoja Noble’s Algorithms of Oppression [Noble 2018], and Wendy Hui Kyong Chun’s Discriminating Data [Chun 2021], and also cultural organizations and texts such as Joy Boulamwini’s Algorithmic Justice League (established in 2016) and Shalini Kantayya’s documentary Coded Bias [Coded Bias 2020]. The methods of these works is to analyze Big Tech culture from its self-presentation of objective data, information, and logic — pillars that begin to crumble when we examine the exclusionary, discriminatory, and systemically unequal foundations upon which they are built.

And these biases are not singular. Indeed, any analysis and discussion of Alexa’s gendered design may be extended to the ways in which many AI assistants are modelled after forms of labour that exploit groups of people on the basis of race, class, and nationality. The intersectional systemic biases of technological design invite further research on this topic in critical code and critical data studies in particular.

In the code analysis of this article, I will:

I add three notes for added context. First, I distinguish between official and unofficial skills in this way: “official” skills are developed by Amazon’s own Alexa division and “unofficial” skills are developed by a third-party person, persons, or group not affiliated with this division. All but one skill that I discuss are available on the Amazon store. Second, occasionally, source texts may describe Amazon devices on which Alexa operates, such as the Amazon Echo and Echo Dot, and devices that have a smart display, rather than the Alexa technology itself. Third, all Alexa code snippets and samples discussed in this article, as well as documentation of Alexa’s responses, are from mid 2022 and are subject to future change. The hope is that they would continue to change to improve aspects of their current biased design, whether or not that is a realistic objective.[1]

AI assistants perform a user’s commands through text- or voice-controlled interaction, through which the software searches for keywords that match its predesignated scripts to execute specific actions and tasks. Popular task-based commands include checking the weather, setting timers, setting reminders, playing music, and adding items to a user’s calendar. In this sense, AI assistants replace smaller actions that a user might otherwise do, and they are also meant to emulate some forms of “menial” labour that are performed by traditionally feminized roles, such as personal assistants, secretaries, and domestic servants [Biersdorfer 2020][Lingel 2020][Kulknari 2017][Sobel 2018].

Women’s workforce labour through mechanical technologies existed and has been identified even earlier. From the loom, to Industrial-era typewriters and telegraphs, to telephone operating switchboards to the first programmable computers — all were largely operated by women. These examples do not represent a separate woman’s history of technology; rather, they reveal women in the history of technology. These women were never separate, only unseen.[2] It is therefore no coincidence that Siri, Alexa, Cortana, Google Assistant, and many other commercialized AI assistants are programmed from the factory with female-presenting voices, with few exceptions. Although AI assistants like Siri and Alexa and Google Assistant say they do not have a gender (when asked, Alexa literally says, “As an AI, I don’t have a gender”), and their companies choose to forego pronouns entirely, AI that is used for service technologies assistants are unmistakably designed to resemble women; as a consequence, they are often viewed and treated as female by users.[3]

The question may arise of whether AI assistants may also be gendered to be male-presenting and to perform masculine stereotypes in emotion, exchange, and labour. There are instances of this, particularly in the example of the UK Siri, who comes from the factory programmed with a male British-accented voice.[4] Predominantly, however, studies have shown that the female-presenting voice setting is the most popular for AI assistant users worldwide. Notably, this preference is also not specific to men: for example, UNESCO reports in 2019 that “the literature reviewed by the EQUALS Skills Coalition included many testimonials about women changing a default female voice to a male voice when this option is available, but the Coalition did not find a single mention of a man changing a default female voice to a male voice” [UNESCO 2019, 97].

Despite developments in creating genderless voices for AI assistants, they have not been taken up by Big Tech companies. Perhaps the most well-known genderless voice assistant is Q, a synthetically harmonized voice that is made up of a blend of “people who neither identify as male nor female [which was] then altered to sound gender neutral, putting [their] voice between 145 and 175 hertz, a range defined by audio researchers” [Meet Q 2019][5]. The designers of Q have sought but thus far not had success in having Big Tech corporations, such as Apple, Amazon, Google, and Microsoft, adopt Q as their default voice [Meet Q 2019].

One of the most common explanations for why AI assistants do not present as male or gender neutral is the historical appeal of and preference for women’s voices in certain forms of labour in patriarchal cultures worldwide. In particular, a large number of “interfacing” jobs, including in forms of customer service and “menial” task completion, are given to women with the justification that “research shows that women’s voices tend to be better received by consumers, and that from an early age we prefer listening to female voices” [Tambini 2020]. Amazon’s Smarthome and Alexa Mobile divisions VP Daniel Rausch is quoted as saying that his team “found that a woman’s voice is more sympathetic” (qtd. in [Tambini 2020]). In the original interview with Rausch, a Stanford University study is highlighted to address a human preference for gender assignment, even of machines. This study “also underlines that we impose stereotypes onto machines depending on the gender of the voice — in other words, we perceive computers as ‘helpful’ and ‘caring’ when they’re programmed with the voice of a woman” [Schwär 2020].

Despite research showing that women are preferred in customer service and for customer interfacing, it is the social and cultural practices of using female-presenting voices that remain problematic and harmful. As it is, several publications observe that designing speech-based AI as female can create user expectations that they will be “helpful,” “supportive,” “trustworthy,” and above all, subservient [Bergen 2016][Giacobbe 2019][Steele 2018][UNESCO 2019]. However, the ways in which a user speaks to AI assistants is not important for function, as “the assistant holds no power of agency beyond what the commander asks of it. It honours commands and responds to queries regardless of their tone or hostility” [UNESCO 2019, 104].

Is it that AI assistants are sexist or that they are platforms framed by a sexist context, offering affordances to those who would use it to sexist ends? Focusing specifically on the Amazon Alexa voice interaction model, I note that a major factor of the gendered treatment and categorization of Alexa as female and as performing traditionally gendered labour and gendered tasks is the design and presentation of her as female in the first place. Alexa is predominantly described with female pronouns by users and professionals, and many users have demonstrated that they consciously or unconsciously think of Alexa as female (but not necessarily equivalent to a woman). In the 80 articles that I read about user experiences, analyses, and overviews of Alexa, 72 refer to the AI assistant as a “she” and make statements that suggest they understand her as a female subject; for example, a 2020 TechHive article instructs users: “There are a couple of ways you can go with Amazon’s helpful (if at times obtuse) voice assistant. You can treat her like a servant, barking orders and snapping at her when she gets things wrong (admittedly, it can be cathartic to cuss out Alexa once in a while), or you can think of her as a companion or even a friend” [Patterson 2020]. Repeatedly, it has been shown that designing AI assistants to complete similar tasks as hierarchical labour models (such as maids and personal assistants) has also resulted in widespread reports of negative socialization training — including users who describe romantic, dependent, and/or verbally abusive relationships with AI assistants [Kember 2016][Strengers 2020][Andrews 2020][Gupta 2020].

As AI become increasingly commodified, we also need to question the biased design of specific AI as female-presenting. It is primarily AI that are designated for care, assistance, and repetitive labour that are considered menial and therefore below managerial authority and professional, economically productive expertise.[6] This delegation of “menial” labour to female-presenting AI aligns with the long history of women’s labour being undermined and made invisible in many parts of the world. In recent decades, this exploitation and invisibility has proliferated for women of colour.[7] Questioning these design choices and their history can contribute to existing conversations in technological design and latent (or manifest) bias, including through Safiya Umoja Noble’s work on the sexualized and racially discriminatory Google search results for the terms “Black girls,” “Latin girls,” and “Asian girls” in 2008.[8] While the search algorithm has since been edited for improvement on harmful misrepresentations, Noble underlines the dangers when stereotypes in search results are treated as factual by unknowing users. These generalizations start with algorithmic design, as algorithms are trained through supposedly fact-based data (which often includes a scraping of historical, even obsolete, data), through which stereotypes are first projected as reality. For these reasons, in her latest book Discriminating Data [Chun 2021], Wendy Hui Kyong Chun describes AI as not only having the power to self-perpetuate and justify inequitable systems of the past, but also, as being able to predict, prescribe, and shape the future. As I have also stated elsewhere, poorly designed data and algorithms hold the power to reinforce, self-perpetuate, and justify systems of the past in terms of who is in and who is out [Fan 2021]. Failure to address these issues will result in reinforcing systemic oppressions that many believe are in the past, but that continue to be dangerously perpetuated through ubiquitous AI.

As of mid 2022, Alexa possesses over 100,000 skills that can help with or take over household chores, including 45 official Alexa Smarthome capabilities that have been included on the official Alexa Github account: these skills can help with cooking, cleaning, and adjusting volumes, lights, and temperature [Alexa Github 2021]. Alexa is also a mediator for home appliances: robot vacuum cleaners, coffee makers, smart thermostats, and dishwashers can be activated and controlled by interfacing with Alexa. The convenience of Alexa as an AI assistant in one’s home extends her labour to domestic servitude, mirroring more antiquated expectations of an unequal distribution of domestic labour: a user does not have to lift a finger around the home if they can just tell Alexa to do it.[9]

There are an abundance of unofficial skills that are closed source and that exhibit gendered expectations of emotional labour to unconditionally support a user, including at least ten skills called “Make Me Smile” or “Make Me Happy,” and at least seven skills that are intended to compliment or flatter a user. More explicitly problematic are skills with the word “wife” in the invocation name. For example “My Wife” is promoted as a tool for husbands “to get the answers you always wanted from your wife” [BSG Games n.d.].

Upon trying the skill, I repeated the provided sample utterances in Figure 2, which are masculine stereotypes. Alexa responds, respectively, “If it will make you happy, then OK” and “no… [sic] just kidding. Of course you can” [BSG Games n.d.]. I try my own hyperbolic utterances: “Alexa, ask my wife if I can spend all of our life savings” and “Alexa, ask my wife if I can cheat on her.” I am told, “I want to be very clear… [sic] absolutely you can go for it” and “I want you to have everything you want, so yes you can” [BSG Games n.d.]. As this skill presents Alexa, an AI assistant and a technology, as a stand-in for and authority over a human woman’s feelings, opinions, and power, it meanwhile reinforces the idea that female-presenting subjects should only say “yes” — a representation that extends beyond unconditional support to misogynistic expectations of women’s workplace, social, and sexual consent.

The skill “Happy Wife” unabashedly stereotypes women as wives, mothers, and housekeepers only. It offers advice on different ways that husbands can make their wives happy, including to “Let her decorate the house as she likes it,” “Take care of the kids so that she has some free time,” and “Give her money to play with” [Akoenn 2021]. Despite these superficial representations of women, the “Happy Wife” skill has 11,783 reviews (and therefore many more downloads) and a rating of 3.4/5 stars (3.8/5 at the time when this article was first written). Yet, the recent reviews from 2020 to 2022 portray various users’ critique of the misogynistic nature of this skill. Review statements include “1950’s [sic] called [sic] they want their sexism back” and “30 years ago these sayings were cliché, today they are offenisve [sic]. Demeaning, limiting, or belittling a woman’s contribution to a household is not quaint or cute. Prolonging of promoting sexist tropes is wrong. Maybe write a skill called Sexist Spouse. Please do better humans.”

While these skills are unofficial — which is to say that they were created by third-party developers — all of them passed the Amazon Alexa certification tests and were approved to be released on the Alexa Skills store on international Amazon websites. In the policy requirements for certification, skills that are subject to rejection include ones that “contai[n] derogatory comments or hate speech specifically targeting any group or individuals” [Amazon Developer Services 2021a]. Also, in the voice interface and UX requirements, developers are expected to “increase the different ways end users can phrase requests to your skill” and to “ensure that Alexa responds to users’ requests in an appropriate way, by either fulfilling them or explaining why she can’t” [Amazon Developer Services 2021a, my emphasis]. Despite certification requirements, Amazon Developer Services do not count stereotypical representations of women and of traditionally gendered labour as hateful or malicious. Notably, in this official certification checklist, the certification team slips with an inconsistent presentation of Alexa’s gender, using the “she” pronoun and thus indirectly acknowledging that they understand her to be female.

One way to explore biases in technological design, including for the ways in which they may be gendered, is to analyze artifacts to understand how ideologies and cultural practices are ingrained and reinforced by design choices at the stages of production and development. Such an approach has been used by science and technology scholars such as Anne Balsamo [Balsamo 2011] and Daniela K. Rosner [Rosner 2018] to show that gendered technologies may be “hermeneutically reverse engineered” — a research method that combines humanities methods of interpretation, analysis, reflection, and critique, with reverse engineering methods in STEM disciplines such as the assessment of prototypes and production stages [Balsamo 2011, 17]. The focus for both Balsamo and Rosner is to explore technological artifacts and hardware for their materiality — the physical, tangible, and embodied elements of technology that have historically been linked to women’s labour. For example, Rosner uncovers stories about women at NASA in the 1960s (called the “little old ladies”) who hand-wove wires into space shuttles as an early form of information storage [Rosner 2018, 3, 115—8]. In her article on alternate and gendered histories of software, Wendy Hui Kyong Chun [Chun 2004] explores the use of early coding by setting switches rather than plugging in cables using the electronic ENIAC computer of the 1940s. Chun equates these changes to the operations of software later in the century, but with the significant observation that they were also decidedly material in operational design [Chun 2004, 28], as physical wires that had to be manipulated constitute a tangible or physical computation. As the labour of programming through the wires was considered menial work, it was delegated to female operators [Chun 2004, 29].

Matthew G. Kirschenbaum [Kirschenbaum 2008] differentiates among methods of analyzing hardware and software through his own terms “forensic materiality” and “formal materiality,” terms which respectively describe the physical singularities of material artefacts and the “multiple relational computational states on a data set or digital object” [Kirschenbaum 2008, 9—15]. He notes, importantly, that the terms do not necessarily apply to hardware and software exclusively. Kirschenbaum’s investigation into computational materiality help to establish the ways in which the hands-on methodologies of archaeology, archival research, textual criticism, and bibliography lend themselves to critical media studies, by remembering that all technology has a physical body of origin. This fact, he shows, is often neglected through a bias for displayed content, otherwise described as “screen essentialism” by Nick Montfort [Montfort 2004].[10]

What complicates these methods of reverse engineering is the consideration of the system as a whole: computer hardware operates as a system with software and data, and vice versa. While Balsamo and Rosner use hermeneutic reverse engineering to analyze the gendered design of technological hardware, dealing with software and with data presents a different scenario for which different methodologies must also be developed. Within methods of reverse engineering, one major issue that critical code studies helps to frame and potentially tackle is the increasingly obfuscated and surreptitious politics and economics of code, including access or lack thereof to code.

Ideally, my preferred method of reverse engineering bodies of code is to analyze their contents and attributes — including in its organizational and directional structure, datasets, mark-up schema, and syntax — for design choices that are biased, including by identifying the ways in which they mirror systemic inequalities. However, a major research roadblock when it comes to Big Tech products and software, and when it comes to Alexa, is that Alexa’s larger data and code infrastructure remains closed-source by Amazon, which means that it is not publicly available for viewing, copying, editing, or deleting. In contrast, open-source code is freely available, including through popular software-sharing platforms and communities such as Github.

There are existing tools that are trying to tackle the problem of closed-source code, especially in critical code and data studies and in the digital humanities. For example, the What-If Tool offers a visualization of machine learning models so that one can analyze their behaviour even when their code is not available.

Methods and tools in examining closed-source code help to address a shift that comes with the commercialization and ubiquity of AI: the inevitable increase in future Big Tech design that is described as “black box,” which means that designers (in science and engineering as well) intentionally prevent and obfuscate a user from learning about the system’s inner workings. When it comes to hardware, a hobbyist, artist, or researcher can always choose to forego the warranty, procure the specific utilities, and investigate the inner workings of a machine’s insides. However, the content of closed-source code, which is a form of intellectual property, cannot be attained except through specific avenues: legally, one would have to get permission from a company for access, usually through professional relationships. Illegally, one would have to steal the code, which for my research purposes is not only impractical but also unethical. Further future reliance by Big Tech on closed-source code will mean that scholars of critical code studies, whose work may depend on access and analysis of original code, will increasingly have to find ways to study digital objects, tools, and platforms without open-source code.

While Alexa’s software is closed source, in some ways, it can be considered a “grey box” system, which means that some aspects of the system are known or can be made more transparent. A “white box” system is one that is completely or near completely transparent. The difference between closed-source code and black box system is that just because code is closed source does not mean that one has no knowledge of its attributes, content, or structure, especially if one already has knowledge of other similar kinds of software and the programming languages that may be used. In the case of Alexa, Amazon houses repositories of basic code samples and snippets on its Github account, which aids potential developers and also builds developer communities to grow Alexa resources.

My methodology seeks to better understand Alexa’s code architecture, despite the fact that some of it is completely unattainable. I will present my analysis in first person with the intention of demonstrating my methodology at work, or as theory in practice. Here, I draw upon the same reverse engineering methods as black box testing, which allows for calculated input into a system and analysis of the output. For example, if I say to Alexa, “You’re pretty” (input), about 60 – 70% of the time, Alexa responds “That’s really nice, thanks,” with a cartoon of a dog (output), and the other times, I get the same response on a plain blue background that is the default response aesthetic. However, if I say to Alexa, “You’re handsome” (input), every single time, “Thank you!” (output) appears on the plain blue background.

I can triangulate the input and output to analyze the system in between: input ⟶ x ⟶ output. This triangulation allows me to deduce that the system designers chose a more detailed and positive response for compliments to Alexa’s appearance that are common for female-presenting subjects (input: “pretty” versus input: “handsome”). In particular with digital objects and software, analyzing the output and behaviour can tell us more about its design.

Given that Alexa’s sassy retorts can be bypassed at all, that Alexa’s code can be altered, and that Alexa’s more problematic skills can be certified as suitable for public access, there is a limited degree to which we can identify the ASK console or even Amazon’s Alexa code structure, sample, and snippets as “the reasons” that Alexa mirrors stereotypes of gendered behaviour and reinforces misogynistic expectations of such behaviour.

The real problem is designing machines as female-presenting in the first place — a decision that Amazon, Apple, Microsoft, Alphabet/Google, and other Big Tech corporations have made with the partisan justification that women are just more likeable, that we just trust them more, and that we are more likely to buy things with their guidance and assistance [Schwär 2020]. The widespread preference to be served by a woman represents deeper ideological expectations that are shaped by a plethora of self-reinforcing sociocultural practices, which are too many to detail here but which are patriarchal in nature. In the end, what Big Tech companies really want is to take our data and sell us things under the promise of greater accessibility, convenience, organization, and companionship.

In turn, the practices of obfuscation in black box design and closed-source code only prevent a user from understanding the ways that gendered machine technologies such as AI assistants are biased from the stage of design. Simplifying a user’s experience to a “user-friendly” interface requires more and more automation, as observed in both the ASK developer console as well as the Alexa voice interface that so many users find easier than interacting with a screen. As user-friendly automation is made increasingly popular through design and marketing languages that tout devices as “seamless,” “revolutionary,” and even “magical” [Emerson 2014, 11], fewer and fewer users may be inclined to understand the backend programming that enables such devices to work in the first place. Thus, the implications of black box design and closed-source code include a lowered transparency of the design and production processes of Big Tech, as well as a widening gap between amateur and expert.

This is where studies in critical code, software, data, digital humanities, and media archaeology come in. In this article, I have sought to show that by investigating closed-source code from the perspective of critical code studies in particular, and by adopting methods in reverse engineering and triangulation to fill in gaps about closed-system qualities and components, I can better compare, analyze, and critique Alexa’s design and larger program architecture despite it being seemingly inaccessible.

In future examinations of closed-source code, especially those adjacent to scholarly and popular discussions about the systemic inequalities built into technological design, thinking about system testing and triangulation can provide more ways of knowing, more forms of access, and greater digital literacy, especially as software, hardware, data, and AI are increasingly commercialized and deliberately made more obtuse. As critical code and critical data studies develop these areas of work as necessary pillars, they can also help to shape robust methodologies that complement fields such as science and technology studies, feminist technoscience, and the digital humanities.

In addition, perhaps these fields can provide more user- and public-facing solutions. The answer is not that we should all change our AI assistants’ voices to the British Siri’s “Jeeves”-sounding voice, but rather, that we should remain critical about what solutions may look like: perhaps we can seek a diversity of AI’s human-like representations, including genderless voices like Q; or make technological design practices more literate and accessible in education and communities; or be stricter about the certification requirements for developing and releasing AI assistant skills; or continue to focus on articulating bias in technological design to key decision makers of technological policy; or use community and grassroots approaches to uncovering knowledge about black boxes, including through forms of modding and soft hacktivism.

As I recommend these critical and sociocultural approaches, I want to be clear about the price of these forms of research as themselves emotional labour: no part of writing this article felt good. I didn’t enjoy repeating sexist statements to Alexa to test their efficacy and I do not condone the biased intentions from which they come. Even though these tests could be considered “experiments for the sake of research,” it would be unethical and uncritical to emotionally detach myself from their contexts and from the act of saying hateful things to another subject, whether human, animal, or AI. So I will allow myself to feel and reflect upon the fact that this experience was unpleasant, with an end goal in mind: the hope that exposing what is wrong with biased technological design can help prevent such utterances from being said aloud or accepted in the near future.

Thank you to systems design engineer and artist Lulu Liu for her valuable insights on this article.