Technosemiotics: languaging

In the fourth seminar, we discuss human–machine interactions through the concept of languaging as an embodied process with Marie-Theres Fester-Seeger. Find the title and the abstract for the talk below.

Marie-Theres Fester-Seeger
Postdoctoral Fellow, Postdoc Network Brandenburg, European University Viadrina

The introduction of ChatGPT turned the world around. A large language model that appears to ‘talk,’ ‘understand,’ and ‘know’ incited a great debate between educators, researchers, and laypeople about how such machines will determine how people think, write, and learn in the future. To which degree are these concerns, however, justified? While set up to appear human, these machines process large amounts of human data turned into tokens and single entities to produce a ‘human-like’ output based on complex stochastic operations. In doing so, language in speech-enabled machines is processed for a machine’s operation and treated as something separate from human living beings. Traditional linguistics understands language as an abstract system consisting of specific grammatical rules and structures. As a result, linguists treat words as a priori entities and, thus, independent of people as human living social beings. In such a view, language is a ‘tool’ to mitigate between an external and internal reality. This view might be helpful in putting ‘language’ in a machine, but once one turns to observations of how people do language, one notices that language is a highly embodied social activity. Most importantly, one notices that language is not a code.

Against such code-like views in which meaning appears stable across contexts, I introduce the languaging perspective to rethink how we treat language in AI. The languaging perspective brings language to people. With the focus on what people do, the perspective holds that language needs to be understood as embodied, dialogical, ecological, and multiscalar. In simpler terms: the languaging perspective sees language as bodily activity. When we engage with others, what we say goes beyond the word.  Subtle changes in gaze, body posture, hand gestures, and, among others, facial expressions accommodate what we say. As we speak with others, we not only closely observe the movements of our interlocutors, but our actions are also impacted by our surroundings and wider ecology. Tracing utterances to articulatory gestures, one, therefore, needs to ask what permeates our bodily activities. As social living beings, people are embedded in distinct and various linguistic communities. Thus, what we say and how we say it emerges from our recursive engagement with others within such communities. In doing so, we bring forth specific norms, values, and distinct realities bound to these communities. Our social embedding determines how we perceive and react to the ‘words’ of others: be that in real-time engagement or in distributed engagement via a machine. Treating language as human bodily activity results in that words cannot be understood as something independent of a person and stable but as something that, through bodily coordination with others, is constantly brought forth. Thus, language cannot exist outside of people, it only exists through people.

The languaging perspective allows us to explore the doing of language in two ways: a) rethinking the role of training data in large language models and b) how users will react to and engage with such models. It gives rise to questions like: “Where does the data come from, what kind of data is used, and most importantly, who is doing the labeling of the data? These questions hint towards to role of the human living being in the machine and how their activities determine how users will perceive what they ‘see’ later from the machine. Further, it gives due weight to the user and how they coordinate with ‘disembodied’ data that once was created by real living human beings.

With this perspective, I propose in this presentation that human engagement with machines cannot be perceived as communication in which a mutual understanding is being created but as a wider distributed human-to-human engagement. When users in real-time engage with machines, they coordinate with past anticipations of programmers and the lived words of other people that appear to be stable entities.

---

Marie-Theres Fester-Seeger is a Postdoctoral Fellow funded by the Postdoc Network Brandenburg and working at the European University Viadrina in Frankfurt (Oder), Germany. She received her Ph.D. at the University of Southern Denmark, where she built her great research interest in distributed, embodied, dialogical, ecological, and systemic approaches to language and cognition. Viewing cognition and language as human activity, she investigates the greater question of ‘how do people ‘talk’ to machines?’ In this research project, she uses cognitive ethnographic methods to investigate people’s engagement with smart speakers (such as Alexa and Google Home) in their home environment. With a great research interest in human lived experience, she proposes to rethink digital technology as part of a ‘coming together’ of human influences.  She asks in this project how people think with and through machines and, in addition, how human-machine engagement changes a person.