Beyond Just the Words

Abstract

Human communication is embodied -- facial expressions, gestures, and prosody (tone of voice and rhythm) add layers of important information to just the words used by a cognitive agent. In fact,just the use of words often fails in expressing the thoughts expressed easily by non-verbal modes of communication. Actions such as the inflection of the voice or the lop-sided smile convey our emotive and expressive thoughts more effectively than just words. Thus, the "something extra" mentioned in the conversational example is constituted by the layers of non-verbal communication. As the bandwidth of communication increases from email to in-person, each layer of non-verbal communication is added, enabling us to have a deeper understanding of the conversation. In this paper, I propose that non-verbal modes of communication are integral in our understanding of human language and thought. Moreover, I argue that a systematic understanding of non-verbal communication produces a novel way of looking at human communication. The incorporation and systematic understanding of non-verbal communication breaks the chain of the traditional syntax-semantic centric viewpoint of language and thought, removing the blinders that have long narrowed the epistemological field of vision. . . . I divide this paper into 5 sections. In section 1, I present viewpoints of prominent scholars like Rene Descartes, Daniel Dennett, Jerry Fodor, and Paul Churchland on the issue of language, representation, and thought. Using their works, I exhibit the lack of attention paid to the roles of non-verbal communication. Then, I consider the reasoning behind this negligence towards non-verbai communication and propose three plausible causes. Next, I begin section 2 by briefly addressing the importance of emotion in cognition and decision making. I proceed to present empirical work done in Affective Computing and Human-Computer Interaction (HCI) and stop at "checkpoints" to consider their philosophical implications. Using the empirical work presented, I also consider both the practical and philosophical implication of the improvement of social signal processing abilities of individuals with Autism Spectrum Condition (ASC). Social signal processing can be roughly defined as the ability to infer one's mental states from one's emotional (or affective states). Next, I present further empirical research that pays equal attention to both verbal and non-verbal communication. In section 3, I extend the philosophical implications mentioned beforehand by focusing on the motivations behind using computer models to gain a systematic understanding of non-verbal communication. Moreover, I focus on the power of an embodied language on thought. I delineate how a systematic understanding of non-verbal communication augments our understanding of human-human interaction. In section 4, I consider objections to the current arguments and provide appropriate responses to them. Prior to the conclusion of the paper, section 5 deals with future implications of the current work both from a societal and philosophical point of view. [From introductory section]