Cognition Social Interaction Communication and Convergent Technologies

Philip Rubin, National Science Foundation'

I am impressed with how my teenaged daughter and her friends marshal current technology for group communication. Most of their use of this technology, including AOL "Instant Messaging," email, cellphones, and transportation, is for social interaction.

The technological world twenty years from now will be a very different one. Prognostication is not my specialty and seems like a dangerous enterprise; however, I can talk about some things that we can do to help shape our future. Some of these are merely extensions of current technology and our current abilities, but the critical considerations I want to mention are well beyond our current capabilities. The unifying vision for these comments is the merging of cognition and communication.

Imagine a future without cellphones, laptops, PDAs, and other cumbersome devices. Going beyond the existing smart environments described by Reg Golledge and his colleagues (see Golledge essay in Chapter B and Loomis essay in Chapter C), we will soon be moving through a world in which we are continuously broadcasting, receiving, storing, synthesizing, and manipulating information. We will be embedded in dynamic, continually changing communicative clouds of data signals that communicate information about place, location, language, identity, persona, meaning, and intent. How will social and personal interaction be restructured in this new world? How can we use cognition to help us fly through these clouds effectively? I will leave the first question to experts like Sherry Turkle (see essay in Chapter B), who have thought long and hard about them, and will, instead, briefly mention where we need to go in the area of cognition.

The approaches that we will use for social and group communication in twenty years will rely on a variety of cognitive considerations. Here is a partial listing.

• Intent. Neuro-nano technology, such as neural interfaces, will enable us to provide the direct guidance of choice and selection of behaviors based on cognitive intent. This will allow for binary and graded choice directly under cognitive control.

• Adaptation. Communication and knowledge systems will learn and adapt based upon an understanding of human behavior. Fundamental to this is a serious consideration of the adaptive landscapes that characterize this new communicative, social world and how they mesh with our cognitive capabilities.

• Perception, analysis, and action. Embedded and distributed systems and sensors will be enhanced by our fundamental understanding of human perceptual and analytic behavior and skills, including the following: auditory and visual scene analysis (Biederman 1995; Bregman 1994); the visual control of action (Loomis and Beall 1998; Turvey and Remez 1979; and Warren 1988); multimodality, including vision, audition, gesture, and haptic sensing and manipulation (Cassell et al. 2000; and Turvey 1996); spatial cognition (Golledge 1999); linguistic analysis, including

1 The views expressed in this essay do not necessarily represent the views of the National Science Foundation.

statistically-based natural language processing and analysis (Biber, Conrad, and Reppen 1998; and Manning and Schutze 1999); and language use (Clark 1996).

• Selection. Cognitive selection, prioritization, and organization of information are essential if the information/communication clouds of the future are not to overwhelm us. Critical abilities to filter, organize, restrict, or enhance information will rely on cognitive selection, personal preference, and automatic adaptation that will evolve based on previous behavior, patterns, choices, and preferences.

• Semantics. Meaning will guide the performance of the systems of the future; it will be grounded by a variety of factors, including ties to the real world and its structure and requirements, biases, and personal and social needs. Semantically based systems will make communication more flexible, effective, and natural.

• Self-organization and complexity. Increasingly, approaches to understanding human cognition, perception, and behavior will rely on more sophisticated analytic, statistical, and conceptual tools. Examples include nonlinear dynamical systems; self-organization, complexity and emergent behavior; complex adaptive systems; agent-based modeling; naturalistic Bayesian-networks that include subjectively-based categorization and representation; and the like (Holland 1995; Kauffman 1995, 2000; Kelso 1997; Varela et al. 1991; and Waldrop 1992. See also essay by J. Pollack in Chapter B).

What is needed to make these changes happen? First, they rely on the presumed convergence of nano-, bio-, info-, and cognitive technologies. Obviously, some of these changes are already on the way, particularly in the realm of nanotechnology, information technology, communication systems, and engineering. Progress has been significantly slower on the cognitive end, for a variety of reasons. The problems to be tackled in areas such as cognition and perception are often broad and very complex. These difficulties have been compounded by the need for noninvasive approaches for probing and exploring the human cognitive system. Mind and behavior have usually been explored from the outside. In essence, the cognitive system has been treated as a "black box" that can be probed in a variety of ways. Often such approaches have been conducted independent of the constraints imposed both by human physiology and by the environment. Other techniques that are more invasive, such as lesion studies, work with a system that is not in its normal functioning state. The difficulties in probing this system hamper our understanding of it.

Recent technological advances have raised the possibility of obtaining additional data about neural functioning during normal cognitive activities that can help to inform and constrain our theorizing. New advances in functional neuroimaging, including fMRI, PET, and MEG, coupled with the detailed study of neural circuitry and the theoretical advances in a number of areas, hold great promise (Gazzaniga et al. 1998; Lyon and Rumsey 1996; Marantz et al. 2000; and Posner and Raichle 1997). Functional imaging has the potential to be the telescope that lets us observe the universe of the mind. The goal is not to localize behavior but to have a tool that can potentially aid in the understanding of a massively complex system and in exploring brain behavior. However, these techniques will not be adequate on their own. They must be used in the context of a basic understanding of human cognition, perception, learning, development, and so forth.

Unfortunately, the fundamental understanding of how cognition works in areas such as spatial and cognition perception (auditory, haptic, and visual) has been massively underestimated. These are complex problems that will require significant basic research. For example, we need to understand our interaction with the world before we can fully understand the role the brain plays in helping us navigate this world. Before we can fully understand the role of the brain in vision, we must have a better depiction of what is available in the world for us to see. Before we fully understand the role of the brain in language, we need a clear theoretical understanding of what language is, how it is structured and organized at a variety of levels. Considerable progress that has been made in areas such as these points to the promise of theory-based research coupled with emerging technologies for visualization and simulation.

The "intelligent" systems of the future that will be fundamental to group and social communication will be far removed from the expert systems and the ungrounded formal systems of the artificial intelligence (AI) of past years. Instead, they will rely on the gains made in the fundamental understanding of the psychology, biology, and neuroscience of human behavior and performance, including cognition, perception, action, emotion, motivation, multimodality, spatial and social cognition, adaptation, linguistic analysis, and semantics. These gains will be enhanced by consideration of human behavior as a complex adaptive biological system tightly coupled to its physical and social environment.

It remains to be seen whether the national support is forthcoming that is necessary to make substantial progress in these areas of cognition that hold such promise. However, if we hope to see truly convergent technologies leading to smart devices and the enhancement of human behavior, communication, and quality of life, we must tackle the difficult problems related to cognition on the large scale more commonly seen in areas such as computer science and engineering. Now is the time to seriously begin this effort.

0 0

Post a comment