Major Objectives Towards Meeting Overall Goals of Visual Verbal Language Research

The achievement of the six goals described above will obviously require intermediate advances on a number of fronts to achieve specific objectives:

1. Diagram an entire branch of science with stand-alone diagrams. In many of the newer introductory textbooks in science, up to one-third of the total space consists of diagrams and illustrations. But often, the function of scientific diagrams in synthesizing and representing scientific processes has been taken for granted. However, recent research cited above (Mayer 2001, Chandler and Sweller 1991) has shown how stand-alone diagrams can significantly enhance learning. Stand-alone diagrams do what the term indicates: everything the viewer needs to understand the subject under consideration is incorporated into one diagram or into a series of linked diagrams. The implication of the research is that the text in the other two thirds of the textbooks mentioned above should be distributed into diagrams.

"Stand-alone" is obviously a relative term, because it depends on previous learning. One should note here that automatic prerequisite linkage is one of the easier functions to imagine being created in software packages designed to handle linked diagrams. One doesn't actually have to take too large a leap of imagination to see this as achievable, as scientists are already exchanging PowerPoint slides that contain many diagrams. However, this practice frequently does not take advantage of either the stand-alone or linked property.

Stand-alones can be done at a variety of styles and levels of illustration. They can be abstract or detailed, heavily illustrated or merely shapes, arrows, and words. They can contain photographs and icons as well as aesthetically pleasing color.

Imagine a series of interlinked diagrams for an entire field of science. Imagine zooming in and out — always having the relevant text immediately accessible. The total number of diagrams could reach into the tens of thousands. The hypothesis of this idea is that such a project could provide an extraordinary tool for cross-disciplinary learning. This prospect directly impacts the ability of interdisciplinary teams to learn enough of each other's fields in order to collaborate effectively. And collaboration is certainly the key to benefiting from converging technologies.

Imagine, further, that using and sharing these diagrams were not dependent on obtaining permissions to reproduce them, which is one of the least computerized, most time-consuming tasks a communicator has to accomplish these days. Making permissions automatic would remove one of the major roadblocks to the progress of visual language and a visual language project.

Then, imagine a scientist being able to send a group of linked, stand-alone diagrams to fellow scientists.

2. Create "periodic" table(s) of types of stand-alone diagrams. Once we had tens of thousands of interlinked diagrams in a branch of science, we could analyze and characterize all the components, structures, and functions of all of the types of diagrams. This would advance the understanding of "chunks of thinking" at a fine-grained level. This meta understanding of diagrams would also be a jumping-off point for building software tools to support further investigations and to support diagramming of other branches of science and the humanities.

3. Automatically create diagrams from text. At the present moment, we do not know how to develop software that enables the construction from text of a wide variety of kinds of elaborate diagrams. But if the stand-alone diagrams prove as useful as they appear, then an automatic process to create diagrams, or even just first drafts of diagrams, from verbal descriptions will turn out to be extremely beneficial. Imagine scientists with new ideas of how processes work speaking to their computers and the computers immediately turning the idea into the draft of a stand-alone diagram.

4. Launch a project to map the human cognome. In the Converging Technologies workshop I suggested that we launch a project that might be named "Mapping the Human Cognome." If properly conceived, such a project would certainly be the project of the century. If the stand-alone diagram project succeeds, then we would have a different view of human thought chunks. Since human thought-chunks can be understood as fundamental building blocks of the human cognome, the rapid achievement of stand-alone diagrams for a branch of science could, thus, be regarded as a starting point for at least one major thrust of the Human Cognome Project (Horn 2002c).

5. Create tools for collaborative mental models based on diagramming. Ability to come to rapid agreement at various stages of group analysis and decision-making with support from complex, multidimensional, visual-verbal murals is becoming a central component of effective organizations. This collaborative problem-solving, perhaps first envisioned by Douglas Engelbart (1962) as augmenting human intellect, has launched a vibrant new field of computer-supported collaborative work (CSCW). The CSCW community has been facilitating virtual teams working around the globe on the same project in a 24/7 asynchronous timeframe. Integration of (1) the resources of visual language display, (2) both visual display hardware and software, and (3) the interactive potential of CSCW offers possibilities of great leaps forward in group efficiency and effectiveness.

6. Crack the unique address dilemma with fuzzy ontologies. The semantic web project is proceeding on the basis of creating unique addresses for individual chunks of knowledge. Researchers are struggling to create "ontologies," by which they mean hierarchical category schemes, similar to the Dewey system in libraries. But researchers haven't yet figured out really good ways to handle the fact that most words have multiple meanings. There has been quite a bit of progress in resolving such ambiguities in machine language translation, so there is hope for further incremental progress and major breakthroughs. An important goal for cognitive scientists will be to produce breakthroughs for managing the multiple and changing meanings of visualverbal communication units on the Web in real time.

7. Understand computerized visual-verbal linkages. Getting computers to understand the linkage between visual and verbal thought and their integration is still a major obstacle to building computer software competent to undertake the automatic creation of diagrams. This is likely to be less of a problem as the stand-alone diagram project described above (objective #1) progresses.

8. Crack the "context" problem. In meeting after meeting on the subject of visual-verbal language, people remark at some point that "it all depends on the context." Researchers must conduct an interdisciplinary assault on the major problem of carrying context and meaning along with local meaning in various representation systems. This may well be accomplished to a certain degree by providing pretty good, computerized common sense. To achieve the goal of automatically creating diagrams from text, there will have to be improvements in the understanding of common sense by computers. The CYC project, the attempt to code all of human common sense knowledge into a single database — or something like it — will have to demonstrate the ability to reason with almost any subject matter from a base of 50 million or more coded facts and ideas. This common-sense database must somehow be integrally linked to visual elements.

0 0

Post a comment