Connectionism and Neural Networks

The form and structure of neurons and the observation of neuroanatomy was made available to optical microscopy by Italian anatomist Camillo Golgi in 1875. He found a method by which, seemingly at random, a very few neurons in a brain region became stained in their entirety, with all their branches becoming evident. The cytoplasm of selected neurons take up a brightly colored stain and are thus exposed against the tangled morass of less visible cells. Golgi's contemporary, Spaniard Santiago Ramon y Cajal (1955) used the Golgi stain to investigate nearly every part of mammalian nervous systems. His neuroanatomy texts are still classic and he resolved the question of whether nerves were separate entities like all other cells, or part of a continual network. He also demonstrated that the complex connections among neurons are not random, but highly selective and specific.

Subsequent generations of neuroanatomists and neuroembryologists including Roger Sperry (1969, 1980) have emphasized the meticulous detail with which neural connections are formed, and initially supported the concept of the nervous system as a pulse logic device, superseding the older concept of the brain as a switchboard of reflex centers. Adrian's discovery that neural pulse coding was limited to a frequency/intensity coupling shifted emphasis to connectionism per se. Because the electric signals the brain uses to communicate among cells were seen as stereotyped, or nearly identical, they were viewed as symbols which do not themselves resemble the external world they represent. The consensus of opinion regarding brain functions shifted to a concept in which the shape of a neuron and its fiber origins and destinations determine mental representation as part of a neural network. The meaning of stereotyped signals was thought to be derived from specific connections of neurons.

The high degree of precision with which nerve cells are connected to each other and to different tissues in the periphery became emphasized in the connectionist concept. Orderliness of connections formed during development became viewed as essential for integrating mechanisms and representation of information in some way. The nervous system appeared to be constructed as if each neuron had built into it an awareness of its proper place in the system. The question of mental representation refocused on the embryological development during which synaptic connections were formed. During development, the neuron grows towards its target, ignores some cells, selects others, and makes permanent contact-not just anywhere on a cell but with a specified part of it. Further, neurons behave as if they were aware when they have received an appropriate synaptic connection. When they lose their synapses they respond in various ways. For example, neurons or muscle fibers disconnected from their neuronal contacts may die, but first develop "super-sensitivity" to their chemical neurotransmitter by means of an abundance of new synaptic membrane receptor proteins. Cell death or dysfunction induced by denervation occurs due to a loss of morphologicial "trophism," a neural function which conveys structural and functional material and information by cytoskeletal axoplasmic transport. Atrophy, dystrophy and spasticity of muscles and limbs which occur after strokes and other nervous system insults are examples of the loss of normal trophism. Microtubules and other cytoskeletal proteins responsible for trophism and axoplasmic transport also allow growth and extension of neuronal axon growth cones, dendrites and dendritic spines and thus play a key role in neural connections. Super-sensitivity, spasticity, atrophy and dystrophy are examples of "synaptic plasticity," changes in connections or connection strength among neurons which are relevant to brain and bodily function.

Association of learning with ongoing alteration of synaptic function was considered by several late 19th century writers and was popularized due to the Pavlovian and behaviorist influences of conditioned responses. Pavlov (1928) proposed that conditioned reflexes are established by forming new connections between cortical neurons that receive a conditioned stimulus (one accompanied by a reward or punishment) and those that receive an unconditioned stimulus. Once a new pathway was established the unconditioned stimulus would acquire the same power of evoking the response that only the conditioned stimulus originally possessed. Pavlov's idea of a new connection became fused with Donald Hebb's (1949) concept of plastic changes in synaptic efficacy to correlate with learning. Because it was believed that new fibers and therefore new synaptic connections, could not grow in adults, long term facilitation of anatomically preformed, initially nonfunctional connections became the likely alternative. This implied that at birth there existed a vast number of redundant and ineffective synaptic conditions which became "selected" during the individual's lifetime of experience. An alternative view is that, at birth, excitations can pass between any two points of the CNS through a random network of connections. As maturation, experience, and learning occurred, synaptic activity gradually sculpted usable patterns by suppressing unwanted interconnections.

Thus the connectionist brain/mind became viewed as one of two types of systems: a blank slate ("tabula rasa") in which acquired learning and internal organization result from direct environmental imprinting, or a "selectionist" network chosen from a far vaster potential network. Selectionists believe that the brain/mind spontaneously generates variable patterns of connections during childhood periods of development referred to as "transient redundancy," or from variable patterns of activity called prerepresentations in the adult. Environmental interactions merely select or selectively stabilize preexisting patterns of connections and/or neural firings which fit with the external input. Selectionists further believe that, as a correlate of learning, connections between neurons are eliminated (pruning) and/or the number of accessible firing patterns is reduced. Supporting a selectionist viewpoint is the observation that the number of neurons and apparent synapses decreases during certain important stages of development in children. However, this reduction could be masking an increase in complexity among dendritic arborizations, spines, synapses, and cytoskeleton. The selectionist view is also susceptible to the argument that new knowledge would appear difficult to incorporate.

On the assumption that the basic mode of learning and consciousness within the brain is based on synaptic connections among neurons (connectionist view) several attempts to model learning at the level of large assemblies of interconnected neurons have been made. Hebb pioneered this field by proposing that learning occurred by strengthening of specific synaptic connections within a neuronal network. This led to a concept of functional groups of neurons connected by variable synapses. These functional groups as anatomical brain regions have been described by various authors as networks, assemblies, cartels, modules or crystals. These models are aided by the mathematics of statistical mechanics and have been rejuvenated due to the work of Hopfield (1982), Grossberg (1978), Kohonen (1984) and others who drew analogies between neural networks within the brain and properties of computers leading to applications for artificial intelligence. They emphasized that computational properties useful to biological organisms or to the construction of computers can emerge as collective properties of systems having a large number of simple equivalent components or neurons with a high degree of interconnection. Neural networks started as models of how the brain works and have now engendered chips and computers constructed with neural net connectionist architectures utilizing hundreds of computing units and linking them with many thousands of connections. Hopfield (1982) remarks that neural net chips can provide finely grained and massively parallel computing with:

a brainlike tolerance for fuzzy facts and vague instructions. Some of the general properties you get in these systems are strikingly like ... properties we see in neurobiology ... . You don't have to build them in; they're just there ... .

Neural networks had formally appeared in Rosenblatt's (1962) "perceptron" model of the 1950's and 1960's. Perceptrons created enthusiasm, but failed to reach their potential due to limitations of the model and its mathematics. Al experts Marvin Minsky and Seymour Papert (1972) wrote a harshly critical review which discouraged neural net research until Hopfield's resurgence in the 1980's. Hopfield introduced an energy function so that information in a neural net circuit would settle into a series of stable energy states much like rain water falling on mountains flows through valleys into lakes and rivers. Depending on the rainfall, an information state (i.e. memory, conscious image, thought) would be a given watershed pattern. Hopfield's neural nets are loosely based on aspects of neurobiology but readily adapted to integrated circuits. The collective properties of his model produce a content addressable memory (described by a phase space flow of the state of the system) which correctly yields an entire memory from any sub-part of sufficient size. The algorithm for the time evolution of the state of the system is based on asynchronous parallel processing. Additional emergent collective properties include some capacity for generalization, familiarity recognition, categorization, error correction, time sequence retention, and insensitivity to failure of individual components. Hopfield nets and similar models are best categorized with the "tabula rasa" view of learning in which the initial state is taken as a flat energy landscape which becomes progressively contoured, eroded and complicated by direct interactions with the environment.

A selectionist approach to neural net theory has been taken by Jean Pierre Changeux, who pioneered description of allosteric interactions among proteins. Turning to the brain/mind, Changeux and colleagues (1984, 1985) have proposed a model of learning by selection based on the most recent advances in the statistical mechanics of disordered systems, namely the theory of spin glasses. Spin glasses are materials which are not crystalline, yet whose atoms possess a high degree of similar neighbor relationships and a finite number (i.e. 2) of magnetic spin states influenced by their neighbors. Aggregates of "like spin" states beget similar states among neighbors. Consequently the spin states of atoms in a spin glass can be viewed as a network (or cellular automaton) much like a collection of neurons in a nervous system. Changeux also uses terms from mathematical chaos theory like basins and attractors to describe the states to which the spin glass model evolves. Unlike the blank slate approach, the brain's initial state is viewed by Changeux as a complex energy landscape with an exuberance of valleys typical of spin glasses. Each valley corresponds to a particular set of active neurons and plays the role of a prerepresentation. An input pattern sets an initial configuration which converges towards a valley whose entry threshold is lowered by synaptic modification. Starting from a hierarchical distribution of valleys, the "lowest" valleys (sea level fjords) would correspond to maximal comprehension, ultimate answer, best correlation. The learning process is viewed as smoothening, gardening, and evolutionary pruning as already stored information influences the prerepresentations available for the next learning event. Changeux's spin glass model of neural nets is elegant, and successfully presents a hierarchical pattern of static information sorting. It's shortcomings are that it is unidirectional and fails to describe dynamic, "real time" information processing.

Another selective connectionist network model of learning is that of George Reeke and Gerald Edelman (1984) of Rockefeller University. They describe two parallel recognition automaton networks which communicate laterally. Automata are dynamic patterns of neighbor interactions capable of information processing (Chapter 1). The two parallel recognition automata which Edelman and Reeke devised have distinct and complementary personalities. They are named Darwin and Wallace after the co-developers of the theory of evolution, and utilize different approaches to the problem of recognition. "Darwin" is highly analytical, keyed to recognizing edges, dimensions, orientation, intensity, color, etc. "Wallace" is more "gestalt" and attempts to merely categorize objects into preconceived classifications. As in all parallel processing systems, output of the individual processors must be reconciled if they are not identical. Lateral communicating networks between Darwin and Wallace resolve conflicting output and form an associative memory. Because they operate on an unchanging connectionist network, Darwin and Wallace are considered "selectionist." Similar recognition automata may be operating in dynamic cytoskeletal networks within neurons.

0 0

Post a comment