Read an Excerpt

Processing French

Yale University Press

Chapter One

Wugs and Goed: Evidence for the Child Acquirer's Use and Overuse of Rules

The basic distinction between morphological regularity and irregularity has figured in child language-acquisition studies since at least the 1950s. For instance, researchers such as Anisfeld and Tucker (1967), Berko (1958), Bryant and Anisfeld (1969), and Ervin (1964) all noted that at a certain point in their linguistic development (roughly between the ages of four and seven), child acquirers of English are able to productively inflect novel verbs with regular past-tense endings and novel nouns with regular plural endings. To take one example, Berko (1958) found that when child acquirers of English were shown a picture of a fictitious animal and told that it was a wug, they were able to produce its correct plural form: wugs. Since the children in these studies had never heard the word wug before (and presumably had never heard its plural form either), the results were taken as evidence that children are able to use a suffixation rule in order to createregular English plural and past-tense forms.

There is further naturalistic evidence that child acquirers of English productively use rules. This evidence comes from documented instances of overgeneralizations of the inflectional endings of the English regular past-tense and plural forms. For example, at about the same time that they become able to productively inflect novel verbs with regular past-tense endings and novel nouns with regular plural endings, children acquiring English will overgeneralize the past-tense suffix -ed both to the root forms of irregular verbs (e.g., sing-singed) and to irregular verbs already in the past tense (e.g., broked; see, e.g., Anisfeld, 1984; Slobin, 1985). Curiously, these children will have previously produced many of these same past-tense forms correctly. Again, these observations have been interpreted as supporting the notion that children are overapplying rules of inflectional morphology to create ungrammatical yet (apparently) rule-generated English regular past tenses and plurals.

The Status of Regular Morphological Rules in the Adult Mind

Of course, pathologically normal children do eventually retreat from the above-described overgeneralizations (see Marcus et al., 1992, for a discussion of how this might happen). Nevertheless, regular-irregular differences in English past-tense morphology have remained interesting to researchers in cognitive science, linguistics, and psycholinguistics who seek to establish the extent to which differences in the morphological structure of regularly and irregularly inflected items reflect differences in how these forms are represented in the minds of both children and adults.

Generally speaking, there are currently two types of explanation for the acquisitional, distributional, and (as we shall see) processing differences between English regular and irregular past-tense forms. Single-mechanism theories hold that both regulars and irregulars are processed-that is, produced and understood by speakers of a language-by a single associative memory system. Dual-mechanism theories instead propose that regulars are the product of symbolic, algebraic rules, while irregulars are stored in and retrieved from a partly associative mental lexicon. Below, I present an overview of and supporting evidence for both theoretical perspectives. Mirroring recent cognitive scientific discourse on this topic (see, e.g., Pinker and Ullman, 2002; McClelland and Patterson, 2002), this book will use the theory of connectionism to represent the single-mechanism perspective and words and rules theory to represent the dual-mechanism perspective.

Example of a Single-Mechanism Theory: Connectionism

As stated above, in this book connectionism (Rumelhart and McClelland, 1986; Elman and McClelland, 1986; Rumelhart, McClelland, and PDP Research Group, 1986; Elman et al., 1996) will represent the single-mechanism perspective. The origins of connectionism lie in research on artificial intelligence (AI), which has been defined as "the branch of computer science that investigates the extent to which the mental powers of human beings can be reproduced by means of machines" (Dunlop and Fetzer, 1993, p. 6). This includes designing machines that will engage in intelligent behavior when made to perform such actions as solving problems, playing chess games, or doing other similar activities. Whatever its purpose, AI research can be classified according to the extent to which it is either strong or weak AI, or to the extent to which it is symbolic (top-down) or connectionist (bottom-up) AI. A strong AI system not only exhibits intelligent behavior but also is sentient or self-aware. Strong AI systems exist only on the silver screen, with one of the more recent (and malevolent) examples being the snappily dressed, pistol-packing Agents of the Matrix movie trilogy. If any strong AI systems have in fact been created, either their designers have not gone public with the news, or the strong AI systems themselves have not volunteered evidence of their own existence. By contrast, weak AI systems exhibit intelligent behavior but are not self-aware. Designers of weak AI systems view their creations not as complete models of human consciousness, but as models of ways in which information processing might proceed in the mind. Weak AI systems have been designed that play games, solve problems, or, as we will see below, learn the English past tense. An example of a product of weak AI research would be Deep Blue, the chess-playing computer that defeated Garry Kasparov. While Deep Blue was clearly able to execute chess moves, it was neither aware that it was playing chess nor capable of other recognizably human behaviors.

The other dimension by which AI research may vary is the extent to which it is symbolic or connectionist. Symbolic AI research views human intelligence as emerging from the brain's manipulation of symbols. In this account of human intelligence, these symbols are variables, or placeholders; as such, they do not represent individual instantiations of objects but instead abstract categories according to which the objects or entities in the world have been classified. The symbol manipulations performed by the brain are conceived as being algebraic-like rules. As a result, symbolic AI researchers attempt to model human intelligence by designing AI programs that mirror this view of human cognition. Symbolic AI programs have built-in, higher-order representations of entities and objects as well as representations of classes of entities or objects, and they include rules specifying the possible operations that can be performed upon an entity or an object.

The reason that the symbols and rules of a symbolic AI program must be supplied by the programmer is because such programs have no built-in capacity to learn from their environment. Thus in most cases, these programs cannot ever know anything more about their world than what was built into them; as a result, they typically have neither the capacity to learn new information nor the capacity to learn through making mistakes. Possibly the most difficult obstacle for symbolic AI programs to overcome, however, is representing and implementing the background or commonsense knowledge of the world that all humans acquire and are able to reason with (Dreyfus, 1992). Attempts have been made to overcome this shortcoming of symbolic AI, usually either by carefully limit- ing the world within which the AI system must function, or by attempting to capture within sufficiently abstract semantic frames or scripts a common core of the seemingly diverse range of situations in daily life. Examples of such attempts include building programs with a tightly constrained, rule-based microworld such as that found in a chess game (see also, e.g., the block world of SHRDLU; Winograd, 1970) or with real-world knowledge but only of select, highly specific situations (such information is contained within semantic frames, or collections of information about a stereotyped situation such as a restaurant setting, a birthday party, etc.; e.g., Minsky, 1975; see also Schank and Abelson, 1977, for a related script-based approach).

In these and other examples of symbolic AI programs, little concern is shown for modeling actual brain processes. Symbolic AI researchers are usually not concerned with how their programs might acquire the knowledge they have of their microworlds, frames, or scripts; in many cases the symbols and rules used to manipulate them are simply taken as a given and are built into the system from the start.

Connectionist AI, by contrast, seeks to create AI systems that to some degree model the structure and functioning of the human brain and human learning processes themselves. In this view of human intelligence, learning occurs through both supervised and unsupervised interaction with the environment. At its most basic level, this learning is thought to occur in a bottom-up fashion: a flow of simple or low-level information enters the system from the outside world, and as it is processed by the brain, the low-level information is transformed into higher-order representations.

The human brain possesses a staggeringly complex circuitry. A recent study estimates that there are approximately twenty-one billion neurons-the number varies by sex and age-in the adult neocortex (Pakkenberg and Gundersen, 1997). Far more important than the sheer number of neurons, however, is the degree of interconnectivity between them: some estimates have suggested an average of two thousand neuronal connections, or synapses, per neuron (Pakkenberg et al., 2003). This means that on average, the twenty-one billion neurons in an adult's neocortex are networked by forty-two trillion synapses. These neuronal connections can either be excitatory (that is, they cause a neuron to fire, or release chemical neurotransmitters across synaptic gaps to the neurons surrounding it, thereby exciting those neurons to fire as well) or inhibitory (that is, they discourage a neuron from firing). It is thanks to this massive interconnectivity at the cellular level that the brain is able to process and integrate information the way it does.

Thus, connectionist AI researchers attempt to model human intelligence by designing AI programs that mirror our current understanding of how the brain processes information at the cellular level. In contrast to symbolic AI programs, connectionist AI programs usually consist of computational matrices that mimic the networked structure and information flow patterns of interconnected neurons in the brain. One part of the computational matrix represents an array or layer of input neurons, or nodes, while another part represents a layer of output nodes. In recent neural network designs, there may also be a "hidden" layer of nodes between the input and output layers. (They are referred to as hidden because they never have direct contact with the outside world.) As in a real brain, the input and output nodes in an artificial neural network are connected; however, they are not connected physically but mathematically, such that as an input node is turned on (i.e., is told to fire by the program), it sends an excitatory message to the output node(s) it is connected to. Through one of several possible learning algorithms, the network receives feedback during its course of training on the accuracy of its output activations. This feedback is then used to mathematically adjust the strengths of the connections between nodes. In this way, for a given input, connections for correct answers are strengthened while connections for incorrect answers are inhibited. The result is that the network is able to gradually tune itself such that only the correct output is likely to be activated for any particular input.

With respect to efforts to model the acquisition and processing of language, possibly the best known connectionist networks are the parallel distributed processing (PDP) models of Rumelhart and McClelland (Rumelhart, McClelland, and PDP Research Group, 1986). Two terms may require explanation: parallel means that processing of input occurs simultaneously throughout the network, and distributed means that there is no command center, or central location in the network where executive decisions are made. These models have also been called pattern associators in that they associate one kind of pattern (for instance, a pattern of activated input neurons) with another kind of pattern (for instance, a desired pattern of activated output neurons).

Two well-known PDP models have been designed to cope with different aspects of language processing. The first model, McClelland and Ellman's (1986) TRACE model of phoneme perception and lexical access, was a PDP network of detectors and connections arranged in three layers: one of distinctive-feature detectors, one of phoneme detectors, and one of word detectors. The connections between layers were bidirectional and excitatory, meaning that while lower-level information could feed forward to higher-level layers, higher-order information could also percolate back down and thereby influence detection at the lower-level layers. By contrast, the connections within a given layer were inhibitory. In detecting a word or a phoneme, the feature detectors first extracted relevant information from a spectral representation of the speech signal. That information then spread according to the relative strength of activation of certain distinctive-feature detectors over others to the layer of phoneme detectors. In the case of word detections, the phoneme detectors then activated words in the third layer of the network. The bidirectional, excitatory nature of the connections between layers, coupled with the inhibitory nature of the connections within a given layer, conspired to increasingly activate one lexical candidate over all others while simultaneously suppressing competing candidates. Information related to representing the temporal unfolding of a word was modeled by repeating each feature detector multiple times, thereby allowing the possibility of both past and possible upcoming information to be activated along with current information. Using this interactive network architecture, McClelland and Ellman (1986) showed that the TRACE model could successfully simulate a number of phoneme and word detection phenomena that had been observed in experiments with human participants. For example, in phoneme detection experiments, it was observed that factors pertaining to phonetic context (i.e., the phonemes that precede and follow a target phoneme) appeared to aid people in detecting phonemes or in recovering phonemes that had been partially masked with noise. The TRACE model also produced this finding.

In a second simulation, Rumelhart and McClelland (1986) used a pattern associator network to model not speech perception, but the acquisition of the English past-tense system. For this study, the researchers began by noting that although the findings surrounding the acquisition of the English past tense have been interpreted as bearing the hallmarks of rule-based development (with clearly marked and nonoverlapping stages of development), the documented facts (some of which were reviewed above) could also be argued to suggest instead that acquisition proceeds on a less categorical, more gradual basis. To a certain extent, therefore, the observed facts would also appear to be in line with the predictions of a more probabilistically based acquisition account. Thus, one of the goals of Rumelhart and McClelland (1986) was to successfully model this kind of gradual convergence upon the correct forms of the English past tense.

(Continues...)

---

Excerpted from Processing French by PETER GOLATO Copyright © 2006 by Yale University. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---