Medical diagnosis. When reading a textbook list of the rules for diagnosing a disorder and looking at the illustrations of the key features, it is hard to believe that diagnosis of the disorder could be difficult. The experience, however, is very different. Our research program has been aimed at understanding some of the sources of difficulty.

1. Difficulty in noticing the obvious. We have two lines of evidence that the perception and reporting of supposedly obvious features of patient appearance is strongly influenced by contextual factors. Both experts and students gained 20% in diagnostic accuracy by having textbook examples of features verbally described for them. Both experts and students reported from 15 to 30% more of these features when the correct diagnosis was suggested to them. The informal report by experts and students alike was that they simply had not noticed features that then seemed clear after they were pointed out. We believe that this is a major characteristic of many medical stimuli, and one that results from the large number of potential categories, the presence of many variations of normal features, and the relatively low redundancy of these stimuli. (Norman, LeBlanc & Brooks)

2. Disadvantage of Searching Medical Stimuli without a Diagnostic Hypothesis. Students who were instructed to avoid making any diagnosis until after they had worked down a list of all potential features for the disorders under consideration came to 1/3 fewer correct diagnoses than did a group that was asked to make a diagnosis first and then report the features. The "feature first" subjects informally reported that they wound up with so many potential features that they became confused with incompatible possibilities when confronting the list of diagnoses. (Brooks, Colle, Hatala & Norman)

3. Influence of similar prior instances. Different prior cases may come to mind under different circumstances. The prior cases that do come to mind might help in guiding the search for relevant features. The variability provided by depending on a wide variety of prior cases might help to explain big variations in the reliability of diagnosis. (Brooks, Allen & Norman, 1991; Norman, Brooks, Coblentz, & Babcook, 1992).

Implicit structure. Without deliberately trying to, people often become sensitive to the structure of a domain, such as musical style or even the design of an experiment in which they participate. We have documented a strong nonanalytic contribution from prior episodes in this kind of learning.

1. Abstract analogies in artificial grammars. In an artificial grammar experiment subjects are given sets of consonant strings to memorize. They then are told that these strings were generated by a complex set of rules and are asked to judge whether new strings are or are not consistent with those rules. Subjects perform well above chance on this even though they claim to not have been looking for rules while memorizing. We have shown that a major portion of this ability is due to similarity between test strings and particular training instances. even when the literal surface elements are changed. Reber (1969, 1989a) and Mathews et al (1989), in experiments on learning artificial grammars, reported good transfer to letter strings consisting of letters not used in the training stimuli, provided that the same grammar generated both training and transfer strings. They conclude from this that the transfer predominantly relies on abstract knowledge. We report an experiment showing that much of the transfer to "changed letter set" strings is due to abstract similarity between test strings and specific training stimuli. That is, a string such as MXVVVM could be seen as similar to BDCCCB without implying that regularities common to a large number of training items had been abstracted. We conclude that reliance on an abstract (relational) analogy to an individual item must be distinguished from reliance on knowledge of the structure of the grammar abstracted across many training items (Brooks & Vokey, 1991).

2. Learning the experimenter's design. One premise of the artificial grammar literature is that people are continually gaining sensitivity to the structure of the world around them. In this paper we demonstrate that this sensitivity to structure extends to general memory experiments as well as the experiments deliberately designed to investigate structural learning. Rules that experimenters use to select words for memory experiments, such as frequency, length and grammatical class, produce consistencies to which subjects can become sensitive. Replicating the key results from the tacit learning literature, subjects in our experiments discriminated new words consistent with the experimenters' selection rules from inconsistent words, even when they could not describe those rules. The results also reveal a close relation between the information underlying recognition memory and classification judgments. In particular, a "mirror effect" (Glanzer & Bowles, 1976) is found with both tasks. (Higham & Brooks, 1997)

Coordination of analytic and nonanalytic knowledge. One of the overarching themes of research on natural concepts over the last 25 years is that our knowledge of the world is generally inexplicit: We judge items to be members of categories and anticipate their properties with reference to category prototypes or by comparison to previously experience instances of the category. However, in laboratory experiments, people persistently look for and use rules. Even some of the evidence that was originally taken to support the use of instances can be better fit by models that assume people are seeking rules (Nosofsky, Palmeri & McKinley's RULEX model). This discrepancy between the observed behavior of people in the laboratory and their inferred behavior in the world is, in our view, the major issue that need resolution in this field.

1. Identification in the service of use: Most people seem to believe that natural categories have perfectly predictive defining features. They do not easily accept the family resemblance view that the features characteristic of a category are not individually consistent enough to act as a simple rule. But common categorization tasks in the laboratory do not produce this "simpler than it is" belief. If there is not a simple classification principle in these tasks, the subjects know that fact and can report it. We argue that most laboratory tasks using family resemblance categories do not result in the everyday "simpler than it is" belief because the stimuli and the tasks encourage analysis of identification procedures during training. In order to simulate the learning that occurs under many natural circumstances, we describe a procedure, diverted analysis, in which the subjects' analytic abilities are diverted from the way in which the stimuli are identified to the use to which those stimuli are to be put. This procedure has the effect of providing a special role for "family resemblance" data structures, unlike the situation with more analytic training procedures. We also suggest that the informational descriptions of the stimuli commonly used in laboratory tasks are better suited to describe the subjects' analytic behavior than their impressions of consistency that are critical to their belief in the existence of perfectly predictive features. Finally, we discuss the prevalence of "diverted analysis" in everyday categorization tasks.

2. The erosion of analytic control of classification: One hypothesis we entertained was that possibly people originally developed rules (attention to salient predictors) to guide classification, but that with practice, they would start relying more on similarity to prior, well known instances. This process of retrieval would likely be easier, faster and generally yield the same answers as the rules, since the more similar prior items would probably be in the correct category. To investigate this, our participants received a classification rule and practice classifying a set of stimuli. Test stimuli included items very similar to training items, with expectation of observing a shift from classification by rule to greater reliance on similarity. However, the only similarity effect occurred by generating false recognitions, which went down with practice. Otherwise, people applied the rule to any items they recognized as new, even though they were slowed at applying the rule than they were in categorizing the new items they mistakenly thought were old. Variables ineffective in producing the shift include: Increasing practice, training with many close variants of category members, confounding many salient features with category membership. If our original hypothesis had been correct, it would have helped to explain the general discrepancy between laboratory performance and inferred behavior in the world. (Regehr & Brooks, Wood & Brooks)

References

Several of these references are to manuscripts that have just been completed and are currently under review. Obviously, these are our more recent work. We will be happy to send them to you if you contact me.

Brooks, L.R., Colle, C., Hatala, R. & Norman, G.R. (1997). A disadvantage for searching medical stimuli without a diagnostic hypothesis. Submitted.

Brooks, L.R., LeBlanc, V., & Norman, G.R. (1997). Some conditions on support theory. Submitted

Brooks, L.R., Norman, G.R., & Allen, S.W. (1991). The Role of Specific Similarity in a Medical Diagnostic Task. Journal of Experimental Psychology: General, 120, 278-287.

Brooks, L.R., & Vokey, J.R. (1991). Abstract Analogies and Abstracted Grammars: A Comment on Reber, and Mathews et al. Journal of Experimenta Psychology: General, 120, 316-323.

Higham, P.A., & Brooks, L.R. (1997) Learning the experimenter's design: Tacit sensitivity to the structure of memory lists. Quarterly Journal of Experimental Psychology. February issue

Norman, G.R., Brooks, L.R., Coblentz, C.K., & Babcook, C.J. (1992). The interdependence of feature and category identification in diagnostic radiology. Memory & Cognition, 20, 344-355

Norman, G.R. LeBlanc, V. & Brooks. L.R. (1997). On the difficulty of noticing obvious features in patient appearance. Submitted.

Regehr, G. & Brooks, L.R. (1993). Perceptual manifestations of an analytic structure: The priority of holistic individuation. Journal of Experimental Psychology: General

Wood, T. & Brooks, L.R. (1997) On the erosion of analytic control. Submitted.