Felice L. Bedford

A single object in the world exists continuously in time and travels along continuous paths in space. When observers sample from the world at different times, from different spatial vantage points, and through different sense modalities, the samples that result are often discon-tinuous. Determining whether or not these discontinuous samples arise from the exact same object is an important problem in perception and cognition. Bedford (2001a) recently put forth a theory ("Object Identity Theory", see Bedford 2001b, in press) in which the object identity prob-lem was shown to be the same for samples from space, time, modality, and different eyes and a solution based on mathematician Felix Klein's (1893/1957) five levels of geometric transformations offered. In the

__________

Correspondence should be sent to Felice L. Bedford, University of Arizona, 312 Psychology Building, Tucson, AZ 85721, USA

According to Object Identity Theory, when samples arise from differ-ent modalities they must be linked to the same object to initiate familiar cross-modal phenomena such as ventriloquism and prism adaptation. An example of the ventriloquism effect involves hearing the sound of a movie as coming from the central movie screen rather than from the nearest speaker located off to the side. Close your eyes, and the sound reverts back to its true location.

Radeau and Colin (this issue) raise four objections: first and second, object identity is not required for either ventriloquism or the aftereffects it can cause. They state "We will show hereafter that this claim is main-ly based on a confusion between the results of ventriloquism studies in which immediate effects and aftereffects are investigated". Third, object identity is also not required for the McGurk Effect, a cross-modal inter-action involving identification of speech sounds which was mentioned but not developed in Bedford (2001a), and fourth, object identity only results from cross-modal integration and not the other way around. In Object Identity Theory, all four claims are incorrect.

The crux of Radeau and Colin's arguments is based on studies which claim that while cognitive factors such as instructions, strategies and realism of displays can have an effect on ventriloquism, they do not in-fluence aftereffects of ventriloquism. A digression is necessary to make certain the distinction between ventriloquism and aftereffects is clear.

When there is a discrepancy between modalities, there are at least two psychological processes that can be observed. Immediate effects, such as ventriloquism, refer to precisely an immediately observable

__________

1. Due to time constraints, I have been unable to analyze and respond to the excellent article by Brian Scholl at this time. Hopefully a dialogue can be continued in this or some other forum.

effect on perception. If a (stationary but noisy) car on the movie screen
is detected visually in one location, but the same car is detected audi-torily in a different location, then the car is perceived as coming from one location despite the conflicting location values fed by the two mo-dalities. Presumably, the preferred unconscious perceptual inference, based on the totality of the data, is that the world contains a single car in a single location. Despite the disagreement between the modalities, this solution is unconsciously judged a better fit than other hypotheses, such as one car is really in two places in the world at the same time (or there are two different cars, one heard, and one seen). If there were a dis-agreement about location between vision and proprioception rather than vision and audition, an immediate effect also occurs. For instance, if you look at your own hand through a prism, the hand immediately feels to be where it looks through the prism; only one hand in one location is perceived despite the conflict. This effect is often known as "visual capture", a term which also can encompass ventriloquism. Immediate effects are short-term, and refer only to the on-line inference. If the ob-server closes her eyes, the sound "snaps back" to its true location, and the hand feels to be where it really is. Removal of one modality removes its influence on the other.

This is in contrast to the second effect that occurs following cross-modal discrepancies, a long-term solution that is observed when a con-flict is repeated. As with the immediate effects, the solution is adaptive and allows adjustments to sensory processes "blamed" for the error (see Bedford, 1999) and recalibrated accordingly. The long-term effects per-sist even when one of the conflicting modalities is now removed; the hand, for example, continues to feel displaced even when no longer looking at it. Or the sound continues to be localized where the noisy object had been seen, even though it is no longer visible. When vision and proprioception are in conflict, the long-term solution is known as "prism adaptation" or "adaptation" and when vision and audition are in conflict, the long-term solution is sometimes known as aftereffects of ventriloquism, or "adaptation" as well.

For immediate effects, you would not hear the car as coming from the screen unless at some level the visual and auditory signals were judged to refer to the same car. To do otherwise, would be a nonsensical exploit for perceptual systems. As discussed in Bedford, 2001a, as you read this article, the television may be on in the background. It would be unadaptive to hear the sound of the television as coming from the loca-tion of the article, since these sounds and sights are independent of one another. In a normal environment, there are multiple simultaneous sights and sounds; only those that are determined to arise from the same source should and will influence each other in this way. For long-term effects, the logic of the necessity of the object identity decision was explained for prism adaptation (Bedford, 2001a, 1999) and applies to aftereffects of ventriloquism as well. To recap briefly, the reason a mismatch be-

__________

2. However, it is worth noting that a function of the immediate effects should be to maintain experience of a coherent world, and the dissociation Radeau and Colin report may be of special interest for issues of how consciousness and perception interact in adaptation (see Reiner & Willingham, 2001). It is also reminiscent of Goodale, Milner, Jakobson, and Carey (1991) finding where dissociations in responding occur depending on whether patients use a motor response or report the answer.

tween vision and proprioception (or vision and audition) leads to adapta-tion is also based on an assumption that one object cannot be in two places at the same time. The discrepant cross-modal signals for a single object indicate that there must be an underlying error in one or both modalities. If there are two objects, there is no reason why they cannot be in two places, and there is no reason for adaptation to occur. Indeed, adaptation under these circumstances would fix something that isn't broken. Radeau and Colin argue that instead synchrony and proximity are required for the aftereffects of ventriloquism. How identity occurs, and that it must occur, are different issues. But note that it was also shown in Object Identity Theory how what are known as synchroniza-tion and proximity are derived from the geometric solution - synchro-nization results from geometry applied to time, and proximity from geometry for space.

__________

3. Conscious reasoning may suggest a male voice and a female face are not the same speaker, but the automatic geometry at the core of identity decision may nonetheless equate them.

then the sight of Henry's lips moving will only help you process what Henry is saying, not Lila, and vice versa, because they talk of different things. Logically, this would be true even if Lila's voice came from a speaker closer to Henry's location, and vice versa, even if perceptually one were misled by such a switch.

Radeau and Colin's final issue refers to neurons in the superior colli-culus that are responsive to input from different modalities simultane-ously. I agree multisensory neurons may play a role in the substrate of cross-modal interactions, but I do not see how the existence of such cells shows that the object identity decision must occur after integration rather than before as they suggest. I invite Radeau and Colin to develop the argument. Note that a potential paradox involving the order of the object identity decision and cross-modal integration, which I labeled "Held's paradox", was offered in a commentary I wrote on an article of Radeau's several years ago (Bedford, 1994) and may be of interest here.

One final note is that Radeau (and Colin) use the notion of "pairing" for ventriloquism here and elsewhere, but it is puzzling that originally Radeau (and long-time collaborator Bertelson) traced what was behind the concept as relating to a "single source, event, or object" (Radeau & Bertelson, 1974, 1976). It is unclear if her present work implies Radeau has had a change in theoretical position, or whether it is just the differ-ent (interesting, but independent) issue of conscious vs. unconscious processing which has gotten in the way.

Object Identity Theory was developed for situations when there is a choice between samples. For instance, will a circle map onto a square or to a circle with a hole in it in an apparent motion paradigm? In the geo-metric solution of the theory, a sample from the lowest level geometric transformation available will always be chosen - isometric preferred to similarity, but similarity preferred to affine, and so on.

Sharon (this issue) raises the issue of object identity when there are no choices. While she notes that Bedford (2001a), discusses that real-world situations present an abundance of samples from which to choose, she correctly argues that there are nonetheless situations in which there may be only one pair of samples. For instance, she suggests that if a man goes behind a house but a woman emerges, there appears to be only one pair of samples, and yet, we make an identity decision ("no"). As she puts it: "However, when more than one possible pairing is not im-mediately present, processing doesn't grind to a halt - people still make identity decisions, and in a non-random way". She further points out that since the man/woman case is the only possible match, it seems like it should be accepted because, in Object Identity Theory, identity is resolved from the lowest level of the hierarchy available, whatever it is.

These are important issues that need to be addressed. One useful dis-tinction is between the theory itself and its development or application. The geometric solution was indeed developed to make predictions for situations with choices between higher and lower levels, but note from the original article: "To summarize, in the present theory, the object identity decision is probabilistic such that the more radical a transforma-tion, the greater the properties of the original form are altered, and the less likely the pre-transformed and post-transformed samples will be judged to refer to the same object" (p. 142) and "In practice, this general probabilistic information can be used for an actual decision - Yes or No - by choosing ..." (p. 142). Thus, development of the theory applied the probabilities to choice paradigms where it is arguably clear-est about how probabilities can be used to make concrete decisions. However, it is important to note that the theory itself involves a general ordering of probabilities.

Probabilistic ordering also applies to a single pair of samples. All else being equal, a single pair of samples will be more likely judged as coming from one object, the less radical the geometric transformation that relates them. As Sharon astutely observes, the prism adaptation situ-ation discussed was, in fact, based on an identity decision with only a one single pair of samples, the hand in one place through vision, and another place through proprioception. As discussed, and reiterated by Sharon, the greater the amount of transformation (as defined by the Klein hier-archy) between the two modalities, the greater the amount of learning there was in an adaptation-like paradigm, even though there was only one pair of samples on each trial. For instance, a mapping in which a visual square was turned into a proprioceptive rectangle lead to less adaptation than a mapping in which the visual square became a smaller square in proprioceptive space. How do graded probabilities get trans-lated into graded amounts of learning? When the probability that two samples come from the same object is high, the probability that they will be judged on any one trial to refer to the same object is high. Since adaptation requires they be judged to be one object, there will effective-ly be more training trials when the probability of object identity is high. But of course, all else is not always equal. And a difficult issue remains - how is a specific probability (and what is the value of the probability) translated into an actual decision for two specific samples?

If there is just a single sample pair, one generality may be that the more radical the transformation, the more evidence required for a same-object conclusion. For instance, Huber and Aust (2001) show a number of leopard poses and note that they can all be viewed as coming from the same leopard. However, if one considers just two pictures at a time, some pairs are easy and some are hard. If we see a leopard from an angle, and then head on after a few contortions, the transformation that results is likely topological or even non-topological; under these circum-stances, we may need more evidence before we are perceptually certain it is still the leopard - perhaps a growl, or a few more glimpses.
Perhaps there is also greater room for domain-specific knowledge when

__________

4. There might be a counter-intuitive prediction here that one pair of samples, x and y, need not be judged the same object, but adding an alternative, z, such that there is now a choice between x and y and a more radical x and z, makes an x-y link more likely than if x and y were the only stimuli present. It the man goes behind the screen and both a woman and a car emerge, are the man and the woman more likely to be linked as the same object than if just the woman emerged without the car? Alternatively, Object Identity Theory can be interpreted that when there is a choice, the sample from the lower level transformation will be picked if any sample is picked at all.

there is a single pair of samples. There have been a number of com-ments on how specialized knowledge can be used to determine object identity (e.g., Bloom, 2001; Wilcox, 2001). While Object Identity Theory argues geometry is still the basic core of the decision (see also Bedford, 2001b), such domain-specific knowledge may play an espe-cially large role for single-sample pairs. The man-woman transformation Sharon mentions, for example, can be judged as two people and not one with help from such specialized knowledge that, except in rare medical facilities, men do not turn into woman and surely cannot even cross-dress so quickly. But such knowledge is painfully specific. This is a long way from Sharon's hope that Object Identity Theory can also pro-vide principled criteria when only one pair of samples is available, as it does for choices between samples. I suggest that if one accumulates all possible single sample pair cases, they will order themselves, as if by magic, such that the higher the level of transformation, the lower the probability the two samples were judged the same object. But the route to that observed regularity may be squishy.

Sharon also suggests that the theory develop further how to charac-terize and pick out samples both to determine how many there are, and as a question in its own right (see also Gauker, 2001).

A sample is part of a proximal stimulus. Identification of samples can begin with discontinuities in the proximal stimulus that occur in space and time (look at first and second order derivatives) - samples from dif-ferent modalities and different eyes have a head-start, as they begin at least more separable from one another. If we survey the samples used to build the case that identity is required in a number of independent domains in Bedford (2001a), we obtain the following:

It is not a coincidence that the samples were chosen to be as small and contentless as possible - a dot or psychological point when that would suffice. This was done so as to facilitate the abstraction away from content and to show the "sameness" that lurks beneath the different phenomena. Indeed, initially pictures of the formal structure were included for each phenomenon, but judged unnecessary by reviewers - presumably because they all looked like the same picture. Other phe-nomena for which the theory argues identity occurs, but with larger richer samples, were typically relegated to a table and not developed:

To identify larger, richer, samples from a big confusing proximal stimulus, a useful idea is provided by Lachter (2001). He suggests that Object Identity Theory be applied at many different scales simultane-ously, the way Marr suggests in general that early vision proceed. Note, however, that while Sharon observes that real-life rarely presents sam-ples in "tidy" fashion, this is a problem shared by many research enter-prises. Pavlovian conditioning, for instance, may present a single tone stimulus, with care to ensure that no other sound reaches the subject's ears. Even perception research, whose raison d'être is perception, assumes as a given the extraction of the perceptual elements; subjects are placed in front of a computer monitor, and fed very specific tidily presented stimuli. The important question of what is a sample, both for space ("objects") and for time ("events"), may be a shared problem with all of perception.

Dobbins (this issue) has concerns with stereopsis and apparent motion examples of object identity. But there is a deeper issue here about dif-ferent approaches to the same problems in perception. Starting with the specific issues, for stereopsis, Dobbins comments on Panum's limiting case. The phenomenon was raised in Bedford (2001a) as an example where even many-to-one matches, which are beyond the official end of the Kleinian geometries, can occur to achieve object identity. It illus-trated the extreme flexibility that object identity decisions can take and how Object Identity Theory is able to incorporate such flexibility. In Panum's limiting case, two dots (or lines) on one retina can match a single dot (or line) on the other retina, when they are the only dots (or lines) available. Observers see two dots (lines) separated in depth. Dobbins raises a fascinating example in which the single line is replaced with two very closely spaced lines - and fusion no longer occurs. He suggests that this is a counterexample to the geometric solution of Object Identity Theory because a non-topological match was accepted in the first case, but the topological match of the second case, a lower level in the hierarchy, was not accepted. However, this is not a genuine re-versal of the hierarchy because the system did not choose a higher-level match over a lower-level one in the same situation. Had there been a choice between mapping one line to one other line, or to two lines, then in the theory, the topology-preserving single line would be selected. As discussed in Bedford (2001a), the same two samples can in one situation refer to one object but in another situation refer to two objects, and more relevant here, two samples related by a high-level transformation can refer to one object (e.g., two bird samples separated in time, related topologically), while two samples from a low level in a different situa-tion can refer to two objects (e.g., two tennis balls separated in space, related by an isometric transformation).

The example Dobbins raises is nonetheless fascinating because the two situations are so closely related. Exploring such cases near the limits may prove interesting. Note that limits occur in the other domains as well. In prism adaptation, for example, there are limits to how large the discrepancy can be to attain a "same object" determination, even when a low-level transformation (isometric shift) separates the two mo-dalities. Large separations of 40 deg. will not produce either prism adaptation (long-term effect) or visual capture over proprioception (im-mediate effect). For stereopsis, as Dobbins explains, the examples of Panum's limiting case and the permutation he raises reflect in part limits in the amount of binocular disparity that will lead to fusion for one pair of points, and limits to the disparity gradient for pairs of points (i.e., the difference between the disparities). Domain specific limitations, or the existence of limits generally, do not detract from a general law (see Bedford, 2001b).

Perhaps most importantly, Dobbins concludes: "I have tried to show how constraints on binocular fusion depend on a detailed consideration of the geometry of local disparity and on the physiological mechanisms of binocular matching rather than on very general abstract considera-tions concerning a hierarchy of geometries." It is here that there is a fundamental difference in viewpoint. I have tried to show just the

__________

5. Dobbins also briefly raises three other specific points about apparent motion, none of which are problematic for Object Identity Theory. He sug-gests that other superficial factors might explain apparent motion choices rather than the geometric hierarchy. However, different experiments would require different explanatory principles while only the geometric hierarchy can explain all the findings. He suggests that subjects may not actually see the motion in apparent motion choice paradigms. We have developed a "clock" paradigm (Bedford & Mansson, under review) to improve these shortcomings in apparent motion methodology and have found the same geometric choices. Lastly, he suggests that since nearly any two stimuli can be accepted for a match in apparent motion, it is difficult to see how the geometric hierarchy is explanatory. As discussed in Bedford (2001a), the theory excels at just such flexibility while at the same time remaining rule-governed.

opposite - that local disparities and their many equivalents cannot be sufficient to understand perception, especially for relating diverse phe-nomena, each with its own set of specialized local rules. Also, why do the disparity gradients have the limits that they do? Do the limits to simple binocular fusion reflect situations where points on the left and right eyes could never have plausibly arisen from a single object in the natural world? Engineering models simply do not ask these kinds of questions.

This work was supported by a grant from the Vice Presidents Office of Research at the University of Arizona funded by the University of Arizona Foundation.