Consciousness and the visuomotor transformation (1)
Pierre Jacob and Marc Jeannerod
Institute of Cognitive Science,
67, boulevard Pinel,
69675 Bron Cedex, France (2)
The goal of this paper is to assess the significance of recent experimental findings on visual illusions for the dualistic model of the human visual system carefully argued for by Goodale and Milner (1992; see also Milner and Goodale, 1995). According to their view, the human brain processes visual information in two fundamentally different ways according to whether its task is to provide a visual representation of the distal layout (visual food for thought, as it were), or to provide visual guidance for action and behavior upon the environment. We often act upon, or react to, objects in our environment without full visual awareness of them. Experienced drivers produce appropriate motor responses to visual stimuli of which they are dimly aware, if at all, when they act. Seeing can make us visually aware of objects, properties and facts in our environment. But it need not. Often, seeing allows us to act efficiently on objects of which we are not fully aware while we act. Primates and particularly humans are unique among animals in being able to grasp and manipulate objects in their environment using the dexterity of their hands. This is why in this paper, as in much of the literature on the so-called "visuomotor transformation", relevant actions are constituted by arm and hand motions directed towards objects, such as reaching and grasping.
Our paper is mostly methodological. After sketching the rationale for Goodale and Milner's dualistic picture of the human visual system, we examine several experiments dealing with visual size-contrast illusions. Visual illusion is a typical feature of normal conscious visual experience. Prima facie, the experiments which we review suggest that visual size-contrast illusions do not affect reaching and grasping as much as perceptual judgement. These experiments have been subject to some methodological criticisms which in turn have given rise to new experimental evidence. We scrutinize both the methodological criticisms and the experimental findings. We conclude that the dualistic model of visual information processing survives the criticisms.
Anatomical and neurophysiological work performed on the brain of macaque monkeys led to the view famously espoused by Ungerleider and Mishkin (1982) according to which the visual system in the primate brain is divided into two major streams or pathways. Whereas the ventral stream projects from the primary visual cortex onto the inferotemporal cortex, the dorsal stream projects the primary visual cortex into the posterior parietal lobe. Based on lesion studies in monkeys, Ungerleider and Mishkin (1982) reported that interruption of the ventral pathway significantly affects monkeys' ability to visually discriminate and recognize objects without disrupting their perception of the spatial relations among objects. Concomitantly, they reported that interruption of the dorsal pathway prompts a spatial disorientation in monkeys characterized jointly by a disruption of their ability to perceive the spatial relations among objects and a deficit of their visuomotor behavior, while the ability to recognize objects is left intact. Consequently, Ungerleider and Mishkin (1982) referred to the ventral pathway and the dorsal pathway respectively as the "What" system and the "Where" system.
Subsequently, much neuropsychological work on brain lesioned human patients has confirmed the significance of the anatomical duality between the ventral and the dorsal pathways in the primate visual system. It has also led, however, to a reassessment of the functional and psychological role of this anatomical duality. The relevant neuropsychological evidence comes from the study of two major complementary disorders: visual form agnosia and optic ataxia.
Consider D.F., a patient with visual form agnosia caused by a bilateral occipito-temporal lesion and examined by Goodale, Milner, Jakobson and Carey (1991) (see also Milner and Goodale, 1995). Basically, D.F. has a lesion in the ventral pathway — the stream projecting from the primary visual cortex onto the infero-temporal cortex. Her ability to recognize objects is severely impaired. She keeps rather good color discrimination and texture sensitivity. She seems better at recognizing the shapes of "natural" objects (such as fruits and vegetables) than the shapes of arbitrary geometrical forms or letters of the alphabet or black and white drawings. Presumably, when visually processing "natural" objects, D.F. can use cues from the color and texture of things which she does perceive together with her encyclopaedic knowledge of fruits and vegetables in order to infer their shape. Whereas in the case of arbitrary objects or black and white drawings, there is no correlation that she can rely on between either color or texture and shape. Also, she is much better at reproducing drawings of objects when relying on memory than just copying drawings (of e.g., a schematic apple and a schematic book) which she could not recognize. Although she is incapable of recognizing the shape of objects, still she has a number of visuomotor abilities pretty well preserved. In a nutshell, there is a striking dissociation between D.F.'s ability to correctly report her perceptual visual experience and her visuomotor control ability even when both involve essentially the same bodily motions such as adjusting the distance between her thumb and index finger. Although she is able to correctly scale her finger grip in order to grasp an object between her thumb and her index finger, she turns out not to be capable of providing a perceptual estimate of the size of an object by matching it to the distance between her thumb and her index finger. In other words, D.F.'s ability to produce one and the same bodily output depends on whether she is engaged in a perceptual or in a visuomotor task.
Visual form agnosia — as exemplified in D.F. — is thus characterized by a dissociation between on the one hand a deficit of visual awareness and visual discriminatory abilities and on the other hand a preservation of the ability to manipulate objects with the fingers of one's hand. This dissociation is reminiscent of the condition known as blindsight which is prompted by lesions in the primary visual cortex (Weiskrantz, 1986, 1997): whereas visual form agnosia results from a lesion disconnecting the ventral pathway from visual inputs coming from the primary visual cortex, blindsight is caused by lesions affecting directly area V1 in the primary visual cortex itself. Unlike visual agnosics, blindsight patients (described by Weiskrantz, Perenin, Rossetti and others) report complete lack of visual experience of objects in their hemianopic visual field. They are nonetheless above chance in forced choice tasks such as discriminating an X from an O in their blind hemifield. This indicates that they process visual information even though they are not visually aware of things and properties in their environment. For example, one blindsight patient examined by Perenin and Rossetti, P.J.G., has a complete right hemianopia caused by a left medial occipital lesion. He can discriminate motion direction in his hemianopic field but he is unable to discriminate between simple geometric forms (e.g., circles vs. triangles). However, when asked to insert a card in an orientable slot located within his right hemianopic visual field with his right hand, he is well above chance in orienting the card. He cannot insert the card into the slot because he has reaching disorders. Also, when asked to grasp horizontal objects accessible in his hemianopic field, the grip aperture between thumb and opposable finger is significantly correlated with actual object size. Importantly, unlike in motor tasks, in verbal reports and matching tasks, P.J.G. was at chance.
By contrast, consider A.T., a patient with optic ataxia caused by bilateral occipito-parietal lesions and examined by Jeannerod et al. (1994). Optic ataxia results from lesions which interrupt the flow of visual information from the primary visual cortex via the dorsal stream up to the parietal cortex. This condition is characterized by a selective deficit of the visuomotor transformation. Reaching accuracy (i.e., the ability to move one's hand towards objects) is somewhat impaired in this patient, whereas her ability to grasp objects between thumb and index finger is grossly impaired in that she cannot adjust the grip to the shape and size of the object. By contrast, her ability to visually identify and recognize objects is normal. Furthermore, although this patient was unable to use visual information about their shape to grasp objects, she was close to normal in using visual information to do perceptual report by matching objects' size by scaling the distance between their thumb and index finger. Again, it is remarkable that optic ataxia patients are capable or incapable of one and the same motor output according to whether the task is a perceptual or a visuomotor task.
Relying on such neuropsychological evidence, Goodale and Milner (1992) have offered a reinterpretation of the functional and psychological significance of the anatomical duality between the ventral and the dorsal pathways in the primate visual system. According to Goodale and Milner's view, the job of the ventral stream is to provide an analysis of visual inputs so as to allow object recognition and conscious visual perception, whereas the job of the dorsal stream is to provide the basis for the visual guidance of actions — particularly, actions of the hand — directed towards objects. In other words, visual information can be processed differently according to whether it provides an individual with evidence for thinking about the relation between objects or whether it guides his or her actions directed toward objects. Along similar lines, Jeannerod (1994, 1997) has drawn a functional distinction between two systems of processing of visual information: what he calls the "semantic" processing of information yields what we will call a "perceptual representation" of the visual stimulus, whereas the "pragmatic" processing of information yields what we will call a "motor representation" of the same physical stimulus.
Goodale and Milner’s dualistic view of the visual system is corroborated by the study of temporal dissociations between motor responses and perceptual responses in normal human subjects. Such dissociations show up when visual disturbances are experimentally provoked while subjects perform a hand movement directed towards an object. In some experimental cases (e.g., Bridgeman et al. 1981, Goodale et al. 1986, Taylor and McCloskey 1990, Meeres and Graves 1990), subjects do not even become aware of the disturbance although they produce an adapted motor response. In other cases (Castiello et al. 1991, Castiello and Jeannerod 1991), there is a significant delay between the motor response and the perceptual response to one and the same visual event. In Castiello et al's experiment (1991), the subject's task is to grasp a translucid plastic cylinder as soon as the cylinder is illuminated. In 20% of randomly distributed cases, after the cylinder has been illuminated in a given location, the light goes off as soon as the hand motion is set in action, and a different cylinder located 20 cm to the right or to the left of the initial cylinder is illuminated. The subject is able to modify the trajectory of his hand motion so as to reach the new target. The first correction of the trajectory occurs at around 100 ms after the change of target. Castiello et al. (1991) asked subjects to emit the sound 'Tah!' upon their conscious detection of the change. It turns out that the motor response precedes the perceptual response by approximately 300 ms. This temporal dissociation does not reflect the interference between two motor responses: the correction of the hand motion and the production of the sound. For the same time difference occurs when the hand motion and the sound are produced separately. The delay thus reflects a dissociation between the perceptual processing and the visuomotor processing of one and the same visual stimulus. The former generates a conscious response; the latter generates an automatic response.
One interesting experimental paradigm on which to test Goodale and Milner's dualistic picture of visual information processing is the study of visual illusions. Particularly relevant are examples of size-contrast illusions in which the perception of the relative size of objects does not correspond to their real size. By manipulating contextual features of a visual stimulus, it is possible to induce changes in the perceptual judgments of the relative size of objects. Consistent with this dualistic picture of the visual system, it is plausible to assume that, unlike "pure" perception, the visually guided control of action upon objects cannot afford the luxury of being fooled by contextual features in the visual array. If so, then it is plausible to assume that the visual guidance of grasping of objects will be relatively impervious to the influence of visual illusions. The trick then is to design experiments which will reveal jointly the sensitivity of "pure" perceptual judgments to the influence of visual illusions and the immunity of visually guided actions to visual illusions. In the rest of this paper, we want to examine the question of whether and how much the experimental investigation of visual illusions corroborates Goodale and Milner's dualistic picture of visual information processing.
One of the best studied visual illusions is the Müller-Lyer illusion in which two segments of equal length are displayed. Although the two segments are of equal length, the segment with two converging arrows seems longer than the segment with two diverging arrows. Consider a recent experiment by Gentilucci et al. (1996) on the Müller-Lyer illusion. In this experiment, two Müller-Lyer segments are displayed on a table perpendicular to the axis of subjects' heads. Subjects are exposed to the stimulus for 5 seconds and then they are asked to point their right index finger onto the more distant vertex of either segment. Gentilucci et al. (1996) compare four conditions. In the full-vision condition, the light is on during the whole trial. So subjects can see both the target and their hand during the trial. In the non-visual feedback condition, subjects see the stimulus and their hand before the trial but they see only the stimulus, not their hand, during the trial. In the no-vision 0-second delay condition, after the 5 second inspection time, the light goes off and this is the go signal for subjects who instantly start their movement. In this third condition, subjects see neither their hand nor the stimulus during the trial. Finally, in the no-vision 5 second delay condition, the go signal comes 5 seconds after the light is switched off.
Basically, what Gentilucci et al. (1996) found was increasing influence of the Müller-Lyer illusion from condition 1 to 4. When both target and hand are visible during the movement, the task is a "pure" visuomotor task. Pointing of the index finger can be guided by what we will call a "motor representation" in which the position of the target vertex is coded in a purely egocentric frame of reference. In other words, the position of the more distant vertex of the segment is visually coded with respect to the axis of the subject's body yielding a "motor representation" of the target of the motion. When neither hand nor stimulus is visible during the movement, the motor representation fades away and the task is increasingly driven by perceptual, cognitive and memory processes. The subject must therefore use a perceptual representation of the length of the segment. The visual system must then code the perceived distance between the two vertices of the segment in an allocentric, not an egocentric, frame of reference. In the last condition, the subject must exploit a perceptual representation stored in working memory. In the third and fourth conditions, the pointing of the index finger toward the target must be guided in part by a perceptual representation and the influence of the illusion increases. When the distance between the target and the subject's body is coded in an egocentric coordinate system, the representation of this distance is not much affected by the illusion (3). However, the representation of the apparent length of the segment (i.e., the distance between two vertices) is very much affected by the illusion. Whether the so-called motor representation of the target of the motion is delivered by the dorsal stream, whereas the perceptual representation of the length of the segment is delivered by the ventral stream, are questions to which we'll return towards the end of this paper.
Another well-known visual illusion is the Ebbinghaus or Titchener Circles illusion. The standard version of this illusion consists of the display of two circles of equal diameter, one surrounded by an annulus of smaller circles, the other surrounded by an annulus of larger circles. Given such a display, the circle surrounded by an annulus of smaller circles seems larger than the circle surrounded by an annulus of larger circles. Aglioti et al. (1995) replaced the two central circles by two thin three-dimensional plastic disks of equal diameter surrounded by two annuli of circles of different sizes displayed within a horizontal plane. Thus, the aim of the experiment was to compare the estimate of disk size made by subjects in the perceptual condition (by reporting or matching the size of the disk) with grip calibration in the visuomotor condition (involved in grasping the disk). Before discussing the results obtained in this and subsequent experiments performed by the same authors, it is important to concentrate on the methods used to measure subjects' estimates of disk size.
In the perceptual condition, several methods can be used. The classical psychophysical method consists in asking subjects to report verbally which of the two disks (that surrounded by an annulus of larger circles or that surrounded by an annulus of smaller circles) appears to be larger. The actual size of one of the disks can be changed on different trials until the subject finds no difference between the two. In this condition, the difference in physical size between the two disks can be considered as a measure of the amplitude of the size illusion. An alternative method, which has been used by several authors to provide a measure of perceived size, is finger matching: the subject is asked to match object size with the distance between his thumb and index finger (Jeannerod and Decety, 1990) (Figure 1a). The advantage of this method over the previous one is twofold. First, it provides a continuous measure of object size, as opposed to psychophysical scaling which is, by design, a discontinuous measure. Second, in the context of the present discussion, it can be directly compared with the finger grip size measured during the visuomotor task. Indeed, this latter method was used by Goodale et al. (1991) and by Jeannerod et al. (1994) to compare object size estimates in different conditions by patients D.F. and A.T., respectively.
In the visuomotor condition, the measure of object size is based on the unfolding of the natural grasping movement performed by the subject while the hand approaches the object. It has been shown that the fingers progressively stretch up to a maximal aperture before they close down until contact with the object. This maximal grip aperture (henceforth, MGA) occurs at a relatively fixed position on the time axis, corresponding to about 60% of movement time (Jeannerod, 1981). The important point is that MGA has been found to be linearly scaled (within limits) to object size: although MGA is, by definition, larger than object size, it is directly proportional to object size (e.g., Marteniuk et al. 1990). Due to this remarkable property, the measure of MGA can be considered as a faithful index of the object size cue that the visuomotor system uses for performing a successful grasp. Indeed, MGA does not depend on a direct visual comparison between the object and the hand during the movement: instead, it results from an anticipatory calibration by the visual system, during the early stage of the movement or even before it starts. Accordingly, the same pattern of grip formation is observed in situations where the view of the hand by the subject is prevented during the grasp (Jeannerod, 1984) (Figure 1b).
Aglioti et al. (1995) used two kinds of trials. In trials of the first type, two disks of identical physical size appeared different as a result of the difference between their respective annulus of surrounding circles. In trials of the second type, the two disks appeared to be of equal size even though the physical size of the disk surrounded by the annulus of larger circles was slightly larger than the disk surrounded by smaller circles. Both kinds of trials were presented in random alternations as were the left vs. right positions of either kinds of annuli. The task of the subjects was to pick up the target disk on the left if they thought that both disks were of equal size or to pick up the target disk on the right if they thought the two disks were different in size. Importantly, the motor response was always subsequent upon the perceptual judgment, since before grasping a disk, subjects had to select it on the basis of a perceptual judgment. The maximum aperture of the subjects grip (MGA) was measured in flight using optoelectronic recording. Aglioti et al. (1995) found that the influence of the illusion was greater on the perceptual judgment than on the grasping task. In other words, in grasping, the scaling of MGA was correlated with the actual physical size of the disk, not with the perceived size, even when the choice of the disk indicated that subjects misperceived its actual physical size.
Although it provides prima facie evidence in favor of Goodale and Milner's distinction between two systems of information processing within the visual system, this initial study raises a number of problems. One problem arises from the fact that in Aglioti's experiment, the measure of the influence of the illusion upon the two tasks — the perceptual task and the motor task — relied on two different methods. The perceptual task is being evaluated by a binary choice or a discrete measure: the subject picks up one of the two disks according to whether he judges them to be equal or unequal in size. But the grasping task is being measured by the distance between thumb and index finger, which is a continuous measure. A second problem arises from the fact that in the grasping task, two factors might jointly account for the scaling of subjects' grip: one is the influence of the illusion upon the computation of the distance between thumb and index finger; the other one is the constraint of the annulus of surrounding circles upon the positioning of the fingers on the disk during grasping. A third problem, which has been noticed by Franz et al. (1998), is the following asymmetry between the perceptual and the grasping tasks: in the perceptual task, subjects are asked to compare the size of two Titchener disks, while in the grasping task, although two Titchener disks are displayed, subjects actually focus their visual attention upon only one disk, i.e., the disk they are about to grasp (Figure 2). Arguably, the fact that two disks are relevant to the perceptual comparison and only one is relevant to the motor response might explain why Aglioti et al. (1995) found that the influence of the illusion was significantly greater in the perceptual task than in the motor task.
In response to the first problem, Haffenden and Goodale (1998) asked subjects to inspect two Titchener disks and decide whether they were equal or unequal in size. If subjects judged them to be equal, the motor task consisted, as in Aglioti et al.'s (1995) experiment, in grasping the disk on the right between thumb and index finger. If subjects judged them to be unequal, the motor task consisted in grasping the disk on the left between thumb and index finger. Unlike in the Aglioti et al.'s (1995) experiment, however, the perceptual task consisted not merely in the selection of either the disk on the right or the disk on the left according to whether the two disks appeared equal or unequal. It also consisted in a manual estimate of the size of the selected disk. In other words, subjects were asked to match the size of the selected disk by the distance between their thumb and their index finger. One and the same motor output was thus used as a measure of both a motor task and a perceptual task.
Haffenden and Goodale's (1998) findings corroborate Aglioti et al.'s (1995): on trials in which subjects indicated that they thought that the disks were identical in size even though they were physically unequal, the calibration of their grip aperture was correlated with the actual physical size, not the apparent size (Figure 3a). Similarly, when subjects indicated that they thought that the disks were different in size even though they were physically equal, the calibration of their grip aperture was correlated with the actual physical size, not the apparent size (Figure 3c). Conversely, when subjects indicated that they thought that the disks were identical in size even though they were physically unequal, their manual estimates of the size corresponded to the apparent size of the disks, not their real size (Figure 3b). Similarly, when subjects indicated that they thought that the disks were different in size even though they were physically equal, their manual estimates corresponded to the apparent size of the disks, not their real size (Figure 3d).
In order to investigate the potential effect of the surrounding annulus upon grasping, Haffenden and Goodale designed two more non-illusory background conditions. The first non-illusory background condition consisted of two identical annuli made of equal-sized circles, midway in size between the large circles and the small circles involved in the pair of illusory annuli. The second non-illusory condition consisted of a blank background without any annulus. When physically different disks were presented against either kind of non-illusory background — either involving no annulus or consisting of two annuli made of equal-sized circles —, both grip scaling and manual estimates reflected the actual difference in size between the disks. When physically identical disks were displayed against either kind of non-illusory background, no significant differences were observed in either grip scaling or manual estimates. Finally, Haffenden and Goodale presented two disks of the same physical size either against an annulus of equal sized circles or against a blank background without an annulus. The authors found the interesting following dissociation: in the grasping task, disks displayed on a background consisting of an annulus of equal-sized circles prompted smaller grip apertures than the same disks displayed on a blank background without any annulus. Conversely, in the perceptual task, disks surrounded by an annulus of equal-sized circles prompted larger manual estimations than disks not surrounded by an annulus. Thus, the presence of an annulus contributes to diminishing the scaling of the grip in the grasping task, whereas it contributes to increasing the manual estimation in the perceptual task.
Both Aglioti et al. (1995) and Haffenden and Goodale (1998) found that perceptual judgment is significantly more affected by the Titchener illusion than grasping. Franz et al. (1998) question the claim that the Titchener illusion has significantly different effects on perception and action. First, they hypothezise that the observed difference arises from an asymmetry between the perceptual task and the motor task: in the perceptual task, subjects are required to compare two disks surrounded by two distinct annuli. In the motor task, even though subjects face two disks surrounded by two distinct annuli, only one of the two disks is relevant to the task: only the disk to be grasped is relevant to grasping. They argue that if the two tasks are carefully matched, then the illusion does affect perception and grasping in strikingly similar ways. In their experiment, only one disk surrounded either by an annulus of larger circles or by an annulus of smaller circles is displayed. In the grasping task, subjects are asked to grasp the disk. In the perceptual task, subjects are asked to match the disk with a comparison circle devoid of annulus, displayed on the monitor of a computer, and whose size they can adjust (Figure 2). In this "single context" experiment, Franz et al. (1998) found that the influence of the illusion was the same on grasping as it was on perception and they found that the level of influence of the illusion was comparable to the level of influence on grasping found by Aglioti et al. (1995). Franz et al. (1998) claim that their finding is evidence against a dualistic picture of visual information processing.
Furthermore, Franz et al. (personal communication, December 1998) compared systematically the following three perceptual conditions. The first condition, which they call "direct comparison" is the standard condition investigated by Aglioti et al. (1995) and by Haffenden and Goodale (1998): (4) subjects are asked to judge whether two equal disks surrounded by two distinct annuli (one consisting of larger circles, the other consisting of smaller circles) are equal or unequal. In the second condition called "single context" condition and described above, subjects are asked to match one disk surrounded either by an annulus of larger circles or an annulus of smaller circles with a circle without an annulus displayed on a computer. In the third condition, called "separate comparison", two Titchener disks surrounded by two distinct annuli are displayed, and subjects are asked to compare one at a time each disk with a separate circle without an annulus. Franz et al. hypothezise that "direct" perceptual comparison of two Titchener disks of equal size surrounded by two distinct annuli gives rise to what they call "super-additivity" or a "super-additive effect". They find that the influence of the illusion prompted by the "direct comparison" between two disks surrounded by two distinct annuli is much greater than the result of the sum of either two "single context" experiments or two "separate comparisons".
Daprati and Gentilucci (1997) also report similar findings using the Müller-Lyer illusion. Subjects had to either grasp or match the size of wooden bars of equal length terminated either by converging or diverging arrows. In the (perceptual) matching task, they found that a bar with converging arrows typically yielded larger estimates than a bar with diverging arrows. However, the perceived difference between the two bars was less than that obtained when the two bars were shown simultaneously (direct comparison). In the (visuomotor) grasping task, almost no difference was found whether subjects grasped a bar with converging or diverging arrows. As a result of the experiment, the difference between the perceptual estimate and the visuomotor grip calibration was relatively small.
These findings raise two complementary questions. First, what is the significance of the "super-additivity" of direct comparison in the experimental study of the Titchener illusion? Second, when the perceptual task and the motor task are symmetrically designed, the influence of the illusion is no greater on the former than it is on the latter. How strongly does this finding argue against Goodale and Milner's hypothesis of separate visual processings for perception and action? In response to the first question, we would like to point out that the claim that direct comparison involves a "super-additive" effect amounts to providing experimental evidence that the display of a circle with no surrounding annulus together with one disk surrounded by an annulus of larger or smaller circles does not trigger the kind of perceptual illusion prompted by the display of two disks surrounded by two distinct annuli. In effect, it is a reminder of the condition in which the perceptual processing of visual information is sensitive to the Titchener illusion. The reason why we are inclined to take a deflationary position with respect to the second question is precisely that in their symmetrical design of the motor and the perceptual tasks, Franz et al. (1998) in effect used a display in which the perceptual illusion does not properly arise. Incidentally, the same comment holds for the Daprati and Gentilucci’s smaller effect of bars presented one at a time. So when you have independent evidence that a stimulus is not a suitable basis for producing the perceptual illusion, it should not be surprising to find that the motor response is not influenced by the illusion either. Nor should it be surprising to find that the perceptual response and the motor response to the non-illusory stimulus do not really differ. In fact, if a stimulus does not give rise to a perceptual illusion, it does not provide an adequate basis for drawing any conclusion upon Goodale and Milner's hypothesis that there are two separate systems of visual information processing for perception and action.
In Goodale and Milner's model, the functional distinction between two systems of visual information processing is superimposed upon an anatomical distinction: the computational distinction between the analysis of visual inputs subserving perception and the analysis of visual inputs subserving the control of hand motion is collapsed onto the anatomical duality between the ventral and the dorsal streams within the primate visual system. The experimental work on size-contrast illusions raises an interesting issue which must be addressed by this dualistic view.
In Gentilucci et al.'s (1996) experiment on the Müller-Lyer illusion, there is a contrast between two kinds of representations of the target of the pointing of the index finger: one is a perceptual representation of the length of the segment, the other is a motor representation of the vertex. The former codes the distance between the two vertices of the segment in an allocentric frame of reference. The latter codes the distance between the vertex and the axis of the subject's head in an egocentric frame of reference. Unlike the former, the latter is relatively impervious to the influence of the visual illusion. In Aglioti et al.'s (1995) and Haffenden and Goodale's (1998) experiments on Titchener circles illusion too, there is a contrast between two kinds of representations of the relative size of a disk. Here again, the motor representation is more impervious to the illusion than the perceptual representation of the size of the disk. However, in these experiments, the motion involved in the motor task has two components: a reaching motion and a grasping motion. Thus, one motor representation must presumably guide the reaching motion; and another motor representation must presumably guide the grasping motion. The former motor representation must code the distance between the target disk and the axis of the subject's head in an egocentric frame of reference. The latter motor representation must code the size of the disk. Unlike the perceptual representation of the relative size of the disk (which is sensitive to the illusion), the motor representation is correlated with the actual physical size of the disk. Unlike the motor representation which guides reaching, the motor representation which guides grasping must code the size of the disk in an allocentric frame of reference.
And now, here is our problem for Goodale and Milner's dualistic model. Both neurophysiological studies on monkey brains and neuropsychological evidence from optic ataxia in brain lesioned human patients indicate that at least some parietal neurones within the dorsal stream code spatial information and properties concerning the position of objects relative to the axis of an individual's head in an egocentric frame of reference. Both neurophysiological studies on monkey brains and neuropsychological evidence from visual form agnosia in brain lesioned human patients indicate that infero-temporal neurones in the ventral stream code spatial information about the relative positions of distinct objects within the visual array or the different parts of s single visual stimulus in an allocentric frame of reference. As some neuropsychological evidence and work on the Titchener circles illusion suggest, if the dorsal stream plays a role in the calibration of scaling of subjects' grip involved in grasping, then neurones in the dorsal stream too must be able to code the size of objects or the distance between objects in an allocentric frame of reference.
In conclusion, we would like to draw attention to two features of our discussion. First of all, Goodale and Milner's dualistic picture of visual information processing is confirmed. A precursor of this dualistic model originated in the neurophysiological and behavioral study of macaque monkeys. Subsequently, it was revised on the basis of studies of neuropsychological disorders in brain-lesioned patients. It is corroborated by the study of size-contrast illusions in normal human subjects. Unlike conscious visual perception, grasping is relatively immune to size-contrast illusions. Depending on the requirements of the task, therefore, one and the same visual stimulus may yield visual illusions of variable strength. We submit that perceptual representations, which are more strongly sensitive to illusory effects, result from a "semantic" analysis of the visual input. Motor representations, which are less sensitive to illusory effects, result from a "pragmatic" analysis of the visual input. One fundamental question raised by our discussion is: How exactly do the computational and functional distinctions between a semantic analysis of the visual input and a pragmatic analysis of the visual input match the anatomical distinction between the ventral and the dorsal pathways?
Secondly, much of our discussion of experiments on size-contrast
illusions has relied on the distinction between two kinds of visual representations:
perceptual and motor representations. This reflects our theoretical bias
in favor of a representationalist approach to the puzzles of the visuomotor
transformation. In this paper, our representationalist bias has been limited
to the exploration of visually guided reaching and grasping. However, it
is our conviction that a representationalist framework can help us understand
better the puzzles of the phenomenology of conscious visual experience.
We will all too briefly outline our hunch with respect to the latter. Rapid
action directed towards an object does not require full visual awareness
of the object: one can grasp a cube without being aware either of its color
or of the fact that what one is grasping is a cube. Unlike the motor representation
of the target of an action delivered by the pragmatic analysis, the perceptual
representation of an object delivered by the semantic analysis makes us
visually conscious of objects. To be visually conscious of an object is
to be aware of several of its visual attributes (such as its shape, size
and color). You cannot be visually aware of the red cube in front of you
without being aware of its color and of its shape. We are not of course
claiming that a visual percept cannot misrepresent either the color or
the shape of the object. Rather, what we are claiming is that a perceptual
representation, unlike a motor representation of an object, results from
the binding of different visual attributes of the object.
Aglioti, S., DeSouza, J.E.X. & Goodale, M.A. (1995) Size-contrast illusions deceive the eye but not the hand. Current Biology, 5, 679-685.
Bridgeman, B., Kirsch, M. & Sperling, A. (1981) Segregation of cognitive and motor aspects of visual function using induced motion. Perception and Psychophysics, 29, 336-342.
Castiello, U. & Jeannerod, M. (1991) Measuring time to awareness. Neuroreport, 2: 797-800.
Castiello, U., Paulignan, Y. & Jeannerod, M. (1991) Temporal dissociation of motor responses and subjective awareness. A study in normal subjects. Brain, 114: 2639-2655.
Daprati, E. & Gentilucci, M. (1997) Grasping an illusion. Neuropsychologia, 35, 1277-1282.
Franz, V., Fahlee, M., Gegenfurtner, K.R. & Bülthoff, H.H. (1998) Size-contrast immusions deceive grasping as well as perception. Perception, 27, 140b.
Gentilucci, M., Chieffi, S., Daprati, E., Saetti, M.C. & Toni, I. (1996) Visual illusion and action. Neuropsychologia, 34, 369-376.
Goodale, M.A., Pélisson, D. & Prablanc, C. (1986) Large adjustments invisually guided reaching do not depend on vision of the hand or perception of target displacement. Nature, 320:748-750.
Goodale, M.A., Milner, A.D., Jakobson, L.S. & Carey, D.P. (1991) Perceiving the world and grasping it. A neurological dissociation. Nature, 349:154-156.
Goodale, M.A. & Milner, A.D. (1992) Separate visual pathways for perception and action. Trends in Neuroscience, 15:20-25.
Haffenden, A.M. & Goodale, M.A. (1998) The effect of pictorial illusion on perception and prehension. Journal of Cognitive Neuroscience, 10, 122-136.
Jeannerod, M. (1981) Intersegmental coordination during reaching at natural visual objects. In : Attention and Performance IX, J. Long and A. Baddeley (eds.), Erlbaum, Hillsdale, pp. 153-168.
Jeannerod, M. (1984) The timing of natural prehension movements. Journal of Motor Behaviour, 16: 235-254.
Jeannerod, M. (1994) The hand and the object. The role of posterior parietal cortex in forming motor representations. Canadian Journal of Physiology and Pharmacology, 72: 525-534.
Jeannerod, M. (1997) The Cognitive Neuroscience of Action. Oxford, Blackwell.
Jeannerod, M. & Decety, J. (1990) The accuracy of visuomotor transformation. An investigation into the mechanisms of visual recognition of objects. In: Vision and Action. The Control of Grasping. M. Goodale (ed.), Ablex, Norwood, pp. 33-48.
Jeannerod, M., Decety, J., & Michel, F. (1994). Impairement of grasping movements following a bilateral posterior parietal lesion. Neuropsychologia, 32: 369-380.
Meeres, S.L. & Graves, R.E. (1990) Localisation of unseen visual stimuli by humans with normal vision. Neuropsychologia 28: 1231-1237.
Milner, D.A. & Goodale, M.A. (1995) The Visual Brain in Action. Oxford, Oxford University Press.
Perenin, M.T. & Rossetti, Y. (1996) Residual grasping in an hemianopic field. A further instance of dissociation between perception and action. Neuroreport, 7: 793-797.
Taylor, J.L., McCloskey D.I (1990) Triggering of preprogrammed movements as reactions to masked stimuli. Journal of Neurophysiology 63: 439-446.
Ungerleider, L. & Mishkin, M. (1982) Two cortical visual systems. In: Analysis of Visual Behavior, D.J. Ingle, M.A. Goodale and R.J.W. Mansfield (eds.), Cambridge, MIT Press, pp. 549-586.
Weiskrantz, L. (1986) Blindsight. A case Study and Implications. Oxford, Oxford University Press.
Using fingers for comparing perceptual and visuomotor effects of an illusion.
Upper diagram: plot of measures obtained in 6 individual subjects during matching object size by interfinger distance. Hand is unseen during the task. Note linear correlation with trend to overestimation in some subjects (from Jeannerod and Decety, 1990).
Lower diagram: plot of maximum grip aperture prior to grasping objects in one individual subject. Black squares : hand seen during the grasp; white quares: hand unseen (from Jeannerod et al., 1994).
The Titchener-Ebbinghaus Illusion.
Upper drawing : display of the illusion as used by Aglioti et al (1995) and by Haffenden and Goodale (1998) during both perceptual and grasping tasks. Lower drawing : Method used by Franz et al. (1998) for comparing perception and grasping.
Results obtained by Haffenden and Goodale (1998) during perceptual estimate (manual estimation, right) and grasping (maximum grip aperture, left) of the illusion.
4 We ignore the fact that, in what Franz et al.
(1998) call "direct comparizon", unlike Aglioti et al. (1995) and Haffendale
and Goodale (1998), Franz et al. (1998) only present pairs of disks of
equal physical size, which look different in size, not pairs of disks of
different physical size, which look the same size.