No Fulfillment Without Anticipations and VRs Place Of Illusions
Interview by Richard Marshall
Michael Madary works on the philosophy of mind and the ethics of emerging technology, especially immersive technology such as virtual reality. His research is interdisciplinary, drawing from psychology and neuroscience. In February of 2016, he published with Thomas Metzinger the first code of ethics for research and consumer use of VR, which has received widespread media attention. In addition to the ethics of technology, he has also published widely in the philosophy of perception.
3:AM: What made you become a philosopher?
Michael Madary: Probably the main factor that led me into philosophy was my educational background. I was raised and educated as a Roman Catholic and decided around the age of 17 or 18 to investigate the philosophical ideas behind the religious doctrine that I had been taught. So I started, at my father’s suggestion, with G. K. Chesterton’s biography of Aquinas. Around that time, a friend of his gave me a used copy of Russell’s A History of Western Philosophy. Russell is unfair to a number of philosophers in there, so it’s not a work I would recommend to beginning students of philosophy, but it did open my eyes to some of the diversity of philosophical thought over the centuries. In an attempt at intellectual honesty, I thought it would be important to learn more about philosophical views that were critical of monotheism. This decision motivated me to pick up Nietzsche, whose writing I found exhilarating at that age.
My formal study of philosophy then began as an undergraduate at the University of Dallas, which has a rigorous “core curriculum” – all students regardless of major are required to study great works of literature, history, and philosophy from the Western tradition. There I studied under Robert Wood, who impressed upon his students that the major thinkers throughout history offer valuable core insights into human experience. He emphasized the commonalities rather than the differences between major historical figures, reading Plato and Aristotle, for instance, as proto-phenomenologists. Instead of focusing on the history of philosophy, I wanted to apply this ecumenical approach to the contemporary landscape. In particular, I decided to pursue the dialogue between analytic philosophy of mind, continental phenomenology, and the empirical sciences of the mind.
3:AM: You’re working in the field of philosophy of mind, and have written extensively about visual perception. You argue that seeing things is an ongoing process of anticipation and fulfillment. Before telling us what you mean by this, can you say what alternative models have been taken seriously to explain this, and in particular what you call ‘the historical tension between subjective and objective methods of investigating the mind’ looks like?
MM: There have been numerous accounts of visual perception on offer throughout history. In recent times, the most influential alternative to the account that I propose has been David Marr’s theory of vision. On Marr’s account, visual perception involves the computation of increasingly sophisticated visual representations in stages that progress from the retinal image to what he called the primal sketch and then the 2.5 dimensional sketch and finally the 3-dimensional model representation of shape for the purpose of object recognition. The process, according to Marr, is entirely feedforward, with information flowing in one direction from the retina. It is important to note that Marr himself was not concerned with explaining conscious visual experience. His ideas were put to the service of explaining conscious vision by Ray Jackendoff and, more recently, Jesse Prinz. They both hold that Marr’s 2.5 dimensional sketch, which is a representation of visual surfaces relative to the perceiver’s perspective, is the level at which visual consciousness occurs.
The historical tension between objective and subjective methods of investigating the mind arises because objective methods in natural science rely on what is publicly observable but conscious experience is, by nature, not publicly observable. This tension can be found both in the history of psychology as well as familiar themes in philosophical work on consciousness. Early experimental psychologists around the beginning of the 20th century, such as Wundt and Titchener, made use of introspection in their laboratories. Behaviorism then followed as an attempt to purge psychology of subjective methods such as introspection and establish psychology as an objective science. This tension is addressed in various ways through canonical works in the philosophy of consciousness, such as Nagel’s “What is it like to be a bat?” and Chalmers’ hard problem.
3:AM: So against such ideas you propose a new model. Can you sketch for us what you mean by saying that seeing is this ongoing process of anticipation and fulfillment.
MM: Both Nagel and Chalmers endorse the positive recommendation that we might make progress with understanding consciousness if we focus on the structure instead of the content of different conscious modalities. My strategy is to follow this recommendation by investigating the general structure of visual experience. To that end, I identify three features of all visual experience that any account of the structure of vision ought to accommodate: visual experience is perspectival, temporal, and indeterminate. Marr’s 2.5 dimensional sketch accommodates the perspectival nature of vision, but neglects the other two features.
My suggestion that vision is an ongoing process of anticipation and fulfillment is an attempt to accommodate all three features at once. Due to the perspectival nature of perception, our experience of factual (non-perspectival) properties such as volumetric shape and size is always incomplete. Some aspects of objects are always outside of one’s current perspective. The basic idea is that vision always involves implicit anticipations about how things will appear as we move and obtain new perspectives on the world. When things are going well, those anticipations are fulfilled. The structure of anticipation and fulfillment accommodates the temporality of visual experience because visual anticipations are inherently future-directed. The indeterminacy of vision is also built into the structure because movements bring what is indeterminate in the periphery into more determinacy as we gain better perspectives on objects of interest.
3:AM: It makes it sound like seeing is a kind of constant guessing, or betting, or anticipating. It doesn’t it feel like that to me, and yet you say that the phenomenology of perception is best described as that. Where am I going wrong? Can you give me an example to get it clear what this anticipation is.
MM: The visual anticipation that I am describing is ongoing and continuous. It’s what enables us to see when we open our eyes and look around. Although it is possible deliberately to anticipate something in vision, deliberate anticipation is not what I have in mind here. It may not be obvious that we always expect the world to appear a particular way as we move, but there are situations in which anticipations can be noticed fairly easily, especially situations that involve disappointment of anticipations or perceptual novelty. Recall a situation in which you moved to gain a better perspective on a novel object and were surprised by how it appeared from the hidden side. That surprise occurs, I argue, because the new appearance doesn’t match what you had implicitly anticipated. Modern sculpture can be helpful for illustrating visual anticipations because the precise shape of the sculpture is often unclear from one’s initial perspective. Our anticipations regarding the hidden side of a modern sculpture tend to be more indeterminate than our anticipations about the hidden sides of more familiar objects. Contrast a situation in which you look at a modern sculpture for the first time with a situation in which you look at a familiar object, such as your favorite coffee cup. I suggest that you will anticipate seeing the hidden side of each object as you change your perspective. But the anticipations in the case of the sculpture will be more indeterminate than in the case of the coffee cup. It may even seem to you as if you are “guessing” in the case of the sculpture.
Another way to see the ongoing nature of visual anticipations is to consider the mechanics of virtual reality using a head-mounted display. VR only works when the visual display changes in precise ways corresponding to our movements. Without such precision, the illusion of being in a virtual world is broken and people experience motion sickness. So it seems as if the visual display must correspond, within a set of possibilities, with how our visual systems expect visual sensations to change as we move.
3:AM: Is anticipation as you’re using it then something distinct from having beliefs? I’m not clear how this works: if I anticipate I’m going to see a tiger in the next room don’t I have to have a belief that there’s a tiger in the next room, or if my anticipation is certain then stronger, I have not a belief but knowledge that there’s a tiger there. Again, where am I going wrong?
MM: It depends on how you want to understand beliefs. If we take a relatively standard understanding of beliefs, then visual anticipations are similar to beliefs in some ways and unlike beliefs in other ways. For instance, visual anticipations are similar to beliefs in that they have a mind-to-world direction of fit. They are unlike beliefs in that they cannot be individuated and completely reported in natural language. Visual anticipations are far more fine-grained than typical reports of visual content using natural language.
The difference between believing that there is a tiger in the next room can be distinguished from visually anticipating that there is a tiger in the next room by considering the details of one’s perceptual situation. Husserl suggested that visual anticipations are “stirred-up” (erregen) based on our movements and perceptual context. So your belief that there is a tiger in the next room may never lead to visual anticipations if you take the safe course of action and avoid entering, or even peering into, the room. If you take action to visit the tiger, then those anticipations will be stirred-up and then fulfilled if indeed there is a tiger waiting for you.
One could have a liberal conception of what counts as a belief in order to include visual anticipations as a type of belief. Or one could place visual anticipations as a separate type of mental state. The important point is that the case for visual anticipations should be evaluated on the basis of my arguments for it, not on whether it fits with an existing taxonomy of mental states.
3:AM: What does your notion of perspectival connectedness explain?
MM: The notion of perspectival connectedness was initially developed by Susanna Siegel. She developed a thought experiment involving odd experiences of a doll on a shelf in conjunction with her method of phenomenal contrast to conclude that we are perspectivally connected to the world during our normal course of perceptual experience. To be perspectivally connected means that when we substantially change our perspective on objects, we expect changes in our visual phenomenology. She leaves it open whether we expect a specific type of change in phenomenology as a result in our change in perspective. I argue that we do expect a specific change by introducing a variation on her thought experiment, one involving surprising changes in experience. We do not just expect any kind of change, but rather expect change with some degree of specificity. When I change my perspective on a tea cup, for example, I expect to see the hidden sides of the tea cup. Nearly anything apart from a change that reveals a hidden side of the cup as a result of my self-generated movement would be surprising to me, indicating that it was an unexpected change. This variation of her thought experiment is at the center of my phenomenological argument in support of visual anticipation.
3:AM: Can you explain how important the notion of conditional contents as well as contents are for your explanation?
MM: There are more-or-less two views on perceptual content that have been taken seriously recently in the philosophy of perception. The first, orthodox view is that perceptual content is a kind of propositional content, such as “I see that the cat is on the mat.” The second view, gaining acceptance in recent years, is in the family of disjunctivism or naïve realism. According to the second view, very roughly, perception itself has no content whatsoever. I propose a middle ground between these two extremes.
On the view that I have developed, which is based on some of Husserl’s work, the content of visual perception is the content of the visual anticipations that are stirred-up as we visually explore the environment. Such content can be expressed in the form of conditionals because visual anticipations are conditional upon self-generated movements or upon changes in the visual environment: If I lean forward in such-and-such a manner, then the handle of the teacup will enter into view. If the ball rolls into the shadows, its surface will appear differently. The content of visual anticipations is messy and dynamic because it is closely dependent upon our ever-changing perspective on the world. I always only see that the cat is on the mat from a particular perspective and in particular lighting conditions. Importantly, even though visual anticipations reflect our perspective, having such anticipations depends upon one’s being intentionally directed to perspective-independent properties such as volumetric shape. We are intentionally directed to volumetric shape (and other perspective-independent properties), but this intentional directedness becomes manifest in perception through the conditional fine-grained contents of visual anticipations. As the visual anticipations are fulfilled, our perceptual evidence for non-perspectival properties increases accordingly.
The orthodox propositional view of perceptual content is flawed because it is not well-equipped to accommodate the perspectival, temporal, and indeterminate features of visual perception. It is a holdover from the dominance of philosophy of language in the 20th century. The view that I am recommending is specifically designed to accommodate those features of vision. Those who would reject perceptual content entirely tend to do so out of the problems with the orthodox propositional view. Charles Travis, for instance, argues forcefully that there is no particular proposition, no “face value” that perception offers for us to reject or accept. I agree with him, and I have tried to show a way that avoids these shortcomings of the propositional view without rejecting perceptual content altogether.
3:AM: Do fulfilled anticipations equate with accuracy about the external world?
MM: No. It is possible to have a series of fulfilled anticipations during, for example, a rich and extended hallucinatory episode. We may conclude that we had been hallucinating due to a subsequent series of fulfilled anticipations that “cancel out” the entire hallucinatory series. It is also possible that we never have a series of fulfilled anticipations that conflict with the hallucinatory series. In that case, we may never realize that we had been hallucinating.
3:AM: One element that you are particularly keen to emphasise, and one that you think some models exclude to their detriment, is the temporal dimension to all this. Can you say why temporal extension is important, and why you conclude that momentary experiences don’t have contents attributing factual properties?
Kant placed great emphasis on the fact that all of our experiences unfold in the flow of time. I take this insight to be undeniable and valuable. Following Kant, the temporality of experience is a central theme in classical phenomenology. Apart from some notable exceptions, such as Susan Hurley, Alva Noë, and, in his own idiosyncratic way, Dan Dennett, recent analytic philosophers of perception have investigated perception as a separate issue from that of temporality. My view is that omitting temporal structure from one’s description of perceptual experience leaves one with a distorted description that can generate pseudo-problems. For example, without temporality, one is unable to include the important ways in which previous experience influences future experiences. Similarly, one is unable adequately to accommodate the fact that visual perception typically involves active exploration. As my friend Sascha Fink pointed out to me, it is not unlikely that the absence of temporality as a theme in recent philosophy of perception is partly due to the fact that much of perceptual psychology throughout history has involved subjects remaining still and looking at relatively static images on a screen. This situation will soon change as the new affordability and availability of virtual reality technology enables perceptual psychologists to investigate the dynamics of visual perception more easily.
On the issue of the content of momentary experiences, I suggest that focusing on momentary experiences is not optimal and prefer to consider experiences as temporally extended, for reasons just given. That said, examples of momentary experiences can be helpful at times for the purposes of illustration. My claim, again following Husserl, is that the representation of factual (non-perspectival) properties such as volumetric shape is always incomplete at any particular moment. The factual properties of objects become increasingly manifest to us through fulfillments that involve multiple perspectives over time.
3:AM: Do you think there are systematic structural correspondence between sub-personal and person level anticipations and fulfillments? Is there evidence you can draw on to support this?
MM: There are a number of empirical models of sub-personal visual processing that are structurally similar to the personal-level description of vision as anticipation and fulfillment. Some of these models are becoming increasingly popular these days under the banner of predictive processing. The evidence in support of these models comes from both perceptual psychology and theoretical neuroscience. For example, there is the psychological evidence in support of the close connection between action and visual perception, familiar from sensorimotor approaches to vision. Also, it well-established that the cortex has massive feedback connectivity as well as its own intrinsic ongoing dynamics. These are the properties that one would expect in a system that is essentially predictive or anticipatory. One of the main themes in my recent book is that these two methods of investigating visual experience – the methods of phenomenological description and empirical research – independently converge on the conclusion that vision is an ongoing process of anticipation and fulfillment.
3:AM: Couldn’t we have anticipations, as you describe them, but they fail to cause fulfillments? Isn’t it possible to have anticipations but that they are causally inert? Doesn’t that look a bit like a Humean problem for you, that there’s no necessary link between anticipations and fulfillments, and without such a link we have still more to explain?
The phenomenological description of anticipations and fulfillments is not itself a causal explanation. It is merely descriptive. Still, if the experiential flow of anticipations and fulfillments share a structural similarity with sub-personal causal models, then that flow should exhibit some kind of causal regularity reflecting that of the models. Here is a sketch of such a causal regularity among anticipations and fulfillments: Suppose that anticipations correspond roughly to predictive top-down signals and that fulfillments correspond roughly to the match between incoming sensory signals, on one hand, and the predicted top-down sensory signals, on the other. (This match likely occurs at multiple levels of neural processing, but the details are not crucial to illustrate the point.) If there is going to be fulfillment, then there must be a match between sensation and prediction. This match is partly caused by the incoming sensory signal and partly caused by the prediction. Thus, anticipations, understood as top-down predictions, partially cause their fulfillments, understood as the match between sensory signals and the top-down predictions. There can be no fulfillment without anticipations because there can be no match between sensory input and predicted input without a predicted input. To give an analogy, the bicycle that you give a child for her birthday cannot fulfill her wish to receive a bicycle unless she had a wish for a bicycle to begin with. The wish partially causes the fulfillment. A similar causal role can be identified when there is disappointment or surprise instead of fulfillment. A perceptual experience that fails to fulfill anticipations cannot occur unless there is some anticipation that remains unfulfilled, some prediction that the incoming sensory signal fails to match.
3:AM: Historically, which philosophers in the past have been important to your thinking in the area of phenomenology of perception – is your work a break from old style phenomenologists or is there a genuine link which would suggest that in this area there has been genuine progress in philosophy of mind?
MM: Easily the biggest influence on my thinking about the phenomenology of perception has been Edmund Husserl. He first developed the idea that the structure of perception is one of anticipation and fulfillment. This idea can be found in his early Logical Investigations (1900) and it is revisited throughout his later works. So, there is undoubtedly a strong link between the view that I am developing and these ideas from early phenomenology. I do see this continuity as progress.
There are two main areas in which there is an obvious difference between my work and Husserl’s project. First, Husserl’s writings did not make reference to empirical results. My work draws heavily from psychophysics and theoretical neuroscience. Already in the 1940s, though, Maurice Merleau-Ponty was exploring connections between phenomenological themes and empirical work, so my interdisciplinarity shares methodological similarities with at least some work in the classical phenomenological tradition. The second main difference is that much of Husserl’s philosophical project – from the first decade of the 20th century onwards – was committed to transcendental idealism as an ambitious solution to the problem of global skepticism. My work in the philosophy of perception so far deliberately avoids any such metaphysical commitments.
3:AM: Virtual Reality is a field that many people are finding increasingly interesting, important and challenging. You’ve recently written about the ethical challenges raised by this field. You published a code of ethics for people working in the field. Why do you see an ethical dimension to the work, and what do you propose? And do you think there’s an analogous need for an ethical code for those people working in other areas of mind research, such as mind-altering drugs?
MM: Thomas Metzinger and I published the first code of ethics for virtual reality (VR), which offers recommendations both for researchers in the field as well as for the general public. Many new technologies raise ethical questions concerning how they should be used in order to respect principles such as beneficence, autonomy, and justice. VR is no exception. A distinguishing feature of VR is that it mimics the way in which we normally perceive the world. As I explained in my replies about my work on visual perception, we normally perceive by implicitly anticipating the consequences of self-generated movements. In VR, our self-generated movements are carefully tracked and the visual display changes quickly and precisely in response to those movements, fulfilling our anticipations. By effectively creating an artificial ongoing cycle of anticipation and fulfillment, VR is able to induce what is known in the literature as the place illusion. We feel as if we are really in the virtual environment, in a different place. Even though we might maintain the true belief that we are, say, in a lab or in our living room, our bodies and our minds react as if the events in the virtual environment are really occurring. Research conducted over the past several years suggests that even brief experiences in VR, of about 10 minutes or so, can have a lasting and unconscious influence on one’s behavior after leaving the virtual environment. While all media can have some psychological effect on us, VR appears to be an especially powerful medium in this regard.
We make a number of recommendations in the code of ethics, summarized in a table at the end of it. We call for research into the long-term effects of VR as well as the effects of various types of content. But we warn that this research may exclude some of the most vulnerable users, such as children and those with latent mental illness, due to ethical constraints. We also think that users should be made aware of the results from the research so far, results suggesting that VR can have a lasting influence on behavior. The power of VR opens up new possibilities for psychological manipulation, so users ought to consider the intentions of those who provide their content. The technology could be used, for example, as a powerful and perhaps subtle tool for political indoctrination. One final issue that I’ll mention is that of privacy. Of course, online privacy is already a major issue in the ethics of technology. The case of VR is especially concerning because motion capture technology can be used to track our detailed bodily and facial movements in order to animate our avatars in real-time. As is well-known from work on embodied cognition, our bodily movements and facial expressions can reveal a great deal about our mental states.
From what I can tell, there are already well-established ethical protocols for research using mind-altering drugs, developed several decades ago. There is a need for a code of ethics in VR because the technology is newly becoming widely available to researchers and to the general public.
3:AM: As technology becomes increasingly sophisticated, and AI looking more and more smart, are you happy that the people working in these areas are sufficiently aware of the ethical challenges. Do you see a need to extend your ethical concerns, and your code, from VR to other areas to look at whether we should be concerned to protect intelligent and/or conscious technology as well as ourselves. Do you think that ideas of the singularity are being taken seriously enough, and do you think it does provide an existential threat of our own making? Should we be alarmed?
MM: No, I do not think that developers of technology are sufficiently aware of the ethical challenges. There is some talk about ethical design within the engineering community, but much of it seems to have a fig leaf quality. We still need a fundamental change in the way we approach innovation. Engineers should strive to develop new technology with the goal of promoting human flourishing. Our devices have been designed to hijack our attention, and they do so remarkably effectively, especially with younger users. The educational system in the United States does a disservice to the youth by promoting “technology in the classroom” that requires them to be in front of screens for hours on end in order to learn, conditioning children to be consumers of proprietary software. We should pursue innovation in the service of humanity, not innovation for its own sake.
Regarding the technological singularity, I think that the ethics of technology can become less effective the more it focuses on speculative future scenarios. It is important, of course, to anticipate undesirable consequences, but we should not be oblivious to our current situation. If you consider some of Andy Clark’s insights about the extended mind, for example in his book Natural-Born Cyborgs, then it’s not important that our nervous systems are not physically connected, not hard-wired, to the web. As long as our devices are ready-to-hand, and they usually are, then our minds are connected to an enormous web of information that deeply influences our thoughts and behavior. This information is curated by a large and powerful company whose business model is the collection of personal data and whose personnel enjoy a revolving-door with some of the highest positions in government. Not long ago, the scenario I’ve described might have been a premise for dystopian science fiction. Today it is reality.
3:AM: And for the curious readers here at 3:AM, can you recommend five books other than your own that will take us further into your philosophical world?
MM: Aristotle’s On the Soul develops themes that are still central today in philosophy of mind, such as the importance of embodiment, the interdependence of action and perception, and a hylomorphic approach to the metaphysics of mind.
I must recommend one of Husserl’s works, and I notice that his Logical Investigations has already been recommended in your recent interview with Peter Simons. Instead of the Logical Investigations, I’ll recommend Husserl’s Ideas II. It’s a work that covers areas for which Husserl is not widely known and it was a major influence on Merleau-Ponty.
Dan Zahavi’s Subjectivity and Selfhood is an excellent treatment of main themes in the philosophy of mind from the perspective of classical phenomenology.
Sherry Turkle’s Alone Together is a rich and insightful exploration of the social and psychological impact of contemporary information technology.
My final recommendation is Sarah Spiekermann’s Ethical IT Innovation. It is a much-needed practical guide to technological innovation from an ethical perspective.
ABOUT THE INTERVIEWER
Richard Marshall is biding his time.
Buy his new book here or his first book here to keep him biding!