This work was funded by the Chinese Scholarship Council (CSC) to K. M., and an infrastructure grant of the Netherlands Research Organization (NWO) to B. H.

<ol>
	<li>Tsakiris, M., Sch&#252;tz-Bosbach, S., Gallagher, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=On+agency+and+body-ownership:+Phenomenological+and+neurocognitive+reflections.">On agency and body-ownership: Phenomenological and neurocognitive reflections.</a> Conscious. Cogn. 16 (3), 645-660 (2007).</li><li>Botvinick, M., Cohen, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Rubber+hands+'feel'+touch+that+eyes+see.">Rubber hands 'feel' touch that eyes see.</a> Nature. 391 (6669), 756-756 (1998).</li><li>Armel, K. C., Ramachandran, V. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Projecting+sensations+to+external+objects:+Evidence+from+skin+conductance+response.">Projecting sensations to external objects: Evidence from skin conductance response.</a> Proc. R. Soc. B. 270 (1523), 1499-1506 (2003).</li><li>Tsakiris, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=My+body+in+the+brain:+A+neurocognitive+model+of+body-ownership.">My body in the brain: A neurocognitive model of body-ownership.</a> Neuropsychologia. 48 (3), 703-712 (2010).</li><li>Kalckert, A., Ehrsson, H. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Moving+a+rubber+hand+that+feels+like+your+own:+Dissociation+of+ownership+and+agency.">Moving a rubber hand that feels like your own: Dissociation of ownership and agency.</a> Front. Hum. Neurosci. 6 (40), (2012).</li><li>Kalckert, A., Ehrsson, H. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+spatial+distance+rule+in+the+moving+and+classical+rubber+hand+illusions.">The spatial distance rule in the moving and classical rubber hand illusions.</a> Conscious. Cogn. 30C, 118-132 (2014).</li><li>Ma, K., Hommel, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Body-ownership+for+actively+operated+non-corporeal+objects.">Body-ownership for actively operated non-corporeal objects.</a> Conscious. Cogn. 36, 75-86 (2015).</li><li>Ma, K., Hommel, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+role+of+agency+for+perceived+ownership+in+the+virtual+hand+illusion.">The role of agency for perceived ownership in the virtual hand illusion.</a> Conscious. Cogn. 36, 277-288 (2015).</li><li>Slater, M., Perez-Marcos, D., Ehrsson, H. H., Sanchez-Vives, M. V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Towards+a+digital+body:+The+virtual+arm+illusion.">Towards a digital body: The virtual arm illusion.</a> Front. Hum. Neurosci. 2 (6), (2008).</li><li>Sanchez-Vives, M. V., Spanlang, B., Frisoli, A., Bergamasco, M., Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Virtual+hand+illusion+induced+by+visuomotor+correlations.">Virtual hand illusion induced by visuomotor correlations.</a> PLOS ONE. 5 (4), e10381 (2010).</li><li>Spanlang, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=How+to+build+an+embodiment+lab:+Achieving+body+representation+illusions+in+virtual+reality.">How to build an embodiment lab: Achieving body representation illusions in virtual reality.</a> Front. Robot. AI. 1, 1-22 (2014).</li><li>Tsakiris, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Looking+for+myself:+Current+multisensory+input+alters+self-face+recognition.">Looking for myself: Current multisensory input alters self-face recognition.</a> PLOS ONE. 3 (12), e4040 (2008).</li><li>Sforza, A., Bufalari, I., Haggard, P., Aglioti, S. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=My+face+in+yours:+Visuo-tactile+facial+stimulation+influences+sense+of+identity.">My face in yours: Visuo-tactile facial stimulation influences sense of identity.</a> Soc. Neurosci. 5, 148-162 (2010).</li><li>Bufalari, I., Porciello, G., Sperduti, M., Minio-Paluello, I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Self-identification+with+another+person's+face:+the+time+relevant+role+of+multimodal+brain+areas+in+the+enfacement+illusion.">Self-identification with another person's face: the time relevant role of multimodal brain areas in the enfacement illusion.</a> J. Neurophysiol. 113 (7), 1959-1962 (2015).</li><li>Ma, K., Sellaro, R., Lippelt, D. P., Hommel, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mood+migration:+How+enfacing+a+smile+makes+you+happier.">Mood migration: How enfacing a smile makes you happier.</a> Cognition. 151, 52-62 (2016).</li><li>Ma, K., Hommel, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+virtual-hand+illusion:+Effects+of+impact+and+threat+on+perceived+ownership+and+affective+resonance.">The virtual-hand illusion: Effects of impact and threat on perceived ownership and affective resonance.</a> Front. Psychol. 4 (604), (2013).</li><li>Suma, E. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Adapting+user+interfaces+for+gestural+interaction+with+the+flexible+action+and+articulated+skeleton+toolkit.">Adapting user interfaces for gestural interaction with the flexible action and articulated skeleton toolkit.</a> Comput. Graph. 37 (3), 193-201 (2013).</li><li>Aron, A., Aron, E. N., Smollan, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Inclusion+of+Other+in+the+Self+Scale+and+the+structure+of+interpersonal+closeness.">Inclusion of Other in the Self Scale and the structure of interpersonal closeness.</a> J. Pers. Soc. Psychol. 63 (4), 596 (1992).</li><li>Russell, J. A., Weiss, A., Mendelsohn, G. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Affect+grid:+a+single-item+scale+of+pleasure+and+arousal.">Affect grid: a single-item scale of pleasure and arousal.</a> J. Pers. Soc. Psychol. 57 (3), 493-502 (1989).</li><li>Guilford, J. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> The nature of human intelligence. , (1967).</li><li>Ashby, F. G., Isen, A. M., Turken, A. U. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+neuropsychological+theory+of+positive+affect+and+its+influence+on+cognition.">A neuropsychological theory of positive affect and its influence on cognition.</a> Psychol. Rev. 106 (3), 529-550 (1999).</li><li>Hommel, B., Haggard, P., Eitam, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=34;Action+control+and+the+sense+of+agency&amp;#34.">34;Action control and the sense of agency&amp;#34.</a> The sense of agency. , 307-326 (2015).</li><li>Akbari Chermahini, S., Hommel, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+(b)link+between+creativity+and+dopamine:+Spontaneous+eye+blink+rates+predict+and+dissociate+divergent+and+convergent+thinking.">The (b)link between creativity and dopamine: Spontaneous eye blink rates predict and dissociate divergent and convergent thinking.</a> Cognition. 115 (3), 458-465 (2010).</li><li>Sanchez-Vives, M. V., Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=From+presence+to+consciousness+through+virtual+reality.">From presence to consciousness through virtual reality.</a> Nat. Rev. Neurosci. 6 (4), 332-339 (2005).</li><li>Slater, M., Spanlang, B., Sanchez-Vives, M. V., Blanke, O. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=First+person+experience+of+body+transfer+in+virtual+reality.">First person experience of body transfer in virtual reality.</a> PLOS ONE. 5 (5), (2010).</li><li>Maselli, A., Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+building+blocks+of+the+full+body+ownership+illusion.">The building blocks of the full body ownership illusion.</a> Front. Hum. Neurosci. 7 (83), (2013).</li><li>Pavone, E. F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Embodying+others+in+immersive+virtual+reality:+Electro+cortical+signatures+of+monitoring+the+errors+in+the+actions+of+an+avatar+seen+from+a+first-person+perspective.">Embodying others in immersive virtual reality: Electro cortical signatures of monitoring the errors in the actions of an avatar seen from a first-person perspective.</a> J. Neurosci. 26 (2), 268-276 (2016).</li><li>Tieri, G., Tidoni, E., Pavone, E. F., Aglioti, S. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Body+visual+discontinuity+affects+feeling+of+ownership+and+skin+conductance+responses.">Body visual discontinuity affects feeling of ownership and skin conductance responses.</a> Sci. Rep. 5 (17139), (2015).</li><li>Kilteni, K., Normand, J., Sanchez-Vives, M. V., Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Extending+body+space+in+immersive+virtual+reality:+A+long+arm+illusion.">Extending body space in immersive virtual reality: A long arm illusion.</a> PLOS ONE. 7 (7), (2012).</li><li>Banakou, D., Groten, R., Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Illusory+ownership+of+a+virtual+child+body+causes+overestimation+of+object+sizes+and+implicit+attitude+changes.">Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes.</a> Proc. Natl. Acad. Sci. U.S.A. 110 (31), 12846-12851 (2013).</li><li>Martini, M., Perez-Marcos, D., Sanchez-Vives, M. V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+color+is+my+arm?+Changes+in+skin+color+of+an+embodied+virtual+arm+modulates+pain+threshold.">What color is my arm? Changes in skin color of an embodied virtual arm modulates pain threshold.</a> Front. Hum. Neurosci. 7 (438), (2013).</li><li>Peck, T. C., Seinfeld, S., Aglioti, S. M., Slater, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Putting+yourself+in+the+skin+of+a+black+avatar+reduces+implicit+racial+bias.">Putting yourself in the skin of a black avatar reduces implicit racial bias.</a> Consc. Cogn. 22 (3), 779-787 (2013).</li><li>Shimada, S., Fukuda, K., Hiraki, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Rubber+hand+illusion+under+delayed+visual+feedback.">Rubber hand illusion under delayed visual feedback.</a> PLOS ONE. 4 (7), (2009).</li><li>Synofzik, M., Vosgerau, G., Newen, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Beyond+the+comparator+model:+A+multifactorial+two-step+account+of+agency.">Beyond the comparator model: A multifactorial two-step account of agency.</a> Conscious. Cogn. 17 (1), 219-239 (2008).</li><li>Hommel, B., M&#252;sseler, J., Aschersleben, G., Prinz, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+theory+of+event+coding+(TEC):+A+framework+for+perception+and+action+planning.">The theory of event coding (TEC): A framework for perception and action planning.</a> Behav. Brain. Sci. 24 (5), 849-878 (2001).</li><li>Hommel, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Action+control+according+to+TEC+(theory+of+event+coding).">Action control according to TEC (theory of event coding).</a> Psychol. Res. 73 (4), 512-526 (2009).</li></ol>

The authors have nothing to disclose.

In this article we described two detailed protocols for the virtual-hand and virtual-face illusion paradigms, in which our virtual-face study was the first to replicate the traditional stroking-induced face-ownership illusion in virtual reality, together with representative results from the two paradigms.
The significant synchrony effects indicate that we were successful in inducing illusory ownership for the virtual hand and the virtual face, similar to more traditional illusion paradigms. Being able to reproduce these effects by means of virtual reality techniques has considerable advantages11,24. Virtual-reality techniques are freeing the experimenter from the rather artificial and interruptive stroking procedure and opens new possibilities for experimental manipulations. For example, morphing virtual effectors allowed us to systematically manipulate the impact of the appearance of the virtual hand and the similarity between the virtual and the participant&#39;s real hand, or the facial expression of the virtual face. The impact of agency can also be systematically explored by varying the degree (e.g., immediacy) to which participants can control the movements of the artificial effector.
Another promising avenue for future virtual reality research are first person perspective (1PP) virtual reality experiences. 1PP experiences can create an immense sense of immersion and feeling of presence, on a completely different scale than a third person perspective virtual reality experience25,26,27,28. In 1PP experiences one can truly feel like one is the avatar, that one is literally embodying the avatar. This opens up possibilities for all kinds of manipulations such as detaching parts of a person&#39;s body28, elongating29, rescaling body parts30, or changing a person&#39;s skin color31,32.
As the present and many other findings demonstrate, controlling virtual events in a synchronous fashion strongly increases the perception of these events belonging to one&#39;s own body. For example, our findings from the hand study suggest that immediate control is an important cue for distinguishing between self-produced and other-produced events (i.e., personal agency) and between self-related and other-related events (i.e., body ownership). The findings presented here and elsewhere suggest that bottom-up information plays a decisive role in the emergence of phenomenal self-representation, even for body parts that are not as identity-related as one&#39;s own body part4.
The most critical part of the described protocols is the induction process, which introduces correlations between visual, tactile and motor (i.e., proprioceptive) information-these correlations allows the cognitive system to derive ownership and agency. As these correlations rely on the relative timing of the respective events, such as the delay between the participant&#39;s own movements and the movements of the artificial effector, it is crucial to keep processing delays (especially with regard to the translation of data from the dataglove to the motion of the virtual effector on the screen) to a minimum. With our experiment setup the maximum time delay is around 40 ms, which is hardly noticeable and does not hamper the perception of causality and agency. Shimada, Fukuda, and Hiraki33 have suggested that the critical time window for the occurrence of multisensory integration processes constituting the self-body representation is 300 ms, which means that longer delays are likely to reduce the perception of control over virtual events.
Another important aspect of the protocol is tight experimental control over the participant&#39;s hand or face movements, depending on the paradigm. During induction, active movements of the respective factor are essential, as the required intersensory correlations rely on active explorative movements on the side of the participant. It is thus important to encourage participants to move frequently and to engage in active exploration. In other phases of the experiment, movements can impair the measurement however. For instance, in the virtual-hand illusion paradigm, moving the left hand (from which SCR was recorded) is likely to render measurements of the SCR level noisy and unreliable.
A limitation of the virtual-hand illusion paradigm technique is that, for practical reasons, participants commonly wear a dataglove and orientation tracker during the entire experiment (so to minimize distraction). This may not be comfortable, which in turn may affect the mood or motivation of the participant. One possible solution for that problem would be the use of lighter equipment or custom-made wearables. Another limitation of our current virtual-face illusion paradigm technique is that the equipment only registers head movements but no changes in facial expression. Allowing participants to control the facial expressions of a virtual face is likely to contribute to ownership illusions, but this would require hardware and software that provides reliable detection and categorization of facial expressions in humans-which we do not yet have available in our lab. The use of for example real-time (facial) motion capture utilities would be of great benefit in overcoming these limitations and would allow us to increase sense of agency and ownership of avatars to significantly higher levels.
As suggested by the findings from our study8, people consider various sources of information and update their body representation continuously. They seem to use bottom-up information and top-down information in a compensatory fashion, in the sense that one source of information plays a stronger role in the absence of the other-similar to what has been assumed for the sense of agency34. This provides interesting avenues for future research, as it for instance suggests that ownership can be perceived even for artificial effectors in awkward postures, provided a sufficient degree of surface similarity, or vice versa (i.e., if the artificial effector perfectly aligns with the real effector but differs from it in terms of surface features). The available findings also suggest that the boundaries between self and others are rather plastic, so that features of another person or agent can be perceived as a feature of oneself, provided some degree of synchrony between one&#39;s own behavior and that of the other35,36.

According to Western philosophy, the human self consists of two aspects1: For one, we perceive our own body and our activities in the here and now, which creates a phenomenal self-representation (often called the minimal self). For another, we create more enduring representations of ourselves by storing information about our personal history, integrating new information into the emerging self-concept, and present ourselves to our social environment accordingly, which amounts to the creation of a so-called narrative self. The minimal or phenomenal self has been argued to emerge from two sources of information. One is top-down information about longer-lasting aspects of our body, such as information about the effectors we own or the shape of our face. The other is bottom-up information provided by self-perception in the current situation.
Investigations of the latter were strongly inspired by a clever study of Botvinick and Cohen2. These authors presented human participants with a rubber hand lying in front of them, close to one of their real hands, which however was hidden from view. When the real hand and the rubber hand were stroked in synchrony, so to create intermodal synchronous input, participants tended to perceive the rubber hand as part of their own body-the rubber-hand illusion. Further studies revealed that perceived ownership even went so far that participants would start sweating and trying to withdraw their real hand when the rubber hand was being attacked by a knife or otherwise being &#34;hurt&#34;3.
While Botvinick and Cohen have interpreted their findings to demonstrate that self-perception arises from the processing of bottom-up&#160;information, other authors have argued that the rubber-hand illusion results from the interaction between intermodal synchrony of input, a bottom-up source of information, and stored representations of one&#39;s own hands, a top-down source of information4. The idea is that the stimulus synchrony creates the impression that the real and the rubber hand are one and the same thing, and given that the rubber hand looks like a real hand, this impression is considered reality.
Later research by Kalckert and Ehrsson5 added a visuo-motor component to the rubber hand paradigm, which allows for the investigation of both perceived ownership (the impression that the artificial effector belongs to one&#39;s own body) and perceived agency (the impression that one is producing observed movements oneself). Participants were able to move the index finger of the rubber hand up and down by moving their own index finger, and the synchrony between real and rubber hand finger movements, the mode of movement (passive vs. active mode), and the positioning of the rubber hand (incongruous vs. congruous with regard to the participant&#39;s hand) were manipulated. The findings were taken to provide support for the notion that agency and ownership are functionally distinct cognitive phenomena: while synchrony of movement abolished both sense of ownership and agency, mode of movement only affected agency, and congruency of the rubber hand position had an effect on ownership only. The latter two result were replicated in a follow-up study in which the distance between real and rubber hand in the vertical plane varied6: ownership for the rubber hand decreased as its position increasingly mismatched the participant&#39;s real hand. However, agency was not affected by misplacements of the rubber hand in any condition.
However, recent research using virtual-reality techniques, which provide the participant with active control over the artificial effector, suggests that the role of the top-down part and the distinction between ownership and agency may have been overestimated7,8. These techniques have replaced the rubber hand by a virtual hand presented to participants on a screen in front of them or by means of virtual-reality glasses9. Participants commonly wear a dataglove that translates the movements of the participant&#39;s real hand into the movements of the virtual hand, either synchronously or asynchronously (e.g., with a noticeable delay). Similar to the rubber-hand illusion, synchronous translation strongly increases the participant&#39;s impression that the virtual hand becomes part of his or her own body10.
Employing virtual-reality techniques to create the rubber-hand illusion has several advantages over both the traditional rubber-hand paradigm and the combination of the rubber-hand paradigm with the visuo-motor components11. Moving one&#39;s hand and seeing an effector moving in synchrony with it creates a much more natural situation than facing a rubber hand and being stroked by an experimenter. Moreover, the virtual manipulation provides the experimenter with much more experimental flexibility and much more control over the perceptual relation between perceiving and moving one&#39;s real hand and one&#39;s perception of the event created by the artificial effector. In particular, using virtual techniques facilitates the manipulation of factors that are likely to influence perceived ownership and agency. For instance, the shape of the virtual hand can be modified much easier and faster than the shape of a rubber hand, and the movements of the virtual hand can be of any kind and for instance involve biologically impossible movements. Among other things, this facilitates exploring the limits of the illusion, as the artificial effector need not look like a hand but may be replaced by any kind of static or dynamic event. Of both practical and theoretical interest, a virtual effector is arguably a lot more immersive and feels a lot more real than a rubber hand, which is likely to reduce the necessity to invoke top-down interpretations to make sense of the present situation.
Ownership illusions have, however, not been restricted to hands. Tsakiris12 was the first to use the stroking technique to create the impression of participants that a static face in a picture presented in front of them is their own. Sforza et al.13 have also found evidence for this phenomenon, to which they refer as enfacement: participants incorporated facial features of a partner when their own and their partner&#39;s face were touched in synchrony. The neural mechanism underlying the enfacement illusion has recently been investigated by various researchers; for a comprehensive commentary and interpretation of the findings see Bufalari et al.14. We have recently turned the regular enfacement illusion design into a virtual-reality version (the virtual-face illusion), in which participants are controlling the movements of a virtual face in front of them by moving their own head15.
Here, we describe two experiments that used the virtual-hand illusion7 and the virtual-face illusion15 paradigms, respectively, to investigate self-representation. The virtual-hand experiment included three, completely crossed experimental factors: (a) the synchrony between (felt) real-hand and (seen) virtual-effector movements, which was either close to zero to induce ownership and agency or three seconds as a control condition; (b) the appearance of the virtual effector, which looked either like a human hand or like a rectangle (so to test the effect of real-virtual effector similarity on the ownership illusion); and (c) the opportunity to control the behavior of the virtual effector, which was either nonexistent in a passive condition or direct in an active condition. The virtual-face experiment included two, completely crossed experimental factors: (a) the synchrony between real-face and virtual-face movements, which was either close to zero to induce ownership and agency or three seconds as a control condition; and (b) the facial expression of the virtual face, which was either neutral or showing a smile, to test whether positive mood would lift the mood of the participant and improve his or her performance in a mood-sensitive creativity task.

<table><tbody><tr><td>Vizard (Software controlling the virtual reality environment)</td><td>Worldviz</td><td></td><td>Vizard allows importing hand models and integrating the hand, dataglove and orientation tracker modules through self-written command scripts. These scripts can be run to control the presentation of the virtual hand in the virtual environment, the appearance of the hand and the way it moves; they also control vibrator activities.</td></tr><tr><td>Cybertouch (Dataglove)</td><td>CyberGlove&amp;nbsp;Systems</td><td>Cybertouch</td><td>Participants wear this dataglove to control the movements of the virtual hand in the virtual environment. Measurement frequency = 100 Hz; Vibrator vibrational frequency = 0-125 Hz.</td></tr><tr><td>Intersense (Orientation tracker)</td><td>Thales</td><td>InertiaCube3</td><td>Participants wear the Intersense tracker to permit monitoring the orientation of their real hand (data that the used dataglove does not provide). Update rate = 180 Hz.</td></tr><tr><td>Biopac system (Physiological measurement device)</td><td>Biopac</td><td>MP100</td><td>The hardware to record skin conductance response.</td></tr><tr><td>Acquisition unit (Physiological measurement device)</td><td>Biopac</td><td>BN-PPGED</td><td>The hardware to record skin conductance response.</td></tr><tr><td>Remote transmitter (Physiological measurement device)</td><td>Biopac</td><td>BN-PPGED-T</td><td>Participants wear the remote transmitter on their left hand wrist; it sends signals to the Biopac acqusition unit.</td></tr><tr><td>Electrode (Physiological measurement device)</td><td>Biopac</td><td>EL507</td><td>Participants wear&amp;nbsp; the electrode on their fingers; it picks up skin conductance signals.</td></tr><tr><td>AcqKnowledge (Software controlling acquisition of physiological data)</td><td>Biopac</td><td>ACK100W, ACK100M</td><td>The software to record skin conductance responses.</td></tr><tr><td>Box</td><td>Custom-made</td><td></td><td>Participants put their right hand into the box</td></tr><tr><td>Computer</td><td>Any standard PC + Screen (could be replaced by VR glasses/devive)</td><td></td><td>Necessary to present the virtual reality environment, including the virtual hand.</td></tr><tr><td>Cape</td><td>Custom-made</td><td></td><td>Participants wear this cape on their right shoulder so they cannot see their right hand and arm.</td></tr><tr><td>Kinect (Head position tracker)</td><td>Microsoft</td><td></td><td>Kinect tracks the X-Y position of the participant&#039;s head. Recording frame rate = 30 Hz.</td></tr><tr><td>FAAST (Head position tracker software)</td><td>MXR</td><td>FAAST 1.0</td><td>Software controls Kinect and is used to track the position of the participant&#039;s head.</td></tr><tr><td>Intersense (Head orientation tracker)</td><td>Thales</td><td>InertiaCube3</td><td>Intersense tracks rotational orientation changes of the participant&#039;s head. Update rate = 180 Hz.</td></tr><tr><td>Facegen (Face-model generator software)</td><td>Singular Inversions</td><td>FaceGen&amp;nbsp;Modeller&amp;nbsp;</td><td>Facegen allows creating various virtual faces by varying various parameters, such as male/female-ness or skin color.</td></tr><tr><td>Cap</td><td>Any cap, e.g., baseball cap</td><td></td><td>The cap carries the Intersense orientation tracker.</td></tr><tr><td>Computer</td><td>Any standard PC + Screen</td><td></td><td>Necessary to present the virtual reality environment, including the virtual head.</td></tr></tbody></table>

creating virtual-hand and virtual-face illusions to investigate self-representation

Studies investigating how people represent themselves and their own body often use variants of &#34;ownership illusions&#34;, such as the traditional rubber-hand illusion or the more recently discovered enfacement illusion. However, these examples require rather artificial experimental setups, in which the artificial effector needs to be stroked in synchrony with the participants&#39; real hand or face&#8212;a situation in which participants have no control over the stroking or the movements of their real or artificial effector. Here, we describe a technique to establish ownership illusions in a setup that is more realistic, more intuitive, and of presumably higher ecological validity. It allows creating the virtual-hand illusion by having participants control the movements of a virtual hand presented on a screen or in virtual space in front of them. If the virtual hand moves in synchrony with the participants&#39; own real hand, they tend to perceive the virtual hand as part of their own body. The technique also creates the virtual-face illusion by having participants control the movements of a virtual face in front of them, again with the effect that they tend to perceive the face as their own if it moves in synchrony with their real face. Studying the circumstances that illusions of this sort can be created, increased, or reduced provides important information about how people create and maintain representations of themselves.

Virtual-hand Illusion
We ran several experiments using the virtual-hand illusion paradigm, to investigate how people represent their bodies, in this case their hands. The number of tested participants depended on the amount of conditions, usually around 20 participants for each condition. Here we provide relevant results for one of the most elaborated studies we conducted in our lab. We will restrict our discussion to the subjective data, the average of the Likert-scale responses to the four ownership questions (O1-O4) and the Likert-scale response to the agency question (A1).
In this study8, we systematically investigated the effects of synchrony (synchronous vs. asynchronous), appearance of the virtual effector (virtual hand vs. rectangle), and activity (passive vs. active) on the participants&#39; sense of ownership and sense of agency (all conditions were tested within participants). The results were the same for ownership and agency. As indicated in Figure 2, perceived ownership and agency were stronger if real and virtual hand moved in synchrony [F(1,43) = 48.35; p &#60; 0.001; and F(1,43) = 54.64; p &#60; 0.001; for ownership and agency, respectively], if the virtual effector was a hand than if it was a rectangle [F(1,43)=14.85; p &#60; 0.001; and F(1,43) = 6.94; p &#60; 0.02], and if the participant was active rather than passive [F(1,43) = 9.32; p &#60; 0.005; and F(1,43) = 79.60; p &#60; 0.001]. The synchrony effect replicates the standard virtual-hand illusion.
<img alt="Figure 2" src="/files/ftp_upload/54784/54784fig2.jpg" /> 
Figure 2: Ownership and agency ratings as a function of synchrony, appearance of the virtual effector, and activity of the participant. <a href="http://cloudflare.jove.com/files/ftp_upload/54784/54784fig2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 3" src="/files/ftp_upload/54784/54784fig3.jpg" /> 
Figure 3: Ownership and agency ratings as a function of synchrony and activity of the participant. Note that the synchrony effect is more pronounced for active participants. <a href="http://cloudflare.jove.com/files/ftp_upload/54784/54784fig3large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Even more interestingly, both ownership and agency showed a significant interaction between activity and synchrony [F(1,43) = 13.68; p = 0.001; and F(1,43) = 23.36; p &#60; 0.001; see Figure 3], but not between appearance and synchrony. This pattern suggests that activity plays a more dominant role for the ownership and the illusion than appearance does, it even showed that the illusory ownership perception is stronger in virtual than traditional rubber hand illusion paradigm. According to Hommel22, objective agency (i.e., the degree to which an external event can objectively be controlled) contributes to both subjective ownership and subjective agency, which explains why in this experiment, active, synchronous control over the virtual effector increased both subjective ownership and subjective agency.
While appearance failed to interact with synchrony, suggesting that the ownership illusion does not rely on appearance, it did produce a main effect. This indicates that appearance does have an impact on perceived ownership. It makes sense to assume that people have general expectations about what external objects might or might not be a plausible part of their body, which supports ownership perception in general but does not moderate the effect of synchrony. We thus conclude that multiple sources of information contribute to the sense of subjective ownership: general top-down expectations and bottom-up synchrony information. The relationship between these two informational sources does not seem to be interactive but compensatory, so that general expectations may dominate in the absence of synchrony, and vice versa.
Virtual-face Illusion
In another study, we investigated how people represent their face. We were able to replicate the traditional enfacement illusion in a virtual environment, which we refer to as the virtual-face illusion12. We further investigated whether people adopt the mood expressed by a virtual face they identify with. There was one within-participant factor-synchrony (synchronous vs. asynchronous) and one between-participant factor-facial expression (happy vs. neutral). The IOS ratings before the induction phase were subtracted from the IOS ratings after the induction phase, also the Affect Grid ratings before the induction phase were subtracted from the Affect Grid ratings after the induction phase, and these rating changes were used as the IOS and affect grid results.
The analysis of the ownership scores (O1-4), the agency scores (A1-2), and the IOS scale18 changes all showed main effects of synchrony [F(1,58) = 38.24; p &#60; 0.001; F(1,58) = 77.33; p &#60; 0.001; and F(1,58) = 43.63; p &#60; 0.001; respectively], showing that synchrony between one&#39;s own head movements and the movements of the virtual face increased perceived ownership and agency, and facilitated the integration of the other&#39;s face into one&#39;s own self (see Figure 4). Synchrony also improved mood, as indicated by a synchrony effect on the affect grid19 changes [F(1,58) = 7.99; p &#60; 0.01].
<img alt="Figure 4" src="/files/ftp_upload/54784/54784fig4.jpg" /> 
Figure 4: Ownership and agency ratings, as well as IOS changes, as a function of synchrony. Note that positive IOS changes imply an increase of integration of the other into one&#39;s self. <a href="http://cloudflare.jove.com/files/ftp_upload/54784/54784fig4large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 5" src="/files/ftp_upload/54784/54784fig5.jpg" /> 
Figure 5: Affect grid changes (positive values imply positive-going affect) and flexibility scores in the AUT, as a function of synchrony and the expression of the virtual face. Note that the interactions between synchrony and expression are driven by more positive-going mood and particularly good flexibility performance for the combination of synchrony and happy virtual face. <a href="http://cloudflare.jove.com/files/ftp_upload/54784/54784fig5large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
There were significant main effects of the facial expression on IOS changes, affect grid changes, and flexibility in the AUT20,21,23but more important was the fact that the affect grid changes and the flexibility score interacted with synchrony [F(1,58) = 4.40; p &#60; 0.05; and F(1,58) = 4.98; p &#60; 0.05; respectively]. As shown in Figure 5, participants reported improved mood and showed more creative behavior after enfacing (i.e., synchronously moving with) a happy face as compared to the conditions where they moved asynchronously with a happy face or synchronously with a neutral face.
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>F/P/PES</td>
			<td>EFF</td>
			<td>ACT</td>
			<td>SYN</td>
			<td>EFF*ACT</td>
			<td>EFF*SYN</td>
			<td>ACT*SYN</td>
			<td>EFF*ACT*SYN</td>
		</tr>
		<tr>
			<td rowspan="3">O1</td>
			<td>11.66</td>
			<td>10.11</td>
			<td>45.38</td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td>10.08</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>0.001</td>
			<td>0.003</td>
			<td>&#60;0.001</td>
			<td>0.003</td>
		</tr>
		<tr>
			<td>0.21</td>
			<td>0.19</td>
			<td>0.51</td>
			<td>0.19</td>
		</tr>
		<tr>
			<td rowspan="3">O2</td>
			<td rowspan="3"></td>
			<td>5.37</td>
			<td>47.65</td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>0.025</td>
			<td>&#60;0.001</td>
		</tr>
		<tr>
			<td>0.11</td>
			<td>0.53</td>
		</tr>
		<tr>
			<td rowspan="3">O3</td>
			<td>10.75</td>
			<td rowspan="3"></td>
			<td>41.30</td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td>9.81</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>0.002</td>
			<td>&#60;0.001</td>
			<td>0.003</td>
		</tr>
		<tr>
			<td>0.20</td>
			<td>0.49</td>
			<td>0.19</td>
		</tr>
		<tr>
			<td rowspan="3">O4</td>
			<td>12.86</td>
			<td>17.17</td>
			<td>15.12</td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td>10.60</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>0.001</td>
			<td>&#60;0.001</td>
			<td>&#60;0.001</td>
			<td>0.002</td>
		</tr>
		<tr>
			<td>0.23</td>
			<td>0.29</td>
			<td>0.26</td>
			<td>0.20</td>
		</tr>
		<tr>
			<td rowspan="3">O1-4</td>
			<td>14.85</td>
			<td>9.32</td>
			<td>48.35</td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td>13.68</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>&#60;0.001</td>
			<td>0.004</td>
			<td>&#60;0.001</td>
			<td>0.001</td>
		</tr>
		<tr>
			<td>0.26</td>
			<td>0.18</td>
			<td>0.53</td>
			<td>0.24</td>
		</tr>
		<tr>
			<td rowspan="3">A1</td>
			<td>6.94</td>
			<td>79.60</td>
			<td>54.64</td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td>23.36</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>0.012</td>
			<td>&#60;0.001</td>
			<td>&#60;0.001</td>
			<td>&#60;0.001</td>
		</tr>
		<tr>
			<td>0.14</td>
			<td>0.65</td>
			<td>0.56</td>
			<td>0.37</td>
		</tr>
	</tbody>
</table>
Table 1: F, P and Partial Eta squared (PES) values for the effects of the questionnaire item ratings, with df = 43. Factors are EFF: virtual effector (virtual hand vs. rectangle); ACT: activity (active exploration vs. passive stimulation); and SYN: synchrony (synchronous vs. asynchronous). Only results for significant effects are shown.
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>M/SE</td>
			<td>H-P-SY</td>
			<td>H-P-AS</td>
			<td>H-A-SY</td>
			<td>H-A-AS</td>
			<td>R-P-SY</td>
			<td>R-P-AS</td>
			<td>R-A-SY</td>
			<td>R-A-AS</td>
		</tr>
		<tr>
			<td rowspan="2">O1-4</td>
			<td>4.37</td>
			<td>3.44</td>
			<td>5.09</td>
			<td>3.50</td>
			<td>3.79</td>
			<td>3.14</td>
			<td>4.68</td>
			<td>3.05</td>
		</tr>
		<tr>
			<td>0.20</td>
			<td>0.23</td>
			<td>0.19</td>
			<td>0.25</td>
			<td>0.23</td>
			<td>0.23</td>
			<td>0.20</td>
			<td>0.21</td>
		</tr>
		<tr>
			<td rowspan="2">A1</td>
			<td>3.59</td>
			<td>3.11</td>
			<td>6.36</td>
			<td>4.36</td>
			<td>3.07</td>
			<td>2.57</td>
			<td>6.09</td>
			<td>3.80</td>
		</tr>
		<tr>
			<td>0.30</td>
			<td>0.32</td>
			<td>0.15</td>
			<td>0.33</td>
			<td>0.28</td>
			<td>0.27</td>
			<td>0.24</td>
			<td>0.33</td>
		</tr>
	</tbody>
</table>
Table 2: Means (M) and standard errors (SE) for the ownership and agency ratings in all eight conditions. H: hand; R: rectangle; A: active; P: passive; SY: synchronous; AS: asynchronous.
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>F/P/PES</td>
			<td>Facial expression</td>
			<td>Synchrony</td>
			<td>Facial expression* Synchrony</td>
		</tr>
		<tr>
			<td rowspan="3">Ownership (O1-4)</td>
			<td rowspan="3"></td>
			<td>38.24</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>&#60;0.001</td>
		</tr>
		<tr>
			<td>0.40</td>
		</tr>
		<tr>
			<td rowspan="3">Agency (A1-2)</td>
			<td rowspan="3"></td>
			<td>77.33</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>&#60;0.001</td>
		</tr>
		<tr>
			<td>0.57</td>
		</tr>
		<tr>
			<td rowspan="3">IOS Changes</td>
			<td>4.03</td>
			<td>43.63</td>
			<td rowspan="3"></td>
		</tr>
		<tr>
			<td>0.049</td>
			<td>0.001</td>
		</tr>
		<tr>
			<td>0.07</td>
			<td>0.43</td>
		</tr>
		<tr>
			<td rowspan="3">Affect Grid Valence Changes</td>
			<td>6.06</td>
			<td>7.99</td>
			<td>4.40</td>
		</tr>
		<tr>
			<td>0.017</td>
			<td>0.007</td>
			<td>0.041</td>
		</tr>
		<tr>
			<td>0.10</td>
			<td>0.13</td>
			<td>0.07</td>
		</tr>
		<tr>
			<td rowspan="3">AUT-Flexibility</td>
			<td>5.42</td>
			<td rowspan="3"></td>
			<td>4.98</td>
		</tr>
		<tr>
			<td>0.024</td>
			<td>0.03</td>
		</tr>
		<tr>
			<td>0.09</td>
			<td>0.08</td>
		</tr>
		<tr>
			<td rowspan="3">AUT-Fluency</td>
			<td rowspan="3"></td>
			<td rowspan="3"></td>
			<td>7.89</td>
		</tr>
		<tr>
			<td>0.007</td>
		</tr>
		<tr>
			<td>0.12</td>
		</tr>
	</tbody>
</table>
Table 3: F, P and Partial Eta squared (PES) values for relevant dependent measures, with df = 58 for questionnaire and IOS results, and df = 56 for the valence dimension of the affect grid mood and AUT results. Only results for significant effects are shown.
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>M/SE</td>
			<td>Neutral-SY</td>
			<td>Neutral-AS</td>
			<td>Happy-SY</td>
			<td>Happy-AS</td>
		</tr>
		<tr>
			<td rowspan="2">Ownership (O1-4)</td>
			<td>2.88</td>
			<td>2.03</td>
			<td>3.38</td>
			<td>2.36</td>
		</tr>
		<tr>
			<td>0.27</td>
			<td>0.16</td>
			<td>0.23</td>
			<td>0.22</td>
		</tr>
		<tr>
			<td rowspan="2">Agency (A1-2)</td>
			<td>5.90</td>
			<td>4.25</td>
			<td>6.16</td>
			<td>4.08</td>
		</tr>
		<tr>
			<td>0.20</td>
			<td>0.25</td>
			<td>0.13</td>
			<td>0.32</td>
		</tr>
		<tr>
			<td rowspan="2">IOS Changes</td>
			<td>0.37</td>
			<td>-0.80</td>
			<td>1.00</td>
			<td>-0.40</td>
		</tr>
		<tr>
			<td>0.21</td>
			<td>0.25</td>
			<td>0.20</td>
			<td>0.24</td>
		</tr>
		<tr>
			<td rowspan="2">Affect Grid Valence Changes</td>
			<td>-1.07</td>
			<td>-1.33</td>
			<td>0.60</td>
			<td>-1.20</td>
		</tr>
		<tr>
			<td>0.42</td>
			<td>0.33</td>
			<td>0.39</td>
			<td>0.31</td>
		</tr>
		<tr>
			<td rowspan="2">AUT-Flexibility</td>
			<td>5.87</td>
			<td>6.07</td>
			<td>7.43</td>
			<td>6.10</td>
		</tr>
		<tr>
			<td>0.31</td>
			<td>0.37</td>
			<td>0.29</td>
			<td>0.39</td>
		</tr>
		<tr>
			<td rowspan="2">AUT-Fluency</td>
			<td>7.27</td>
			<td>8.27</td>
			<td>9.73</td>
			<td>7.37</td>
		</tr>
		<tr>
			<td>0.51</td>
			<td>0.68</td>
			<td>0.68</td>
			<td>0.49</td>
		</tr>
	</tbody>
</table>
Table 4: Means (M) and standard errors (SE) for relevant dependent measures in the four conditions. Neutral: neutral facial expression; Happy: happy facial expression; SY: synchronous; AS: asynchronous.

Watch this Scientific Journal Video about Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation at JoVE.com

Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Here, we describe virtual-hand and virtual-face illusion paradigms that can be used to study body-related self-perception/-representation. They have already been used in various studies to demonstrate that, under specific conditions, a virtual hand or face can be incorporated into one&#39;s body representation, suggesting that body representations are rather flexible.

Studies investigating how people represent themselves and their own body often use variants of &#34;ownership illusions&#34;, such as the traditional ...

creating-virtual-hand-virtual-face-illusions-to-investigate-self

Universidad de Cartagena UDEC

Research

Education

JoVE Journal

JoVE Core

Social Psychology

Behavior

Foundations of Social Psychology

The Social Self

SCIENTISTS IN ACTION

13.1K Views.  Leiden University. Here, we describe virtual-hand and virtual-face illusion paradigms that can be used to study body-related self-perception/-representation. They have already been used in various studies to demonstrate that, under specific conditions, a virtual hand or face can be incorporated into one's body representation, suggesting that body representations are rather flexible.

Video: Creating Virtual-hand and Virtual-face Illusions to Investigate Self-representation

Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues

Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated characters) can evoke negative or positive affect (valence) depending on the object's degree of visual and behavioral realism along a dimension of human likeness (DHL) (Figure 1). But studies of affective valence of subjective responses to variously realistic non-human characters have produced inconsistent findings 3, 4, 5, 6. One of a number of reasons for this is that human likeness is not perceived as the hypothesis assumes. While the DHL can be defined following Mori's description as a smooth linear change in the degree of physical humanlike similarity, subjective perception of objects along the DHL can be understood in terms of the psychological effects of categorical perception (CP) 7. Further behavioral and neuroimaging investigations of category processing and CP along the DHL and of the potential influence of the dimension's underlying category structure on affective experience are needed. This protocol therefore focuses on the DHL and allows examination of CP. Based on the protocol presented in the video as an example, issues surrounding the methodology in the protocol and the use in "uncanny" research of stimuli drawn from morph continua to represent the DHL are discussed in the article that accompanies the video. The use of neuroimaging and morph stimuli to represent the DHL in order to disentangle brain regions neurally responsive to physical human-like similarity from those responsive to category change and category processing is briefly illustrated.

Investigation of the Uncanny Valley Hypothesis and affective experience requires an understanding of the hypothesis' dimension of human likeness (DHL). This protocol allows representation of the DHL and examination of categorical perception. Use of the same stimuli and fMRI to distinguish brain regions responsive to physical and category change is illustrated.

Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated ...

The Crossmodal Congruency Task as a Means to Obtain an Objective Behavioral Measure in the Rubber Hand Illusion Paradigm

The rubber hand illusion (RHI) is a popular experimental paradigm. Participants view touch on an artificial rubber hand while the participants' own hidden hand is touched. If the viewed and felt touches are given at the same time then this is sufficient to induce the compelling experience that the rubber hand is one's own hand. The RHI can be used to investigate exactly how the brain constructs distinct body representations for one's own body. Such representations are crucial for successful interactions with the external world. To obtain a subjective measure of the RHI, researchers typically ask participants to rate statements such as "I felt as if the rubber hand were my hand". Here we demonstrate how the crossmodal congruency task can be used to obtain an objective behavioral measure within this paradigm.
The variant of the crossmodal congruency task we employ involves the presentation of tactile targets and visual distractors. Targets and distractors are spatially congruent (i.e. same finger) on some trials and incongruent (i.e. different finger) on others. The difference in performance between incongruent and congruent trials - the crossmodal congruency effect (CCE) - indexes multisensory interactions. Importantly, the CCE is modulated both by viewing a hand as well as the synchrony of viewed and felt touch which are both crucial factors for the RHI.
The use of the crossmodal congruency task within the RHI paradigm has several advantages. It is a simple behavioral measure which can be repeated many times and which can be obtained during the illusion while participants view the artificial hand. Furthermore, this measure is not susceptible to observer and experimenter biases. The combination of the RHI paradigm with the crossmodal congruency task allows in particular for the investigation of multisensory processes which are critical for modulations of body representations as in the RHI.

We demonstrate how an objective measure can be employed in the widely employed rubber hand illusion paradigm. This measure is obtained by modifying the well-established crossmodal congruency task. This task allows the investigation of multisensory processes which are critical for modulations of body representations as in the rubber hand illusion.

The rubber hand illusion (RHI) is a popular experimental paradigm. Participants view touch on an artificial rubber hand while the participants' own hidden ...

Methods to Explore the Influence of Top-down Visual Processes on Motor Behavior

Kinesthetic awareness is important to successfully navigate the environment. When we interact with our daily surroundings, some aspects of movement are deliberately planned, while others spontaneously occur below conscious awareness. The deliberate component of this dichotomy has been studied extensively in several contexts, while the spontaneous component remains largely under-explored. Moreover, how perceptual processes modulate these movement classes is still unclear. In particular, a currently debated issue is whether the visuomotor system is governed by the spatial percept produced by a visual illusion or whether it is not affected by the illusion and is governed instead by the veridical percept. Bistable percepts such as 3D depth inversion illusions (DIIs) provide an excellent context to study such interactions and balance, particularly when used in combination with reach-to-grasp movements. In this study, a methodology is developed that uses a DII to clarify the role of top-down processes on motor action, particularly exploring how reaches toward a target on a DII are affected in both deliberate and spontaneous movement domains.

It is unclear how top-down signals from the ventral visual stream affect movement. We developed a paradigm to test motor behavior towards a target on a 3D depth inversion illusion. Significant differences are reported in both deliberate, goal-directed movements and automatic actions under illusory and veridical viewing conditions.

Kinesthetic awareness is important to successfully navigate the environment. When we interact with our daily surroundings, some aspects of movement are ...

Brain Imaging Investigation of the Neural Correlates of Observing Virtual Social Interactions

The ability to gauge social interactions is crucial in the assessment of others&#x2019; intentions. Factors such as facial expressions and body language affect our decisions in personal and professional life alike 1. These "friend or foe" judgements are often based on first impressions, which in turn may affect our decisions to "approach or avoid". Previous studies investigating the neural correlates of social cognition tended to use static facial stimuli 2. Here, we illustrate an experimental design in which whole-body animated characters were used in conjunction with functional magnetic resonance imaging (fMRI) recordings. Fifteen participants were presented with short movie-clips of guest-host interactions in a business setting, while fMRI data were recorded; at the end of each movie, participants also provided ratings of the host behaviour. This design mimics more closely real-life situations, and hence may contribute to better understanding of the neural mechanisms of social interactions in healthy behaviour, and to gaining insight into possible causes of deficits in social behaviour in such clinical conditions as social anxiety and autism 3.

This article demonstrates an experimental design in which whole-body animated characters are used in conjunction with functional magnetic resonance imaging (fMRI) to investigate the neural correlates of observing virtual social interactions.

The ability to gauge social interactions is crucial in the assessment of others&#x2019; intentions. Factors such as facial expressions and body language ...

Neuroscience

Holistic Facial Composite Creation and Subsequent Video Line-up Eyewitness Identification Paradigm

The paradigm detailed in this manuscript describes an applied experimental method based on real police investigations during which an eyewitness or victim to a crime may create from memory a holistic facial composite of the culprit with the assistance of a police operator. The aim is that the composite is recognized by someone who believes that they know the culprit. For this paradigm, participants view a culprit actor on video and following a delay, participant-witnesses construct a holistic system facial composite. Controls do not construct a composite. From a series of arrays of computer-generated, but realistic faces, the holistic system construction method primarily requires participant-witnesses to select the facial images most closely meeting their memory of the culprit. Variation between faces in successive arrays is reduced until ideally the final image possesses a close likeness to the culprit. Participant-witness directed tools can also alter facial features, configurations between features and holistic properties (e.g., age, distinctiveness, skin tone), all within a whole face context. The procedure is designed to closely match the holistic manner by which humans&#8217; process faces. On completion, based on their memory of the culprit, ratings of composite-culprit similarity are collected from the participant-witnesses. Similar ratings are collected from culprit-acquaintance assessors, as a marker of composite recognition likelihood. Following a further delay, all participants &#8212; including the controls&#160;&#8212; attempt to identify the culprit in either a culprit-present or culprit-absent video line-up, to replicate circumstances in which the police have located the correct culprit, or an innocent suspect. Data of control and participant-witness line-up outcomes are presented, demonstrating the positive influence of holistic composite construction on identification accuracy. Correlational analyses are conducted to measure the relationship between assessor and participant-witness composite-culprit similarity ratings, delay, identification accuracy, and confidence to examine which factors influence video line-up outcomes.

This applied experimental paradigm replicates circumstances by which an eyewitness to a real crime may create a holistic facial composite of the culprit from memory, and then attempt to identify the culprit from a video line-up containing the culprit or one in which he or she is not present.

The paradigm detailed in this manuscript describes an applied experimental method based on real police investigations during which an eyewitness or victim ...

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine

Proprioceptive drift, which is a perceptual shift in body-part position from the unseen real body to a visible body-like image, has been measured as the behavioral correlate for the sense of ownership. Previously, the estimation of proprioceptive drift was limited to one spatial dimension, such as height, width, or depth. As the hand can move freely in 3D, measuring proprioceptive drift in only one dimension is not sufficient for the estimation of the drift in real life situations. In this article, we provide a novel method to estimate proprioceptive drift on a 2D plane using the mirror illusion by combining an objective behavioral measurement (hand position tracking) and subjective, phenomenological assessment (subjective assessment of hand position and questionnaire) with a sophisticated machine learning approach. This technique permits not only an investigation of the underlying mechanisms of the sense of ownership and agency but also assists in the rehabilitation of a missing or paralyzed limb and in the design rules of real-time control systems with a self-body-like usability, in which the operator controls the system as if it were part of his/her own body.

This article describes a novel method to estimate proprioceptive drift on a 2D plane using the mirror illusion and combining a psychophysical procedure with an analysis using machine learning.

Proprioceptive drift, which is a perceptual shift in body-part position from the unseen real body to a visible body-like image, has been measured as the ...

Applying Incongruent Visual-Tactile Stimuli during Object Transfer with Vibro-Tactile Feedback

The application of incongruent sensory signals that involves disrupted tactile feedback is rarely explored, specifically with the presence of vibrotactile feedback (VTF). This protocol aims to test the effect of VTF on the response to incongruent visual-tactile stimuli. The tactile feedback is acquired by grasping a block and moving it across a partition. The visual feedback is a real-time virtual presentation of the moving block, acquired using a motion capture system. The congruent feedback is the reliable presentation of the movement of the block, so that the subject feels that the block is grasped and see it move along with the path of the hand. The incongruent feedback appears as the movement of the block diverts from the actual movement path, so that it seems to drop from the hand when it is actually still held by the subject, thereby contradicting the tactile feedback. Twenty subjects (age 30.2 &#177; 16.3) repeated 16 block transfers, while their hand was hidden. These were repeated with VTF and without VTF (total of 32 block transfers). Incongruent stimuli were presented randomly twice within the 16 repetitions in each condition (with and without VTF). Each subject was asked to rate the difficulty level of performing the task with and without the VTF. There were no statistically significant differences in the length of the hand paths and durations between transfers recorded with congruent and incongruent visual-tactile signals &#8211; with and without the VTF. The perceived difficulty level of performing the task with the VTF significantly correlated with the normalized path length of the block with VTF (r = 0.675, p = 0.002). This setup is used to quantify the additive or reductive value of VTF during motor function that involves incongruent visual-tactile stimuli. Possible applications are prosthetics design, smart sport-wear, or any other garments that incorporate VTF.

We present a protocol to apply incongruent visual-tactile stimuli during an object transfer task. Specifically, during block transfers, performed while the hand is hidden, a virtual presentation of the block shows random occurrences of false block drops. The protocol also describes adding vibrotactile feedback while performing the motor task.

The application of incongruent sensory signals that involves disrupted tactile feedback is rarely explored, specifically with the presence of vibrotactile ...

Virtual Hand with Ambiguous Movement between the Self and Other Origin: Sense of Ownership and 'Other-Produced' Agency

The feeling that a body part is one&#8217;s own body (sense of ownership; SoO) and the feeling based on the causal relationship between one&#8217;s will and action (sense of agency; SoA) have been recognized as the basis of our bodily self-consciousness. Previously, the illusory SoO over a fake body part (e.g., rubber hand) was introduced as the rubber hand illusion (RHI). Furthermore, it was determined that one could also evoke a SoA over an object with movements linked to the one&#8217;s prior intention. On the other hand, the postdictivity of our spontaneity implies that it is essentially inseparable whether actions originate from self or others. In other words, our SoA or daily experiences are obtained in such as inseparable scenario. Previous research, however, has maintained the premise that self- and other-origin movements are perceptually distinguishable. Here, we implement a protocol to make these aspects ambiguous for the participants and to estimate whether they can feel SoO and/or SoA and how. To this end, we employ an experiment using virtual reality, under which participants observe virtual fingers moving very slowly (or quickly or not moving) while their own fingers do not move. For evaluation of the illusory SoO, measurements of skin conductance responses against a knife threat are adopted. Additionally, we introduce face-to-face interviews to determine whether the feelings regarding the slow movement match the conventional SoA definition. Our representative results suggest that the SoO is evoked over the hand, and various attitudes to accept its movement as the participant&#8217;s own with awareness that they did not originate it are reported by the majority. As the results show, the novelty of this protocol is discovering that in such a situation, the SoO cooperates with an externally produced SoA to establish one&#8217;s own bodily experience rather than the independence of the SoO and SoA.

While previous research on bodily self-consciousness assumed that self- and other-origin movements were perceptually distinguishable, this protocol allows them to be ambiguous on a virtual hand with unintentional slight movements. This enables us to observe one&#8217;s experience formed by SoO and other-produced SoA, rather than the absence of SoA.

The feeling that a body part is one&#8217;s own body (sense of ownership; SoO) and the feeling based on the causal relationship between one&#8217;s will ...

Real-time Video Projection in an MRI for Characterization of Neural Correlates Associated with Mirror Therapy for Phantom Limb Pain

Mirror therapy (MT) has been proposed as an effective rehabilitative strategy to alleviate pain symptoms in amputees with phantom limb pain (PLP). However, establishing the neural correlates associated with MT therapy have been challenging given that it is difficult to administer the therapy effectively within a magnetic resonance imaging (MRI) scanner environment. To characterize the functional organization of cortical regions associated with this rehabilitative strategy, we have developed a combined behavioral and functional neuroimaging protocol that can be applied in participants with a leg amputation. This novel approach allows participants to undergo MT within the MRI scanner environment by viewing real-time video images captured by a camera. The images are viewed by the participant through a system of mirrors and a monitor that the participant views while lying on the scanner bed. In this manner, functional changes in cortical areas of interest (e.g., sensorimotor cortex) can be characterized in response to the direct application of MT.

We present a novel combined behavioral and neuroimaging protocol employing real-time video projection for the purpose of characterizing the neural correlates associated with mirror therapy within the magnetic resonance imaging scanner environment in leg amputee subjects with phantom limb pain.

Mirror therapy (MT) has been proposed as an effective rehabilitative strategy to alleviate pain symptoms in amputees with phantom limb pain (PLP). ...

Motor Imagery Using VR Digital Avatars in a Brain-Computer Interface System

Motor Imagery Performance Through Embodied Digital Twins in a Virtual Reality-Enabled Brain-Computer Interface Environment

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) technologies with the customization of three-dimensional (3D) avatars. Traditional approaches to rehabilitation often fail to fully engage patients, primarily due to their inability to provide a deeply immersive and interactive experience. This research endeavors to fill this gap by utilizing motor imagery (MI) techniques, where participants visualize physical movements without actual execution. This method capitalizes on the brain's neural mechanisms, activating areas involved in movement execution when imagining movements, thereby facilitating the recovery process. The integration of VR's immersive capabilities with the precision of electroencephalography (EEG) to capture and interpret brain activity associated with imagined movements forms the core of this system. Digital Twins in the form of personalized 3D avatars are employed to significantly enhance the sense of immersion within the virtual environment. This heightened sense of embodiment is crucial for effective rehabilitation, aiming to bolster the connection between the patient and their virtual counterpart. By doing so, the system not only aims to improve motor imagery performance but also seeks to provide a more engaging and efficacious rehabilitation experience. Through the real-time application of BCI, the system allows for the direct translation of imagined movements into virtual actions performed by the 3D avatar, offering immediate feedback to the user. This feedback loop is essential for reinforcing the neural pathways involved in motor control and recovery. The ultimate goal of the developed system is to significantly enhance the effectiveness of motor imagery exercises by making them more interactive and responsive to the user's cognitive processes, thereby paving a new path in the field of neurological rehabilitation.

Author Spotlight: Enhancing Neurorehabilitation Through EEG, Motor Imagery, and Virtual Reality

Motor imagery in a virtual reality environment has wide applications in brain-computer interface systems. This manuscript outlines the use of personalized digital avatars that resemble the participants performing movements imagined by the participant in a virtual reality environment to enhance immersion and a sense of body ownership.

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) ...