#1811: [2018] LBEの先駆け「VR_I」――スケールを操る集団ダンスと演劇体験

The Voices of VR Podcast. different immersive festivals since 2018. So this first interview is with a piece called VR underscore I, which was at Sundance New Frontier in 2018. And the directors were Sylvain and Cecilia. They had previously done in 2016 a piece called Real Virtuality, Immersive Explorers. And that was the first time that I had like a full body tracked immersive experience that made me feel like I was completely immersed within a world. I had just also seen The Void, which had redirected walking, walking through a space. But there wasn't as much fidelity in terms of the full body tracking.

And so it was really quite an experience back in 2016 for me to see that because it started to give me this sense of, OK, this is what it means to have my body in a space. And a couple of years later, they came back with the VRI and they had actually collaborated with Gilles Javan to be the choreographer for this piece that ended up being like you're in this kind of space where there's like other people that are there. And so it's a shared virtual reality experience, but there's also like performance capture of dancers throughout the space. And so the dancers have all these different scales, they're like giants, and they're, I think, also changing scale. And so it ended up being this whole wild, trippy adventure as a kind of collective experience where people that were actually there were also dancing around and moving their bodies with these other dancers, but also kind of seeing this larger performance within the context of this world. And so this is a piece that also went off and started touring around and was actually really quite successful in terms of a broader location-based entertainment. And so in my interview that I did with Gilles Jaban at Venice Immersive 2026, he sort of does a whole recap of some of this whole journey into XR.

And so this was just giving a little bit of like where some of that actually began with this VRI project that was premiering at Sundance New Frontier in 2018. So we're covering all that and more on today's episode of the Voice of the VR podcast. So this interview with Sylvain and Cecilia happened on Friday, January 19th, 2018 at Sundance New Frontier in Park City, Utah. So with that, let's go ahead and dive right in. So I'm Sylvain Chaguet. I'm French. So we started working with VR now four or five years ago when first Oculus came back and everyone was really excited about it and so on. So Sylvain and I, we are motion capture specialists. So we have a research center where we carry out a lot of research projects using motion capture technologies. so we thought it was interesting to try to combine the two technologies together so basically having mocap and VR at the same time to allow multiple person at the same time

to be in the same shared physical space to be able to see their own avatar, the avatar of the other persons be able to interact with physical props as well and so on yeah, that's what we are doing in VR right now and so this has been going on since Sundance now two years ago already and there's a lot of exciting development around that. Yeah, so I'm Cecilia Charbonnier. I'm also one of the co-founder of Artenim, so I work with Sylvain. We are both, I would say, the creator of that technology, in fact. And then I think you said already everything regarding what we are currently trying to do with VR, so yes, that's it. Cool, so yeah, I had a chance to see the real virtual reality two years ago And for me, that was the first time that I had had a chance to be able to have a full immersive experience with both my feet in my hands and an experience and to be able to walk around. And I know later, he's a VR researcher. He talks about the virtual body ownership illusion.

And I would say that that's the first time that I had an experience of having my full body and feeling like I was fully immersed. And so, yeah, maybe you could talk about that and being able to actually have all of your limbs in the experience and then what that psychologically does. for people? So first I would say that technically we were trying to because there are lots of ways to do motion capture of a body especially if you look at just yeah movies and so on where you have huge shoot with tons of markers on it and so on that gives you really accurate but a kind of very cumbersome setup. Our plan was to be able to allow people to quickly jump in this type of experiences so basically having a setup that's like super minimal i would say so that in like less than 30 seconds people can be equipped and start the experience so we ended up using tracking the feet the hand the headset and the backpack and based on that we're using inverse kinematic to be able to recreate the whole body animation so it's not perfect but it's already close enough

from the reality so that when you look at yourself in vr you have this feeling of ownership of that body and that's in our opinion one of the really key elements to immerse yourself in a vr experience the difference it makes of having a body compared to not having one in a vr experience is amazing if you for example take let's say something where you have height and maybe fair of height or something like that or you have to cross a bridge over something very high if you don't have a body and you don't see your feet, you don't really feel scared or something like that. But you do the exact same thing with your body and your feet and so on. And you look down and you see your feet across the void. And then you really start to feel that you're in there. And, yeah, the scaravite or whatever starts to really get very much more important, I would say. But body is one important part for having, you know, a strong feeling of immersion. but we also believe that social VR, so being able to share the experiment with several people at the same time,

give you even more of that feeling, you know, of presence and so on. And the last part as well, which were also very important and also guide our development from the beginning, is to be able also to touch things. I mean, having this sense of touch somehow. And for this, I mean, we are using like a natural concept in that sense, because you can just touch with your hands and they are also tracked very accurately. you can just grab your hand of your friends, high five or things like that. Or as you were mentioning, also we are not on that experience that we are presenting here in Sundance, but we could also integrate like physical props. So meaning that if you remember like three years ago, our experience in Sundance, you had a kind of stick in your hand, but then it was a torch in the virtual space and things like that. But it very effective because we are mixing physical and virtual elements which give you really that sense of presence and the sense of touch Yeah with this VRI experience there like four other people that were in there at the same time with backpacks and they also had both their hands and feet tracked And, you know, at the very beginning, there was a prompt for us to shake each other's hands, to give each other high fives.

And to me, that was such a visceral moment to have our full bodies tracked and be able to kind of like interface as I would to other people as well. And so you have this social dimension, but also throughout the course of this experience, you have a lot of motion-captured bodies, and they're dancing, and then I was dancing around quite a bit, and people, I think, were confused as to whether or not I was, like, a real person or if I was an animation. And so you have this ability to, I guess, cultivate this feeling of almost like being on a dance floor with other people and to create this virtual vibe, but to also just have this level of embodiment of interacting with other people. So I'm just curious to hear your design process of bringing people together and what you've been able to see so far. people have been trying to do this experience. So it's funny you mentioned shaking hands and high five in the beginning. That's, I think, really one point where people start really to be immersed when they have seen in front of them an avatar, which you know it's someone behind, but you don't really

know for sure. You see his hand reaching out to you, and then when you shake it and you get that physical contact as well that matches what you're doing and so on. that really connects you both to the environment in which you're in and to the other persons that are doing the experience with you. So the whole concept of that piece was created by a choreographer called Gilles Jobin. So basically the idea was really to put persons inside of a dance performance instead of being seated in the chair and looking at the stage in front of you. So really mix the spectators with the dancers somehow. And yeah, an interesting aspect of it is that in terms of rendering, both the users and the dancers that have been looked at before are kind of on the same level. So it can be from time to time difficult to say who is who, who is real, who is not and so on.

And I think that's kind of an interesting part of that piece. Yeah, and most of the people also, since it's a piece for five persons at the same time, they don't know each other at the beginning. So that's quite funny because everybody is mixing and so on. And then I think most of the people are asking at the end of the piece, who are you? Are you the girls that is black or are you the guys with the black shirts and so on? So that's funny the way the people are also feeling in that piece. I think it's an interesting one. It's also calling for meditation. A lot of people are asking that it's, like, calm because it's very bright. Most of the VR piece, if you notice, it's always, like, very dark or things like that. This one is very bright, I would say, in that sense. So you are transferred to somewhere else. As you were saying, same level of the dancer. But what is also nice in that piece is that we are playing with some effect of scale. So you have, like, dancers that are giant, then at the same size as you, and then the very tiny one as well. And I think that kind of effect brings a lot to the piece as well.

Yeah, I think the effect of seeing giant people, and then there's a sort of mechanism as well to kind of teleport around the space. And, you know, I had noticed the house that was, you know, up on top. And, you know, just sort of the, just moving around also, using virtual reality to kind of like locomote you through space, I think is a nice way to also kind of feel like this transporting feeling. And I'm curious to hear you talk a little bit more about dance as a form of embodied communication, because it feels like when I go to immersive theater pieces, like either Then She Fell or Sleep No More, instead of using a lot of dialogue, they're using the movement of the body to be able to communicate in that way. And I think that in this piece, you had quite a lot of different choreography and dancing around. I'm just curious to hear that process of trying to communicate different messages without any words, but just using the movement of bodies. That's not an easy question, but that's for sure. I mean, I think that those bodies are trying also to communicate a little bit. They are maybe inciting you to dance with them and so on.

So, I mean, people are reacting completely differently. It really depends the direction of the people. There are people, as you were saying, and I think you do. I mean, you did quite a lot of dancing as well in that piece. There are people that are more contemplating and so on. So everybody is reacting a little bit differently. So maybe that's a way the body are, you know, transmitting the, I don't know, information and things like that. But, I mean, I think the choreographer make a really nice job, try, you know, to mix. We capture five dancers and then they multiply themselves so you see them around you, everywhere around you and so on. But that's, as you were saying, it's another way of communication and to transmit something, maybe more emotions as well. Because body motion can be more emotive than just words sometimes. and that's maybe an interesting aspect of that piece as well. Yeah, for me, it was an opportunity to kind of play around in a way that I've never been able to play before in VR, to be able to kind of jump around, to kind of spin around. You know, it would have been interesting to kind of dance with other people, but the other people that were doing it with me, I guess,

were not as much into the dancing aspect. But a very surreal moment for me was I was dancing around and then when the other virtual avatars came in and started to dance with me and I was sort of dancing and mirroring them in a way. And also the shadows were really powerful to be able to actually see my body in the shadows and moving around. And I think that just having that level of embodiment and have my feet into the experience and be able to kick around and kind of jump and just mirror all the different movements. So for me, it was an opportunity to kind of experiment and play around with embodiment and really just invoking this virtual body ownership illusion. But, yeah, just to me, it was something that was probably one of the most immersive experiences that I've had in virtual reality just because it was so embodied and surreal, just being transported into these different places and to kind of see things that I'd never experienced before and at different scales and teleporting around. So, yeah, that's just kind of like some of my impressions. So I don't know if you've noticed how other people kind of react in terms of different things that they do. So, yeah, as Cecilia was saying, I mean, the kind of reaction of people,

I mean, you have the ones that have barely experienced VR before, or very few and yeah for them it's kind of a really how to say that new and very immersive experience that well this is in fact what you were expecting from vr since the beginning i mean vr for me should be like being able to be in another world and i mean with this kind of system since you have a high feeling of immersion that's the way to unlock some world that you will have you you would like to experiment yourself. So I think that just the beginning and I sure that VR will still expand improve a lot and so on and maybe I don know let see if people are transitioning from the physical to the virtual world only or if they are spending more time in the virtual world than the physical world. I'm not sure about that but I mean that's one step forward. Yeah and what were some of the biggest technical challenges that you had to overcome and solve in order to produce this experience? So there were

several ones I would say so the VR platform in itself I mean it's been under development since four years now compared to what you experienced two years ago so the I would say the major novelties and improvement were adding more people in the experience back then it was only two now it's five and it can be even more if we wanted so that's one of the technically challenging aspects because it's more complicated to track a lot of people in the same space. It requires also then more bandwidth to transmit all the information to the users, etc., etc. So, yeah, I mean, these are, I think, the most challenging part on the platform side. On the creation side, so all the dancers that you've seen, they've been motion captured. And this was also quite a relatively complex shooting. so yeah five dancers in a relatively small space with all the full body suits and so on

the props the walls etc etc which was kind of a yeah challenging shoot at least compared to the kind of things that we've done in the past and i think the yeah so gil the choreographer was really smart in planning this shooting and trying to have already a good idea of what this data somehow would translate in a VR experience while you're really surrounded by the dancers and so on. So yeah, he did a good job on that side. And for each of you, what do you each want to experience within VR? I think there are a lot of possibilities but I mean since we are really on the interaction and immersion side so I must say that most of the time I'm a little bit disappointed when I cannot interact with my buddies when I have just an experience that it's the headset and then moving around I mean with the head

and so on just 360 view in that sense so I'm expecting you know like content where I am the actors of the piece somehow where I can maybe interact to take decision in the piece that's why I'm looking for most of the time yeah I agree I mean I think what VR is for is really to be able to transport you to places that you will never be able to see to experience things that you will never do in your life that doesn't completely exist and in the way of doing that I think one aspect that I find really interesting with what we are doing is that the interaction with the world is completely natural you don't have like a controller or joystick to interact with the world you're in you're just using your hands to i don't know grab objects and your body in the end is the controller somehow so you're yeah you don't rely on any additional technological elements somehow which makes that

And it's for almost all the people that are trying this. It's very quickly natural that, okay, I'm forgetting the technology that's around me. I am just, okay, surrounded by a dancer in the desert and I'm there, basically. So that's, I think, the kind of experiences that I'm really looking for in VR is to, yeah, be able to travel somewhere else in a natural way and be able to experience with friends or other people those adventures. And what's next for this project and the work that you're doing? Should we expect to see at some point some of these experiences in a location-based entertainment where people can have these fully tracked experiences and be able to interact with each other? Or maybe you could talk a bit about where you're at now and where you plan on taking this work. So, well, this technology, in fact, we also have a commercial entity that is using that technology that we are developing at Artanim. So it's called Dreamscape Immersive.

I think most of the people have heard about it. So because they are still expecting the big launch that will arrive soon. So the idea would be to have location-based centers. So it's what we call multiplex VR. So we'll have, like, different pods or capsules, what we call it. I mean, that's like theater somehow. So you will pay a ticket not just to see a movie, but in that sense, you will experience a VR experience. So the idea would be exactly the same business model, I would say, as movie theater. But in that sense, we have the opportunity to try different VR experience. So that's where we are. And that's the goal of Dreamscape Immersive. And for this, in terms of, yeah, content that we would like to show, would be like something that is more related, you know, to storytelling, not really like focus on gaming things and so on, But really to design experience that is available to a large audience and in public venues. Yeah, so for this interesting little anecdote, I would say.

So when we were at Sundance two years ago now, that's kind of where we met for the first time our American partners now. We were able then to introduce us to all the Hollywood ecosystem, etc. And now that's kind of interesting to have these very good relations with movie studios and IP holders, because I think that they are all trying to figure out a way to use VR as a new way of telling stories and using their existing IP, but in a different medium somehow. and yeah with these partners and these connections we're I think now able to offer somehow a new medium for storytellers to tell stories in a very immersive way so I think that's quite exciting on that side great and and finally what do you think is kind of the ultimate potential of virtual reality and what it might be able to enable well i mean i think yeah the ultimate game

of virtual reality is to completely fool your brain and make you believe at 100 percent that you're actually living something else so that probably passes through being able to fool all your senses So now I mean I think we still far until we can get to what we could call perfect virtual reality which would be the place where you cannot make the difference between what a virtual world and what real world i think somehow that's one of the end goal on the technological aspect of vr and ultimately i think VR will be a very strong enabler to allow people to discover new places and enjoy stories in a very different way than what we are used to now on a 2D screen or something like that.

No, I think you summarized quite nicely. well still not there to ready player one but maybe one day we'll be there you mentioned story i'm just kind of as a follow-up question like how do you see the story of vri like how do you think of it do you think of it as a story or do you think of it as an experience because there's not any dialogue per se but there's things that are happening and so i'm just curious if you think of these kind of like immersive experiences if there's a story and how you design that narrative and how you sort of tell that? I mean, for me, VR, it should be an experience because, again, we want to touch all our senses. So that's an experience per se. Then, of course, you should have a story to guide your user somewhere and to transport them to the right way. So for me, that's both aspects that are important in that sense. In the case of VR Eye, the piece that we're showcasing here, I mean, there is some sort of element of the story, but it's more an experiential piece, I would say.

The only story element somehow that's in there is, I mean, there is some kind of continuity in what you're experiencing. When you are with the giants in the desert first, and then you're teleported to the house, you notice that the dancers were already existing somewhere else. So that's kind of an interesting aspect that, yeah, I mean, those places were coexisting and maybe you were at two places at the same time. That's kind of an interesting experience for that. Is there anything else that's left unsaid that you'd like to say? Well, I think we cover quite a lot. No, on my side, that's right. And you? So maybe it could be interesting to just get a bit more on how this piece was created and how it reached that. So if you remember two years ago in the piece that we showed, there was already two dancers at some point.

If you remember in the second scene where you were in a futuristic city on a kind of a sliding platform. So there was already two dancers that are Gilles and her dancer Susanna. So that was the first collaboration that we made together. so basically gil came to us at the beginning because he wanted to experience a bit with motion capture and dance for a film i would say so it started really not at all with vr in mind and so we made some first motion capture and get some avatars in motion and so on and we were developing the vr platform at the same time so at some point we tried oh let's put those avatar in VR and see what that means and for us it was like yeah really a very interesting moment where you have yeah virtual person that are the same size are you really into a one-to-one relationship and so we showed that to Gilles so he was able somehow to be able to see himself dancing in front of him and for him I think that was a really yeah interesting discovery and he

thought that it was so interesting and cool that he decided to create a whole piece out of it. So that's kind of how we decided to create all that and arrive to what you were able to try today. I see. So as a dancer, capturing their own dance and then being able to step outside of themselves and watch themselves dance for the first time in a volumetric way, I guess, was something that was totally unique and different from them. Because you can record somebody 2D, but it's a lot different than having a volumetric 3D immersive experience of yourself dancing recorded. It's like a time travel in some ways. Yeah, and since then, I think Gilles fell in love in VR, so I think he has many, many ideas that he will concretize after that project. And I think, yeah, for him, it's a new tool to explore new avenues in contemporary dance where you can play around with space and scales and things like that. So I think, yeah, he will have a nice opportunity to continue that work. Yeah, and just one more thought that came into mind is, in the experience, there's a the moment when the giants are looking down on you.

And the thing that I really wanted to happen at that moment was for a sense that the giants were looking around and that if I were to walk around, maybe they would follow me or be able to kind of react to me. So this dynamic interaction of something that is perhaps prerecorded, but to have some ability to kind of move things around in reaction. So for me to have an opportunity to sort of react to people and for their eyes to be able to watch me in a way that has their eyes moving around. because I think there's still an element of the Uncanny Valley that sort of falls into this without having their eyes move around. They have this kind of dead zombie look into their eyes. But to have those eyes just track around, I think, would make a huge difference in sort of the presence. And this is, in fact, what we are actually already working on. So to add the other, you know, finger tracking, so you have your fingers are still still in that sense. So we are also adding new features like finger tracking, emotions, So maybe eye tracking is also coming with the new VR headset and so on. So that's the nice addition that makes you even more believe and providing also more emotions as well.

Awesome. I just wanted to thank you both for joining me today. Thanks a lot for interviewing us. It was really interesting. Thank you very much. Thanks so much for listening to the Voices of VR podcast. And I just want to say that I'm really trying to live into the transformation economy, meaning I want to give away all this information for free, make it public accessible for not only you, but the entire community. And if you find value in that, then please do consider supporting me on Patreon. You know, I'm on this journey of trying to reach my 1,000 true fans. And if I have just $5 a month from that 1,000, I'll be in a much more solid and stable financial position. So really need your help to be able to get to that point. So if you enjoy what I'm doing here at the Voices of VR podcast, then please do consider supporting me at patreon.com slash voices of VR. Thanks for listening.