#1813: [2022] 『Cosmogony』— Sundance New Frontier の仮想宇宙船シアターにライブ配信されたダンス公演

The Voices of VR Podcast. Leading up to his latest piece that was showing at Venice Immersive, which is called Sunset Motel. So this piece that we're covering today is called Cosmogony, which was a live motion captured performance that was cinematically captured and then broadcast within a virtual theater that was happening at Sundance New Frontier in 2022. And so this was kind of like in their spaceship that had created a whole virtual theater during the pandemic. You go into the virtual theater and you see broadcast on the screen this live performance of Cosmogony, which is essentially kind of the next iteration of the different types of things you can do with choreography and dance when it is cinematically captured and broadcast to a virtual audience.

So we're covering all that and more on today's episode of the Voices of VR podcast. So this interview with Jill happened on Friday, January 21st, 2022, part of the virtual Sundance New Frontier. And the conversation happened virtually via Zoom. So with that, let's go ahead and dive right in. Yes, hello. My name is Gilles Jobin, and I'm a director and choreographer from Geneva, Switzerland. And we present here a piece called Cosmogony, which is a live dance performance being motion captured in Geneva and screened in the spaceship. Great. So maybe you could give a bit more context as to your background and your journey into virtual reality. Yes, well, I started, well, I'm a choreographer, so I started around 1995, so that's a long time ago Contemporary dance, mainly stage performances Maybe we used to be called maybe the new tendencies of choreography in the mid-90s, early 2000 And in 2016, I did a 3D movie called WAND

and at the same time I experimented with motion capture because I met with Artanim. They have a studio in Geneva, a motion capture studio and we did this piece that was a duo with motion capture and I was projecting on the stage piece and then one day they called us and said oh, we put this duo in the DIMO or immersive system and so we went with Susanna, my partner, we went to sit and we could see ourselves dancing real size and we had our own avatars, we could move around each other and I thought, wow, this is amazing. I was just blown away by the possibilities. And we decided to make a piece. So Artanin, they let us use that technology and we did this piece called VRI and that's how we came at Sundance for the first time in 2018. So the piece came out in 2017 at FNC in Montreal and the next one was Sundance. And since then, we've been touring a lot, really a lot, a lot. We had a lot of success with this piece. and then we went back to Sundance in 2020

and that was with an app called Dance Trail. So we continue with the motion capturing movement and having the possibilities of inserting inside the 3D space and playing with sizes, having giant dance work like you can see behind me or really small ones. I mean, all the possibilities that you have. And we start to be really interested also in doing live stuff. but I think in 2020 it was still a bit early for this so we went with this app and I remember over at Sundance when I got this call from China from a partner in China telling me, you know, cancel your trip everything is closed the city of Beijing is empty and I was like, oh oh, this is looking bad and then we came back to Geneva we were on stage rehearsing for a duo with live motion capture and projection on screens on stage and one month and a half before the opening, the COVID came, lockdown, and everything had canceled. It was like, you know, the disaster we all know. And I kind of took the decision to cancel any stage performances that we had

and to go full on digital. So what we did, we adapted all the technology we developed for the live show for performances such as Cosmogony. But, I mean, so many things have happened during this short time at the end of the day. So that was in, we came back from Sundance was in February 20. And then we did very quickly we did a piece for the Venice Festival, which was an immersive VR piece, a multi-user VR piece with the live dancers. We had three live dancers in Geneva, one in Bangalore and one in Melbourne. And we were all dancing together in the same space and we had a global audience. And that was a bold move because we developed the tech ourselves in-house. So that was a little bit risky, but we took the chance. And it was really during lockdown, we did a lot of experiments, you know, connection at a distance, and we managed to invite any motion capture system into a 3D space from anywhere in the world. So that was also like a breakthrough moment for us.

And we started to have our own motion capture camera in the studio, so that also totally changed. It's a different ballgame now because we can really develop for us and for the use that we have every day, learning how to use the motion capture. How do you move inside motion capture? How do you make it better and less data cleaning? I mean, all these things, you know, and really develop the tool that we need. So, yeah, that was the choice that we took when COVID came to say, well, let's, we have the tools. I had a team around me, and I said, well, let's go. You know, I said, let's try, and apparently it's working. yeah i've had a chance to see a series of these pieces over the years all the way actually back to 2016 was a piece that was like a full body track that i saw at sundance and then i think you came in in 2018 with that same type of technology stack which was uh to be able to kind of walk around with the you know computer and to to be in this space with other people and so you're immersed in this space with other people dancing but you're also with other

participants and then again and you know in the sundance 2020 was with the you know the last you know physical festival gathering we've had at sundance for a number of years with that ar application uh but then the following um you know later on at the venice film festival uh being embodied in that space which you had recreated this whole theater that to be able to then have this performance um and then kind of experimenting with this idea of live performance while you're in vr and in this year i think the difference is that you're in a virtual cinema and you're watching a 2d screen of those performances and what i found was interesting was that this question of how do you know it's live type of thing where when you're in vr you tried to have a live stream of what was happening and so but there was a delay and it was difficult to know exactly what the the dancers were doing and so in this latest piece of cosmogony you kind of in a few times you you're you're

switching back and forth between these people that are dancing live and you see them in this studio with the different grid on the ground and all the motion tracked gear so that they can be you know translated into these virtual body representations and then you kind of cut back and forth between those and because you're live streaming then it's more seamless there's no delay so you can see more directly what the dancers look like and how that's being translated which was actually kind of as a conceit that was used in dream as well to try to you know convey the audience that this is in real time this is happening but you know it's different than being embodied and watching the dance and so what i found striking this year was that there was things that you're able to do by us watching it on a td projection that was different than what I was able to experience when I was embodied in the world. And so maybe just before we start to dig into the other innovations and choreography that you're able to do in Cosmogony, maybe you can describe this, you know, challenge of trying to communicate the liveness and as a director,

what the benefit of breaking that fourth wall is to try to show that. And because it, you know, it kind of risks being this kind of tech demo thing, but at the same time, you don't want to have people think that this was all pre-recorded or whatnot. So, yeah, maybe you could just talk about that liveness component and your latest iteration that you do with Cosmogony. It's funny that you mentioned this question of the video feed in La Comédie Virtuelle, which is the piece in Venice, and you saw the first version, and now we found a solution for this problem. But this is a typical technical problem that has huge artistic implications. because for me, I thought, you know, I mean, let's just, for the people who can understand what we're talking about, you are in this VR space, which is a theater, and on the wall you have three TVs, and on each of these TVs there is a live feed of the studio where the dancers are, so you have the three in Geneva, one in Bangalore, and one in Melbourne, and it says the name of the city, and also there's the clock, so you can know what time it is because it's obviously a different time zone.

But, so for me, it was like, yeah, that's the solution, And then they say, yeah, but the problem is that we have a delay with the video streams, a 30-second delay. And I was like, yeah, this is like really problematic because people don't really understand why the avatars move before their images. It's just like not logical. Technologically, it's totally logical. But for some reason, we don't understand it like this. And there was too much distance between the two images. So you couldn't really believe. I mean, it created doubt in the head of the spectator. And they tried. They tried another system. It was working, but then everything was crashing. And so it's like, you have to say, oh, that's it. We have to deal with it. And then when we did the second version, our developer, Pierre-Igor, he found a solution. And he went deep and he dig deep to find a solution. So now we have a solution. We don't have delay anymore. And actually, it's thanks to this that we can do cosmogony the way we do it. Because that's how we can have... Because if you notice carefully, you will notice that you have the same problem.

the video feed, the image moves a little bit slightly after the avatar because the transmission of the data to the build is faster than the stream. But we found a way to move so you don't notice so much. We create enough movement and confusion so you don't notice so much. But it's like really a fraction of a second. But you see, this is typical. I mean, I thought that was obvious and simple for me. I never thought that would be an issue. And when I realized it was too late to change, I didn't have another solution, so I had to live with it. And then we found the tech solution. So every time you do something, you get blocked, and then another door opens. Or sometimes you're blocked, and then there's a new version that comes out like a miracle. And you can move on to what you really wanted to do. So, I mean, for me, the question of life is not only about performing life. is it's also about creating

because it's much more organic to be live and also can retouch. You know, I changed Cosmogon. If you've seen the first version, now it's a different version. We reworked, we changed some avatars, we have more motion capture camera, the mo-cap is better now. The dancers dance it better, they know it better. We reworked some scenes that we felt were a little bit less interesting. So it's like a live performance. When I do a choreography, I change it every show. I have to adapt it also to the stage, make some changes, and normally it may take me one year before I find the final cut, and that's the piece and I don't touch it anymore. So it's very practical. And also as a collaborative space. So we do also projects called Virtual Crossings, which are connecting clusters of digital artists and movers in motion capture and work together for one week or two weeks on research projects.

So it's kind of exploring those new spaces also as collaborative spaces. Yeah, I mean, this dancer, for instance, her name is Zilia. She's from Hong Kong. and I saw her on Instagram because she was having a Rococo suit, a mock-up suit. And I said, ah, well, you have one. I exchanged a little. And we invited her to come inside our theater and we did a little performance with her. And we never met for real. Now I'm waiting for her to come to Geneva probably next year for residency. But so there is this kind of new network also of live dancers using motion capture. There's another performance. I'm going to try to see tomorrow. Sugar from Valencia James at Sundance, which is also a live dance performance. So there's two live dance performances in the program this year, which is, I mean, from my point of view as a choreographer, is quite relevant. So, yeah, I think maybe something is happening. It's difficult to know exactly what.

Also, I've been, you know, you probably saw that now they did this, you know, in Fortnite they do concerts, and then they have this Serpentine Gallery now, so you can go to Fortnite and visit Serpentine Gallery, So it's like, what are those new spaces? I mean, what is this? You know, what is... And I think dance is very... Dance is meaningful, but it's not narrative. So, you know, narrative creates a lot of difficulties because you have to kind of make people understand what you... You know, so when it's meaningful, you have a bit more room to kind of, you know, move. And so you're more flexible and you can adapt. and I always try to find artistic solution to technical problems, as I say. So when it's too complicated, I say, look, wait, let's think, maybe we can find a way with an artistic twist. And then also, for the performers, I mean, it's a different energy, of course. Because it's a continuous, you know, it's not like a dancer in a movie.

You know, a dancer in a movie is like, you wait, wait, wait, wait, wait, do your thing for three minutes, then wait, wait, wait, wait. You don't have any continuity. In cosmogony, they dance full on from beginning to start. They're never out, very few seconds out of the frame. So that's what I found super interesting about those spaces, is that they have many different entrances. You can come in VR, you can come in AR, you can come with a 2D projection. Going 2D has a lot of advantages. One is that it makes it more accessible. That's for sure. But also you control the frame. So you become a filmmaker. And a filmmaker is, you know, you can really like... Because in VR, people do whatever they want. So you never know, you know, what you're going to do in the comedies of sales, what you're going to be looking at, you know. So there is a different kind of way to deal with the audience. But in these cases, like in the cinema, they have to see what we show.

They don't really have the choice. so it's yeah I mean that's what I like about those worlds they are so flexible and have so many entrances yeah well after seeing the progression of your pieces I have my own experience of what I saw was new and interesting and different but also before I start to share before I start to share some of that I reminded back to Dream because there that similar conceit where it on a 2D screen I not in the world but I started off looking at behind the scenes breaking the fourth wall showing me where they at They're in these mocap shoots. Okay, this is live. This is happening right now. It's always really difficult to know if it's really actually happening now. But I think, you know, by the Q&A, you start to have this opportunity to have live back and forth interaction where you can start to interrogate that. But, you know, setting that to a side, you know, the trying to prove that it's actually alive, it's, you know, I think very difficult problem. And I think you the audience takes a leap of faith, the trust that it's alive by these different conceits.

But in Dream, you have that you're breaking the fourth wall, but then you're in the story and you come back out. And then, you know, so it's just a way to kind of enter in this larger narrative or story that they're telling. And that's an adaptation of Shakespeare's, you know, a Shakespeare play. So but for you, I'm just curious how you describe what is happening within Cosmogony, what the story is, or if there were if it was more driven by technical things you wanted to do aesthetically, or if there were a deeper story you're trying to communicate with the type of techniques that we're using within Cosmogony. So how would you describe this piece and what's happening within it? well i mean it's i mean the way i work as a choreographer um i'm you know i'm quite experimental so i will i will create situation like for instance you know rule of a game for instance i would establish the rule of the game or say i put the grid on the floor and then people have to walk on the on the line of the grid and

stop at the corner and and then so i give them the speed and the quality of movement and then the organization and then like the kind of general motion but then they can do whatever they want on that grid because they have enough it's like you know when you play football i mean soccer for you you know it's very simple rules but that's it to give the ball give the rules and that's football any good footballers bad footballers going to be football anyway because they play you know they don't throw the ball with the hand so i i have this kind of way to construct things in terms of the choreographic material and in terms of the content I think what I do is suggestive so I suggest ideas and you make your own mind about what's the meaning, you know, it's like you think for us, you're like, oh he's doing this oh, that, you know, it kind of creates a sensation of a kind of inner dialogue with yourself and with the piece and I try people to go along, so for me it's always important that it's very there is a nice flow that you really

go from one thing to another that you don't have like this kind of you don't have any doubt it's kind of always a bit trippy a bit psychedelic as well and then in terms of meaning it's difficult because it always appears almost at the end of the piece I never know why I put like a flat earth in cosmogony for instance you know it's like i don't know but it it seems important to me like it's a sign it's like to me it's a sign of ignorance you know is is i don't know so i think i don't know i think i have to put it you know it's like i don't think so much and and then it's like this kind of logic and um and then you then there's the earth and then there's all this trash coming you know out of the earth and and it just and and there's this idea of of i think maybe the main idea is this journey of this camera that goes through this city and meets, you know, new section of movement and go through those movements in different places.

And it's just like a, and then you realize that, you know, it's a small earth and then it's a flat earth and then it's like, you know, so I don't know. And the music also, you know, this is a little bit unusual for me to have such a rock soundtrack. I have more electronic and general stuff. I personally like rock music, but in this case, as we were rehearsing, a friend of mine who's from this band, Tarpon, we were playing their record and we liked it and we thought, let's keep it. And I said, could you do the soundtrack? We used some of the tracks from the record and you do the in-between. And that's how it came out. So this is always this kind of very kind of, you know, when I was starting, when I was a young choreographer, I was always like finding something in the studio, like whatever, like something like this, for instance. And I would put it in my piece and it would stay, it was like an object. And I was like, what is this? Like, yeah, I just found it in the studio. But then I make some, you know, that's what I like about contemporary dance.

You can change it. Some choreographers are famous for changing everything the last day or, you know, in the afternoon of the premiere, they reverse everything. So it's a very open language. And the choreographic language is quite powerful, actually. And even the contrary to what people think, you don't need to understand choreography to understand the language because it's meaningful. Again, you know, it's like if you know what you're doing, if the dancers are convinced. And maybe one last thing also that is quite specific in dance, I find, is that the dancers, they are themselves. They're not a character as such. Well, of course, there is some dance theater type of dance, but like in what I do and most of the abstract dance, it's like Gilles dancing or Susanna dancing, and she becomes the situation and the movement, and she grows emotion from herself, from inside herself as a performer, but not as the character of, you know.

so that's also a very open kind of way to be in movement and in space and in a 3D space I always say dancer is the 3D specialist because that's what we do because we understand space we understand space being in space but also the space between the people the space between the things space that creates tension between people or things or objects so yeah so that's why I think dance always has been in text. If you look, you know, TikTok is a dance, like it's mostly dance. Yeah, it's interesting to hear you speak about both your process, but also, you know, the language and how you think about it and talk about these pieces. And before I get into some of my own direct experiences, I want to unpack a few things that you mentioned there in terms of your process. So you have like three dancers that are in this piece, and they're kind of multiplied out into this virtual world. And so you give this sense of having the dancers, these virtual spaces you're taking us through of being populated by lots of people, even though it's just three people being replicated and kind of rotated in different ways.

And so I'm curious about your process of, you know, as you are the choreographer, are you specifically instructing each of the dancers with the specific movements that they're doing? or do you set forth like some general rules or like i'm just curious how much is generative from the dancers and how much is choreographed from you um and if part of that choreography is the architecture of these spaces but also these different embodiments and these different rules are coming up with and so talk a bit about this split between how much of what we're seeing is coming from you as a choreographer and how much is coming from the dancers who are maybe improv-ing to a certain degree? Well, I wouldn't call it improvisation. Because, again, they know exactly what they have to do. And in Cosmogony, the space is not so big. It's like, I think it's like seven meters by four. And there are three dancers. So they have to be, because of the frame, they have to be in very precise position at some moment.

So they have a lot of marks on the floor. So they are very, they have to be really precise. so they cannot improvise. They always do the same. If you look at the piece twice, you're going to see exactly the same. There's no big differences. In terms of the creation of the movement, or more than movement, the sequences, then it's already a long time for me that I don't choreograph myself anymore. Like I do a little phrase behind me, and they have to learn that it's not necessary anymore. I don't do ensemble, you know, like people dancing together. so you know away from this tradition of more classical choreography really and actually contemporary dancers they are very flexible we call them interpreters really because they make an interpretation of instructions so they we expect them to produce some of the material because that's how it's going to be the most personal to them because they understand the condition you know when Susanna

she does the giant in the garden, I don't tell her what to do. I kind of tell her something. She does and I say, ah, I'm doing a little bit like this, a little bit like that. Maybe they could react a little bit more, a little bit less. Maybe there could be one up, one down. So I kind of place the context really. And the more we move on with the creation, the more the context gets clear and precise. And actually, the work we did this last week before Sundance was to clean all those moments of doubts where they don't know exactly, they're not sure what I'm supposed to do exactly, what is the energy so we really re-clarify so it's not about like choreography in the sense of writing movement it's about putting people in movement it's like directing them into shaping them and and putting them in the right direction. And I work a little bit the same with the tech and with the 3D artists and with everybody.

You know, I kind of guide them because I don't want to lose their creativity, you know. I don't want to love the good ideas that they have. So I don't want to go like this. You know, and they need to give their own... I mean, they need to be involved, you know, and everybody needs to be involved in this kind of work. So you have to give room, you know, for their, their, you know, like, for instance, the set of the way we did the set of Cosmogen is like a collage of, of, so it's like, I say, well, we don't have much time. We have to do it quick. I want something really big. So don't start to go like super detailed, three stories or 3D artists. So I want something like rough, like a collage, like let's try to make a 3D collage of postcards. And so we took some planes and some stuff from Google Earth and do like a mashup of elements. And again, you know, signs of different cities that you kind of recognize. You think you're in China. You think you're in Buenos Aires. You don't really know where you are.

So it's a kind of, yeah, it's the same for movement. So it's really kind of like every time giving more definition. It's like, yeah, it's like sport. It's like, you know, even like American football. You need to rehearse a lot. You need to have a lot of strategies. You need to have, but then in the moment, you wouldn't call it improvisation. Well, American football is quite different, but soccer is like, soccer is not improvised. You cannot say it's reactive. You have to be reactive to what's going on, but you don't improvise, you know? So that's, for me, the difference because what we tend to understand by improvise, like, you know, whatever, do whatever you want. No, improvising is not. and then improvising is a technique of composition by the way many choreographers use it to compose so i think i do a mix a bit of both okay and one thing that i found interesting in the q a after the cosmogony was hearing a little bit from the dancers about how you know they're

they're dancing with each other in the grid relative to each other but they also have the screens that they're able to look at their virtual bodies and their representations and how that when they embody different avatar embodiments that they're actually changing how they move uh in some ways and which i found was interesting and so maybe you could talk about that uh process of you know having the dancers in a space and they don't they're not immersed within vr so but they have screens that they're able to see and they're able to orient themselves about what their representation is and so maybe you talk about that that balance between us dancers what that means for them to be able to be immersed within this virtual space, but also in the physical space relative to each other, and that they can't run into each other, but they're still at the same time seeing the mirror of themselves and that virtual representation is changing how they move, which I thought was interesting. Yeah, I think it's a strange sensation. I did some, but not so much as they do,

so they're much more expert than I am. But it's a bit like being the puppeteer of your own body, you know, because this avatar is you as well, you know. It has an existence, and you see it more clearly when they walk like normal, and you totally recognize that is the way they walk normal, and you catch, obviously, every movement. So, sorry, I lost my thought. I was asked, the question was around the mirroring of how you're seeing themselves yeah they have that's how they place themselves in space so, but it's also how they see the reaction of their avatars at some point they have some really fat big ones so they can move only in a certain way because otherwise it doesn't go well they have like longer arms so they have to kind of get into the motion of that character

and so they have connection between themselves as well but they are like because they have like they have two brains working at the same time because they have to think what's going on on the screen which is what people see and then what is going on between themselves so they don't hit each other because they have some time to look for some cues or whatever you go first I go after so they have to look at each other, look at the screen but then they still have to do it. We have made a little movie behind the scenes called Beyond Cosmogony. We're going to launch it soon. And it's very nice to see how much they interact and how they look and how they hold the different angles with their eyes, how they have to kind of... And, you know, dancers are very good for this because, you know, they can dance in a football stadium alone or they can stand on a one square meter space.

You say, like, well, okay, let's do something. That's the stage. It's very small. That's the stage. It's very big. They are very adaptable. You know, like we were talking with Susana the other day, and like in the last two years, she danced outdoor in the swimming pool, in VR, you know. So it's like they're very adaptable, and that's characteristic also. So yes, there is really a kind of score, which is the score of their eyes, what they're looking at to place themselves in the right space. Because what's happening in Cosmogon, because of this camera that goes through, so the trick is that we have the motion capture space, so we multiply it further away and we change the direction. So that means you have the same dancer you can see twice, but he goes in one direction and on the next square he goes in another direction because he would turn the square. So when we discover the little trick, I'm using it a lot in everything I do because that and I can change avatars because in the next square I can have the same dancers but with different avatars and different sizes also so by changing the size it changes the

avatar even if they do the same movement but just because then have a slightly different angle that's the symmetry thing and and it's totally yes totally like particle physics so there is this this entanglement between their real body and their avatar body. And I think now the experience for them is even going further because now they're starting to experience this at a distance. So, Suzanne a little bit more, the other, but it's like one year, one year and a half. No, I mean, one year we're doing this. So, the experience of performing for remote bodies with your remote body on the other side of the world is a new experience. So, I don't think they still yet have understood exactly what's going on. But I can see that something is happening. It's a very strange connection. And I call it teleportation. I'm like, yeah, that's it. We've done it. We can teleport. Because what's the body anyway? Who cares?

I mean, I'm here, okay, but I'm wherever you are. And if I'm in 3D, if I move, so then I'm teleported. And for Sennan, we were supposed to screen it at the Egyptian in Park City. So that was the whole challenge of this project this year. And Shari Freilott, she was very into this idea of doing a simultaneous presentation in the real world, in the Egyptian, and simultaneously in the spaceship. But it's not going to happen, so it broke my heart. because, you know, for me, the Egyptians, like, ah, I'm, like, dreaming, you know, when I go, I'm, like, ah, and then you hit it, and it's gone. But, you see, that was the demonstration. So this is powerful also, you know. It's like we can do three, four, five, ten cinemas at the same time. It's not a problem. We just multiply the system. It's not very complicated, really. Yeah, that was the things that I noticed in particular, a lot of the technical evolutions of your process over the years

is that you have, you know, because it's in 2D, you have the freedom to be able to move through the space in a way that may make someone sick if you were to do something similar in virtual reality. So you're able to have that cinematic feel. But with that, give us a sense of a larger virtual space that you're exploring, but also play with scales in a different way. And then going all the way back to VRI, you already had dancers that were a larger scale. So you have this kind of fractally nested, you know, space that in one level than in the much larger level and you're doing similar things here which i think is interesting uh but you're having the dancers that are giants and then maybe a medium size and then small so maybe three different sizes as you are are zooming through this city but i felt that being able to copy the the dancers and rotate them was actually interesting because what it did was that you know normally if you get things that are copied because there's no delay or anything your brain just automatically thinks that that is the same person because it's just

you know the this the movements are so distinct that it's very difficult to get that level of precision when it comes to trying to choreograph a piece i mean even when you see a group dancing they are in sync but they're not so perfect as to make you of course not yeah this is because yeah because that's they don't know because instead of recording one dance and multiply 20 you have to record 20 times the same dancer doing the same phrase and then you can put the 20 animations together and then they will be together but not together but if you multiply then they become clones like in a millisecond so it's impossible but when you change the direction then it doesn't matter anymore because you don't see the so you see if you look at so I think I leave it, you know, I don't try to hide it we don't try to hide it There's one moment in what we call the market when you can still see some big giants and you totally see that they are moving exactly the same than the dance that you have in the market.

But it's part of the magic of this world, you know? And for some reason, the audience accepts it as a fact and they don't think it's strange. So that's when I think, okay, if they accept it, then that means that it's feasible, you know? It has its own logic. So then, okay, you can accept this logic of this world, which is a bit okay. And then the liveness, the fact that in real time allows us also maybe, you know, some glitches and some stuff that you wouldn't accept if it was post-produced. So I think there's many different elements of, you know, I saw yesterday 32 seconds. Wow, this blew my mind. I was like, wow. and you know again it's like you know such an experience of like it's such a spatial experience so I think that's and it's so clear and the subject what he's talking about is like so you totally know what's the subject from beginning

to start and you just go wow you can let yourself go you know I thought that was beautiful and that's what I try to do in my thing you know so people can relax they don't have a doubt and like oh you know like this video problem and like a video, it really annoys me, you know, because I don't like doubt, I don't like escape rooms, I don't like, like you have to push buttons to do things, you know, I want things to kind of just unfold like it's, you know, so that's all, in VRI was the same, in VRI the main problem we had was how to move the people, like the real people, the audience, the five audience members in the same, because as you know, it's exactly the same mock-up space. VRI, the contrary that we do with cosmogonics, is what they do in Fortnite. The audience is live, but the concert is pre-recorded. So VRI was pre-recorded, but the audience was live. So because they were live, they felt that it was in real time, so they had a real-time experience. To them, it was in real time. And because the giants were not really looking at them, not paying real attention, they noticed them, but they didn't care.

So they said, okay, so we are visitors, like invisible visitors. Then how do I move this square of mo-cap, this motion capture? square into another place. We thought, okay, so if we close them with walls around them, then they are somewhere else. And when we take out the walls, then they're also somewhere else. So we teleported them. We needed to teleport them from one place to another, but we couldn't move them because if we moved them, they would go like, wow, they will feel sick. Or if you put them in a telepharic or something, they will feel sick. So we thought, how can we do that? That was a really difficult problem. And finally, we found a theatrical solution. And I thought, if I was in a theater, what would I do? I would change the set. Because a change of set is a teleportation device, by the way. You teleport the actors in a different space by changing the set. So I said, yeah, but I don't have technicians. And I said, yeah, but I have giants, so they can carry the walls. They can take care of closing the walls around the people

and then opening up again. So, you know, these are very typical theatrical tricks. And I think theater, I mean, live performance, performing arts are really comfortable in those spaces because it's in real time. Real time is what we do. Yeah, I felt that it worked really well to be able to clone the dancers and rotate them 90 degrees or 180 degrees or 270 degrees. and offset offset them in space and to fill out a space so that you get that similar feeling that i was talking about where you have like it feels like a larger group choreography but you know there's only three dancers and then so there's like in the beginning i was you know i guess thinking about not really thinking about it too much and i was just like thinking oh that's interesting this is maybe got some pre-recorded stuff or something but then realizing later that oh wow now this is actually happening all live that these are just three people but they're being replicated in a way to give me more of a sense of that it's more people that are there so i thought

as a technique that was really effective because it was like i said it was just enough of like when you rotate it my brain is not automatically thinking that's a clone i can sort of surrender and immerse myself into the the spatial dance that's happening across this top about uh this virtual space um but um the other thing i i just wanted to say is that like playing with scales is another thing that I think is a theme that you do a little bit more interesting stuff here in this piece that I've seen before. But also there was a scene near the end where the dancers were in space. And what was interesting was that their bodies were kind of like limp in a way where they were kind of falling through space, but they were still dancing. And so it was almost like they were in a virtual context where you're trying to give this illusion that they're falling through space, but yet they're in you know in their actual bodies or or you know still they're not in zero gravity environments and so try to recreate this kind of feel of them their bodies being spun around and

also having part of particle effects on their bodies on their virtual bodies so that when they move you kind of see different traces of their of their movement so i thought that was as an effect something that was both new and interesting and different but also um started to explore the dance in new ways. And the other part that I would say is that when the dancers were being mirrored, so you have a virtual embodiment, but it's mirrored underneath, and so there is another layer of symmetry when it came to the dancing. So that's another one. Yeah, there's another, that other symmetry where they could start to play with the shapes of their virtual bodies in a way that's on multiple dimensions as well. So not only kind of recreating this, you know, feeling of them flying through space and then particle effects, but also ways in which that they're being mirrored more in the vertical space and up and down and horizontal and the shapes that they create with the sort of their body being replicated in that way.

So those are some other things that I just thought were kind of interesting and new technical innovations that were starting to maybe change the different types of choreography or how you think about choreography when you think about this as their bodies being spaced out and rotated, but also replicated in a vertical plane? Well, the thing is, I mean, that's the beauty of the virtual process, is you can add motion that don't exist. So in this case, you have the three dancers, and they have a motion. They turn around each other in a kind of creative motion. Camillo did this motion. and the camera, we have different positions of camera that follow either their head or their torso or their full body image. So when they turn, the background turns. So that's why it makes it so animated because every time they move, everything is moving. So you have, and because we change cameras, then you don't really have a reference, but they are really falling. So they are turning around themselves and falling really.

so that's also something that i found crazy with these unities and these 3ds was like because you create like a real size things like you know dancer is really 25 or 30 meters high and the the island is really you know like i don't know 40 or 100 kilometer wide or and and when they fall down from the sky they come from i don't know how many thousands of you know hundreds of kilometers and there's a lot of things that because they really go and maybe you notice at some point they look there is like a comet passing and it's these guys, the three guys they keep on going they go through the earth and then they continue so what I think is so great and to me it's only the beginning because I I don't like to do things that don't work and I don't like to do things that are too like you know, tech for tech or too techy or too like if it doesn't have like

it's not useful artistically, I'm like so I'm like, you know, for example eye or faces animation, we never really got into it now we added great novelty they blink the eyes once in a while that is really, really new you know, or the hands, all this we kind of like say, okay, it's not really important we can arrange we can go around we can find ways that it will come you know but and in terms of movement is the same because I know there's many ways that we could affect the movement we could start I would like to start to do to work with some AI stuff and see how you know we could teach an AI to dance or like you know how what will be the intervention and maybe not some realistic results also, you know. So we're doing this. We're also inviting some other artists in our studio. There was this company from Spain, and they were working on motion capture and sound. And so there is many different roads to explore still,

and especially in terms of movement. But, you know, it's time-consuming. I have a small team, so I always have to think before I ask them to do something, if they will manage, you know. So we have to kind of find the right balance in terms of resources, in what is our interest, where do we invest time into this. Sometimes they tell me, I ask something, and they go like, oh, okay, I can answer you, but I need three days just to give you an answer. It's like when you go with your car to the store, and they go like, oh, you're going to need two days just to open and to see how much it's going to cost. Well, okay, forget it. You can buy your new car. so you know it is kind of sometimes it's difficult I think for us to but we're getting every time better one big improvement for us is that we the dancers now they have more access to the tools they have spent every time more time playing with the system and they wait less so that was something that was not so easy to

implement for instance to say geeks stop working that's it now give it to the dancer they need you know you cannot they cannot have only 10 minutes with your system they need to spend time with it so so it's as a producer i have to find the right balance also between the text the movement and the possibilities and the dancers they need to learn also to kind of get into the new this new world it's super interesting and fun yeah and i guess as we start to wrap up here, I'm curious what you think the ultimate potential of virtual reality might be and what it might be able to enable. I don't know. I think we're in a strange moment because now, I mean, we went quite far in VR. We can do a lot of very kind of next level thing and now we have to downgrade everything because if you don't do something for Quest 2

or Quest, whatever, integrated, then that's yet. Nobody will use, you know, cable VR anymore. So we have to kind of downgrade. So I'm like, hmm, okay, so now. So there will be a downgrade, but then you will be upgrading again. So it's kind of a strange moment. And then there's all this ambience about this metaverse and all this business around this, which is. and I like better the bio-digital continuum that they talk about in Sundance I find it more real and more actual I feel that I living this reality I not sure about the metaverse and all this And you know licenses you know we need open source we need more research Meta is installing themselves in Zurich, like in Switzerland. They're hiring everybody, you know. So that's the other one. They come, big companies, and then they suck all of the good people.

But I'm independent. So, yeah. I don't know. I mean, I believe in independence. I think we're in a world of niche. I think that's what's going to happen. I think these spaces that we're creating will be every time easier to create. And I can totally imagine that, you know, some fan of doom metal in Mongolia or somewhere in Kazakhstan is quite difficult to get to see a concert. But if they can access virtually, that's a whole different ballgame. So these kind of hybrid events as Sundance planned, I think that's the way. And also this remote collaboration and maybe traveling less, you know, traveling less, thinking more before you travel, do more work without traveling. I think we all educate it ourselves. I mean, you know, even Zoom, even this kind of communication tool, even for me, was not something I was using every day. But now it's, like, common.

So I think we are all better educated in the digital somehow, and worse on the other side. I don't know. I mean, yesterday I had a nice experience, but sometimes I'm not so... It was the opening of the festival. in VR I mean in virtual and sometimes I feel like really like awkward you know so I don't know he's but I see my son you know he's 16 I have to one is 16 he's very comfortable with making friends that he never met in you know in video games and for me I'm like who is that he's a pedophile or something you know they have it's just it's just changing so it's I don't know but I think we cannot say really what's going to happen. I mean, that's the thing. I decided to do things. I do things, and I don't care if it works or not, or irrelevant or not, because I don't know. It's too difficult. I try. We have this idea of touring without traveling,

doing performances from our studio in Geneva and show live show across the world without traveling anymore for a while and see what happens. But it's fun and sad sometimes. because so now what we do, we had this because we also project Cosmogony as a video mapping, we did what in Bucharest and that was crazy, so we did it only again from Geneva so now what we do, we eat the food the local food, so like that was in Bulgaria so we had some Bulgarian food so that's how we try to entertain ourselves because we don't travel anymore but yeah, so we're starting in some months, we're starting in three days So it's going to be nice. We have four performances and looking forward. Yeah, I just want to reflect on a thing around the Quest versus the PC, because as I think back about even just the lineage of the projects you've been working on all the way back to 2016,

of having my first experience of having full body tracking in VR at that experience at Sundance in 2016. and then and the vibe trackers came out in 2017 at ces where i was there where they launched it and then the following year at sundance there was a lot of different folks that were using those trackers but i think you were still using like motion capture and so have that full body event but at sundance this year there's a documentary called we met in virtual reality by joe hunting shows a lot of the full embodiment with people using those vibe trackers and so there's things that are kind of prototyped out in these festivals, like at Sundance, you know, a couple of years before it hits the consumer market. And so even though it still takes these PCs and all this technology that maybe with cloud streaming and other ways that, you know, the level of graphical fidelity that you're still achieving with this PC-based experiences that maybe we'll get that same point in the mobile market. And there'll be other solutions that are able to have maybe external cameras when you're playing the Quest or something that allows you

to have the same type of full body tracking and have the similar types of dance performances that we're seeing, but it's using consumer technology rather than this, you know, super high-end motion capture. So it seems like an actual – Super high-end is very relative because now you have suits that cost like $2,500, you know, which is the price of a PC computer. So, you know, we're not talking about $50,000. And I saw the other day this dancer from England, Clemence de Begge, She does some connection at a distance, dancing with another girl wearing, I think they have a perception neuron suit, and they have one half of the body is one dance, and the other half is the other body. It's fun. I mean, it's fun, and it's quite accessible. You don't need to have a massive studio to do this. You just need a good internet connection, a bit of gear. There's all those actors also I found quite impressive, what they do, how they invent the way to change avatars and costume.

And I saw one and I went to a room I was not supposed to go and I could see him, how he was changing. I went backstage basically. There was a glitch somewhere and I could see the actor getting ready for his next thing. It was so funny. So I thought that's a good idea actually. And yeah, and I saw some people, they do hip-hop battles on VRChat with the Vive trackers. And quite interestingly, I noticed that they learn how to use the trackers because the trackers are not so precise. So they dance in such a way that it works. So they kind of adapted their dance to make it work, which is exactly what we do, basically, because I always say you need to learn. You know, every mock-up system has its learning curve. You have to kind of understand. And so I think this is that's the thing that's going to change for sure. And I see it every time more dancers being connected, finding the money. There's also this project where people can apply

and they get a six-month mentoring and a bit of money and a suit and they get to keep the suit. And this is an idea I had actually. And I think that's a super idea because like a dancer, he does a six-month mentorship at a distance with the suit and then he keeps the suit. So he will continue to do stuff, you know. So I think it's going to grow. It's going to grow. And also all the cheap motion capture, DIY motion capture, also I think this is going to happen with iPhones. Even what Valenza Jam with volumetric performance also is super interesting. It's like the lowest possible, cheapest motion capture system that you can do because they have this kit for $200. So it's not great, but they know it's not great, and they don't care because they think that's the space they want to be in because they know it's going to get better. They know it's... And I think that's so right. So I think that there will be a lot of different... You know, we are like on the other side.

We have like $150,000 of equipment in our studio in Geneva and it's like next level equipment by Qualysys. It's like equivalent to Vicon. It's a Swedish company. They are very good and very supportive. you know it's just like a different totally different but at the end of the day we met and we talk and and we we are in discussion and and something is happening in movement i'm i'm i'm sure i mean on my level i'm i'm seeing it and i see also you know people know me also in dance of course they follow what i do so i think oh maybe there's something they you know we break the ice. We create an ecosystem of movers and mock-appers that are not only for video games, not only for commercial stuff, not only for publicity, but really research in movement with the mock-up system. Yeah, it makes me want to do a whole series of different podcasts, of conversations I've recorded but haven't aired yet, that really explores this kind of evolution of these different dance elements. Because

in the film We Met in Virtual Reality, it actually features a number of the main protagonists, are doing different types of dance within the VR chat worlds. And so Joe Hunting was following the International Dance Association, which is actually doing a lot more, but isn't as featured as much in this specific documentary. But that full body tracking and full embodiment is something that is moving into the consumer space after seeing it in the artists like yourself and other technologies, the super high-end technologies, that eventually will, the type of stuff that you're doing now with the super high-end will be available to the consumer within the next five or ten years and maybe even sooner. so we'll see how it continues to progress and the filters as well like something I don't do personally but I see a lot of people doing things with filters and this is something also I think you can do a lot of stuff with this so you know and the integration I mean I come from the world before it was digital so I can totally see where we come from and in the last 15 years

it's been just incredible and it's the digitalization for us it was already to be able to record in video that was something and to see you know before the video camera then certain choreographers couldn't see themselves they could only film themselves send the film to develop very expensive and see and now every choreographer they film they see back and forth back and forth back and forth all the time and now even with the digital you can cut and paste which you cannot do in video so now you can cut and paste you can go backwards you can really tweak your images by yourself with your phone now. So this is such an amazing evolution and the quality and now I have this new phone and it's just like, I cannot believe the images I'm making with this machine. So this, and I could make a Hollywood movie with my phone, you know. So it's, that's what I think is fantastic to see. And the youngsters also are super interesting because they're very different than my generation. They learn very fast. they know how to use

tutorials and I mean they learn they don't know, they learn they go, and then three days after oh yeah, yeah, I got it, I mean more or less and then three months later they're like experts and talking with other experts so it's this I found like super interesting this capacity and you know, those old algorithm and I was talking with this guy yesterday also doing AI and movement and it's like, wow, you know, it's, yeah, and the particle physics and the new telescope, you know, this is crazy. Yeah, it's a good point, the fact that there's people who are watching themselves and be able to document their dances and share them with each other, and I was part of the hula hooping community, the hoop dance community for a while, and there's a lot of innovations that are happening within the form of hoop dance, but also the dance that now the TikTok has gone fully mainstream with the youth of this next generation.

And so there's a documentary called Tick-Tock Boom. There's a film called – there's a play called The Tick-Tick Boom about Jonathan Harsin that was translated by Lin-Manuel. Yeah, that was basically a film, but then there's a play on that called Tick-Tock Boom, the documentary that's premiering that's showing the evolution of tiktok as a medium so i'm very curious to to catch that because i think it's going to be a lot of similar themes of the this process of people who are documenting their embodiment their movements and learning from other people how to move and that's a big part of the new language that i think is being created is being able to communicate with your body but if you look at the history of tech you will always find dancers or people coming from the stage melias he was a magician and he's the one who invented cinema really. I mean, he didn't invent the tech. He was not the Frères Lumière. They were the geeks and the engineers. But he was the artist to say, hey, there's something there. I can

adapt, you know. And it's not... Every time the genie technology will see some movement, they use dancing. Like, it's... People don't really realize, but it's always dancing. Because dancing is very abstract. It's meaningful, but abstract. And abstract and concrete because you can do something very abstract, but it's very concrete because I'm a body first, then I'm a male body, and then with a female body or another male body or two female bodies or two, you know, the combinations are meaningful, but they can stay abstract. So I think that's why it's such a powerful canvas for tech. You know, Merce Cunningham was at the beginning of the video. He was very important in the video art, for instance, even though he's famous as a choreographer. We didn't give him as much space as we did to his partner, John Cage, because music is more important than choreography. But he was as important for the art in general, not just for choreography,

as John Cage was for music, as we saw in 32 Sound again, actually. So, yeah. Yeah, well, is there any other final thoughts? Anything that's less than said that you'd like to say to the broader immersive community? no I think we I think we we talk a lot and I think we we are in the right direction I see a lot of people working in the right direction and and experimenting being very experimental and very open it reminds me when I was when I was young when I was a young choreographer that we were like old friends and colleagues and we would exchange a lot of tips and tricks and ideas and in a very generous way and I know we became a little bit more famous and older, then we became more selfish and isolated. But I feel at this moment, people are eager to share and to experiment. So that's what I find very exciting and positive in these not-so-positive times.

So, yeah, it's a nice community, really. Awesome. Well, Jill, thanks so much for joining me today to be able to unpack a little bit more of both the process and the techniques and the experience of, or your own journey of creating Cosmogony and the next iteration of this continued over the last number of years, experimentation in this and kind of continuing to push forward new innovations when it comes to all this stuff. So yeah, thanks for being able to join me today and be able to unpack it all. It's a pleasure. Thank you so much for all these interviews that you did. They're so interesting. Thanks so much for listening to the Voices of VR podcast. And I just want to say that I'm really trying to live into the transformation economy, meaning I want to give away all this information for free, make it publicly accessible for not only you but the entire community. And if you find value in that, then please do consider supporting me on Patreon. You know, I'm on this journey of trying to reach my 1,000 true fans. And if I have just $5 a month from that 1,000, I'll be in a much more solid and stable financial position.

So really need your help to be able to get to that point. So if you enjoy what I'm doing here at the Voices of VR podcast, then please do consider supporting me at patreon.com slash Voices of VR. Thanks for listening.