
Pika Enact:再生中のAI動画を「演出」できるのか
Pika Enact: Can You Direct AI Video While It Plays?

Pika Enact:再生中のAI動画を「演出」できるのか
The Neuron: AI Explained
要約
The NeuronのホストがPika CEOのDemi Guo氏、クリエイティブリーダーのMatan Cohen-Grumi氏とLindsay Brillson氏を迎え、インタラクティブなAI動画生成「Pika Enact」を議論した回。プロンプトを入力して完成クリップを待つ従来方式と異なり、シーン再生中にキャラクターを動かし、俳優に話しかけ、カメラも変更できる。リアルタイム生成を支える独自モデル「H3 Flex」、合成データ、品質と速度のトレードオフ、AI作品の権利、3年後の展望まで語られた。
この回のポイント
- ●Enactは、シーン再生中にキャラクターをドラッグしたり、声で演技指示を出したり、カメラを調整できる対話型の動画生成で、現場での演出に近い体験を目指している。
- ●基盤は同社のモデルH3 Flex。Demi氏によれば、既存のリアルタイムモデルは約5秒のチャンク単位で遅延があるが、Enactは遅延のないストリーミング型を新たに訓練したという。
- ●学習データは3Dで描画した映像を、低速だが高品質な動画モデルでフォトリアルに変換する合成データのエンジンで作っている。リアルタイム版は非リアルタイム版より画質がやや劣ると認めている。
- ●エージェント層が俳優(キャラクター)に自己認識と記憶を与え、「Sarahはもっと感情的に」といった指示を覚えさせて、誰に向けた指示かも区別する。
- ●演技を固めてからカメラを別工程で試せるため、実際の撮影のように同時進行で全てを制御する必要がなく、大きなアンロックだとMatan氏は述べた。
- ●Lindsay氏は、AIで作ったキャラクターは自分のIPで、Pikaは商用ライセンスを提供すると述べた。またAI時代にも創作者自身が倫理と独創性を守る責任があるという見方を示した。
章立て
導入:プロンプトから演出へ
従来のAI動画ツールとの違いと、ゲスト紹介。Pikaが創作の未来をどう捉えてきたかをDemi氏が語る。
Enactが変える演出体験
MatanとLindsayが、キャラクターを動かすと実際に反応する体験の直感性と没入感を語る。
構想の背景と実現プロセス
モデル、テンプレート、エージェントを経て、現実の演出を模したリアルタイム演出に至る流れ。Matan氏のデモ映像が北極星になった経緯。
技術の裏側:H3 Flexと合成データ
遅延のないストリーミング型モデル、3Dと動画モデルを組み合わせたデータ生成、品質とのトレードオフを説明。
デモ:キッチンでの口論シーン
背景変更、セリフのリハーサル、本番の撮影までをライブ指示で行うデモ。
俳優への指示とエージェント層・世界モデル
記憶を持つ俳優、マルチモーダル入力、カメラ操作を別工程にできる利点を議論。
提供形態と創作物の権利
H3 FlexのAPI、Hugging Faceでの重み公開、創作プラットフォームへの搭載と、AI作品の所有やオマージュの線引き。
3年後の展望
恐竜のような非現実的な演出や、クリエイターが作品との一体感を取り戻す未来像。
解説記事
AI動画ツールの多くは、シーンを文章で説明して生成ボタンを押し、結果を待つ方式だ。The Neuronのこの回では、Pika CEOのDemi Guo氏と、クリエイティブリーダーのMatan Cohen-Grumi氏、Lindsay Brillson氏が、再生中の動画に手を入れられる新しい試み「Pika Enact」を語った。
プロンプトボックスは「ブラックホール」
Lindsay氏は、プロンプトを入力する方式を「ブラックホールに投げ込むようなもので、対話がなく、届けられるだけ」と表現した。Enactではキャラクターを持ち上げて動かすと、その動きに合わせて実際に演技し反応するという。再生中のシーンから人物だけを切り分けて、その場で物理的に自然な反応を返すこと自体が技術的な偉業だと同氏は述べている。
監督経験を持つMatan氏は、体験そのものが他のAI制御手法とまったく違い、現場に近い感覚だったと振り返る。プロンプトの精緻化やエージェントによる制御が進むなか、むしろ「基本に戻る」直感的な手法だという見方だ。同氏は、AI制作では創作者が「プログラマー」のようになりがちだが、Enactは人間的なやり取りを取り戻すと語った。
Demi氏は、Pikaの歩みをモデル、テンプレート、エージェント、そしてリアルタイム演出へと続く流れとして説明した。創作を「より身近にする」ことが一貫した問いで、対象は非クリエイターだけでなくプロも含むという。将来は、現場の演出と同じく環境を整え、俳優に話しかけ、カメラも動かす形が最も直感的になると考えている。
技術:遅延のないストリーミングと合成データ
Enactは、同社が最近公開したオープンウェイトモデルの流れをくむ「H3 Flex」上に構築されている。Demi氏は、このモデルがオープンウェイトモデルの中で速度とコストの効率が最も高く、品質は同程度だと述べた。
同氏によれば、既存のリアルタイムモデルは約5秒ごとのチャンク単位で生成するため、ストリーミングではあっても対話的ではない。Enactでは遅延のない、より効率的なリアルタイムモデルを新たに訓練した。多様な入力を条件として受けるアーキテクチャも開発したという。
データは市販品では足りず、独自のデータエンジンを構築した。3Dでシーンをレンダリングし、エージェントで3Dデータを増やし、低速だが高品質な動画モデルでフォトリアルな映像に変換する合成データだという。品質については、リアルタイム版は非リアルタイム版よりやや劣ると率直に認めた。それでも、損失のない推論の効率化技術など、PikaFlexで培った経験で補っているという。
俳優に記憶を与えるエージェント層
デモでは、キッチンの夜という設定に変更し、AliceとBobのセリフをリハーサルして、「アクション」で本番テイクに移った。俳優は指示を聞き、セリフを試し、受けた修正を覚えて最終演技に反映するという。
Demi氏の説明では、モデルはテキストや参照に加えて「インタラクション入力」も受ける、世界モデル的な構造だ。通常の動画モデルには記憶もキャラクターの概念もないため、エージェント層がキャラクターに自己認識と記憶を与える。「Sarahはもっと感情的に」という指示を覚えさせ、「Sarahが近づく」とDarrenを取り違えないようにするためだ。氏はこれを、細かなカメラ指定を書き込むプロンプトから、意図を伝えるエージェント、そして演出そのものへという、思考の抽象度の変化だと捉えている。
Matan氏は、カメラ操作を演技の確定後の別工程にできる点を強調した。現場では演技とカメラを同時に決めなければならず、あの瞬間にカメラがなかったという悔いが残りがちだ。Enactでは完璧な演技を固めてから、カメラワークの違いを試せる。
提供形態と権利、そして3年後
H3 FlexはAPIで利用でき、オープンウェイトはHugging Faceで公開され、同社の制作プラットフォームにも新たに入ったという。
権利についてLindsay氏は、AIで作ったキャラクターも、粘土や絵画で作ったものと同じく自分のIPだと述べた。Pikaは商用ライセンスを提供しているという。参照の扱いは、オマージュか盗用かを創作者自身が律するべきで、道具が何であれ基準は同じという立場だ。
3年後について、Demi氏はエージェントを超えて、実際の演出と同じ直感的で視覚的な制作に移ると見る。しかも恐竜を演出するような、現実にはできない表現も同じ体験でできるという。Lindsay氏は、AIに慎重なクリエイティブ職の人々が抱える、新たなスキル習得の負担や、作者性が薄れる感覚を和らげたいと語った。
まとめ
Enactは、AI動画の制御を「文章を書く」作業から「現場で演出する」作業へ近づけようとする試みだ。遅延を避けるためのストリーミング型モデルや合成データの工夫は、画質との折り合いを伴うが、同社はそれを認めたうえで進めている。日本の制作現場にとっても、プロンプト技術の習得より演出の経験が活きる可能性を示す点が注目に値する。権利や倫理を創作者自身が担うという考え方も、AI活用の議論の参考になるだろう。
文字起こし(英語・自動生成)
When you're typing into a prompt box, it goes off into a black hole, and there's no interaction, there's just a delivery. But with an act, even when you're moving a character around in a scene, they are acting and responding as though they are really being picked up and moved. Everything about the creation process with AI is about controlling exactly what you want. We become, you know, progress of like, you know, the creative process. And something about it, bring it back to a very basic human level, interacting and trying to mold together what you want to achieve with an actor or a character. It's definitely a challenge because making more efficient and more real-time obviously will have some trade-offs in terms of quality. The real-time model is slightly worse quality than our real-time version. Welcome, humans, to the Neuron AI Explained. So most AI video tools have you describe a scene, hit generate, and wait to see what comes back. Pika has a much more ambitious idea.
What if you could reach into the video, move an actor, talk to the cast, or change the camera while the scene is still playing? Today, I'm joined by Pika CEO Demi Guo and creative leaders Bataan Cohen-Grumy and Lindsay Brilsen to discuss Pika Enact, the technology behind it, and who really controls the creative process when AI starts taking direction. Before we get started, please take just a second to like this video and subscribe so you don't miss our other interviews and live streams moving forward. And on that note, everyone, welcome to the Neuron. So excited to have you here. Thank you. We're excited to be here. Yeah, thank you for having us. Excellent. Support for today's show comes from Alumni Ventures. Your financial advisor might be perfectly happy keeping you in public index funds and spending the afternoon out golfing, hoping you don't call. But public markets are not the whole market anymore. For more than a decade, Alumni Ventures has given individual accredited investors another option, access to blue chip venture investing.
Their team does the professional diligence while you get the opportunity to build your own portfolio and invest alongside leading VC firms. Time and CB Insights both rank Alumni Ventures, a top 20 U.S. venture firm. You can sign up for free, see the weekly deal flow, and decide for yourself. Or write one check into the Foundation Fund for a diversified portfolio of about 25 private venture deals. Go to av.vc slash neuron. That's av.vc slash neuron. Now back to our regular episode. I guess to start, Demi, Pika began as a way to generate AI video from prompts, but now you're showing something that looks much more like interactive film set. And you kind of walk us through what is Pika in ACT, and what was the moment when you realized that just generating a finished clip wasn't going to be enough? Yeah, for sure. I guess our company, you know, since the beginning, we were always very excited.
building towards the future of AI for creative work. We feel like AI will transform our industry as we see as we transform engineering, legal, and prior work and so on. And what we're very excited about is how AI will transform the creative industry. I think since the beginning, we founded a company a couple of years ago, and we've been always trying to build research and products towards this mission about defining and thinking about what's the future of creative work. Generally speaking, we just feel like creative work, the hope is that in the future people will be a much more accessible way for people to, you know, do the creative work and to really express their creative ideas. And this approach, I think something we're going to share today, is also one along the line that what we're trying to innovate,
how the future of Krayt will look like. Yeah. I can talk more about it, but happy to, yeah. Yeah. Share it later. I guess I want to ask a question of Matan real quick here, too, because I know that you've worked in traditional productions in the past. You are a director. And I'm wondering, what was the first thing here with an act that made you think like, okay, this thing really changes how I direct a scene. This is something. I think for me, like conceptually that sounded interesting, like the vision for it when Demi suggested it. But I think my mind was blown away just experiencing it for the first time because it was like the first time that, you know, it felt very close to home, I would say, But also, even without thinking about, like, old experiences for directing, just the experience on its own just felt, like, so different. We tried to control the creation in AI in, like, many different ways, but that felt like something else.
Yeah. What can a human director communicate by grabbing a character or giving a spoken note that a long text prompt just can't get or may never? I think there's something just very primal about it, right? Like very intuitive and primal about this method of communication because, again, it feels very close to how you would do it on set. So in one sense, like when AI is advancing and we're trying to control it better with better prompting and agents and whatnot, this feels like going back to the basic in a very intuitive way. So, you know, just talk it out, right, like you would do on set. yeah and i just want to build on what matan's saying like there's something very visceral and um just sort of magical about the response that you get when you're typing into a prompt box it goes off into a black hole and there's no interaction there's just a delivery but with
um an act it's even when you're moving a character around in a scene they are acting and responding as though they are really being picked up and moved. I notice they swing. Yeah. Yeah. I mean, I was blown away by that. Like, first of all, it's just like a feat of technology to be able to isolate that person while the scene is still playing. But on top of that, to have them reacting in real time physically in a way that really matches the action that is being enacted upon them is just incredible. But, yeah, to have that kind of immediate feedback is just truly a different feeling, a different experience, experience a different degree of control, I think, that you feel as a creator that makes you feel just much more connected to the output.
Definitely. Yes, not only that you're part of the experience, I think it's different than trying to control the experience. Yeah, I'm also happy to maybe also talk a little more about the premise of this. So I think we started a company three years ago. So I think, like I mentioned, that was kind of our goal to really explore and innovate along the line of defining how the future of creative production or creative work will look like. Initially, it was mostly the model side of things, so building video models. Later on, it might be templates, having more templates for people to more easily be able to create videos. And nowadays, we're about agents, right, building agents to make the creation more accessible with AI. and enact, like, the real-time directing experience is another step that we think potentially in the future we will, again, transform how people do the creative production. Like, we're thinking, okay, like, I think the overarching question we're always asking is, like, how do we make the creation process more accessible?
It's not necessarily more accessible to non-creators. It's also more accessible to creative professionals. Like, how do you – it's really about making the creative idea more accessible. into production. And we feel like potentially, Asian obviously really improved the accessibility much easier than you have to, in the past, you have to write very long prompts. Asian may get much easier. You were just able to talk about intentions. But we feel like potentially in the future, the best way for people to create videos or create films or create advertisements is actually mimic a same directing experience as how people are doing in real life today, but with real-time AI technologies. So, and in that, there's, you know, design developed in that premise that we're thinking, okay, in that can, you know, it's just like how you would direct in real time. Like, in real life,
we feel like that probably might be the most intuitive interface, because that's how people are doing in real life. Like, you're able to real-time stage the environment and real-time, like, talk to the characters to direct them on how they should act. And also, real-time controlling the camera. So, all the pieces that you would do in real-time, like, real-life directing today, you will be able to do it with real-time ad technology. which I think is really transformative because it's a really different – we feel like it's a very innovative and a very different experience than the current creation today in which it's still a little bit accessible for some people. That makes sense. Yeah. This is – I mean, we haven't even talked about this, but this is an ambitious project.
Like, this was not a thing that is, it had not even crossed my mind yet that this is a way we would want to interact. And, I mean, obviously it is. The second I saw it, I was like, oh, of course. And one of the things I'm wondering is when this idea came about that we're going to do this, how can we make this work? Did it seem like we can do this? Or was there a lot of, is this even possible kind of mind? what was the mindset around it? Yeah, it's, we're always like, you know, for us, like innovation is like, I think our company is creative at heart and also really always thrive to innovate. And also we all like whether you in junior research or creative team obviously are all like both technical and also creative And then we always you know creative at heart innovative at heart We're always willing to take challenges and to invent what's not impossible. So it's not some – it's, like, we're always, like, that's kind of our philosophy.
And kind of, obviously, it's very, like, it seems very, like, impossible. and it has really developed through like a constant iteration between the creative and researching. So, yeah, it's really maybe Matan can talk a little bit more about process. Yeah. Wow. I think before I talk about the process, I think one thing that excites us a lot of the time is seeing something for the first time that you've never seen before. I think in this process, Demi just kind of had this vision and outlined the idea. And at first, you know, very visionary ideas, even as simple as they, you know, may seem. Sometimes they are hard to grasp because you never saw them before. So for me, my process was like confusion, like what, how? And then usually what I do is I just try to quickly visualize that before we even try to build. Because there's, you know, there's one way you can talk about an idea, and I think it's a whole different process to see that idea.
and I'm a very visual person. So the first thing that I was trying to do was just, like, trying to visualize, like, okay, wait a second. How does this look like? And I think once we saw it, like, we, like, was maybe, oh, this is very exciting. And then came the part of, like, actually making, turning this into reality with research and, you know, making this a possibility. And it starts looking like you can do it, I guess. Yeah. And I just to share a little behind the scenes, you know, Matan is such an incredible bridge between creative and technology. He frequently does visualize these ideas by turning them into video demos that become kind of the North Star for everybody in the company to work towards. and I would say I remember seeing this video a few months ago and thinking, like, that is insane. That is, like, years in the future.
And the demos that we're looking at today are very, very, very close to what he had put forth in that video. The best ideas always sound like crazy talk for a minute. Like, I imagine the idea for the iPhone was, it had to just sound insane to people. But what you all brought together here is something really cool, and it seems very complex. I want to ask a few more questions about the live staging and kind of work through each of the pieces here. One of the things that's really interesting about this to me is as I'm watching, for starters, watching a character be moved, I feel like there must be a lot of things that the tool has to get right for that to work.
Is this really complex? Yeah, obviously. So I guess this model is developed on top of first. So we actually recently launched our own, like, model, developed open-weight models. So our own, like, big models, developed open-weight models, it's actually rent number one among all the open-weight models. And it is, like, the most efficient in terms of speed and cost while maintaining really high quality with similar quality with the other open-weight models, but the most efficient. And this, yeah, this ENACT is developed on our own, like, model. That model is called H3Flex. So it's developed on our PicaFlex. And we, you know, it's a very, like, because it's a very innovative task, we have to do a lot of innovation around technology as well,
both including how do we build the data and how do we train design architecture and train the models. So we, on a high level, we build our own data engine for this specific task using a combination of 3D approach and AI-generated videos to construct the data. for this task. In model architecture-wise, we also have to develop new technology that help the model to condition on these very diverse type of inputs. There are many real-time models out there, but most real-time models actually have some delay. I'm not sure that everyone understands. So all the current real-time model models out there, like the common one, like I think H3 Max or something, they all have like five-second delay.
So it's really like chunk-based. So it's like you have to – it's real-time streaming, but it's not real-time interactive because it's like generating like five-second by five-second by chunk. But our model, because we require real-time interactive activity, we actually had to train a much more efficient, real-time model for this use case, which are really streaming, so then it doesn't have this delay issue. So that's a very, I would say, innovative technology as well. And on top of that, we also have to build technology for specific, for the condition that we want to achieve. You mentioned that in talking a little bit about the data for creating this, that you've had to do a lot of that yourself. And I understand you had simulated 3D stand-ins and working with mouse trajectories
and all kinds of things, trying to keep backgrounds straight. Like, that feels like that's not just data you go buy off a shelf or download off the Internet. Yeah. So for the data, we basically do our own engine for generating. It's actually more synthetic data. So it's more like we, it's a combination of, like, we render in 3D first and using agents to render more 3D data. and then we render it into photorealistic videos through a slower but higher quality video model. Okay. Very cool. Very cool. It seems like one of the hardest parts might be turning a model that renders a complete clip into a continuous stream that's able to respond to you as you go. And I'm wondering kind of how did you make that work? to what extent you're able to share without the quality falling apart.
Yeah, for sure. It's definitely a challenge because making it more efficient and more real-time obviously will have some trade-offs in terms of quality. To be frank, like, obviously the real-time model is slightly worse quality than our, like, non-real-time version. But, yeah, indeed it's a challenge. So it requires a lot of, like, innovation on, like, it requires, like, making sure the data is really high quality and making sure the architecture is the right architecture, but also a lot of, like, inference, efficiency inference techniques that are, like, lossless or, like, and requires some engineering and some experience as well. which is for us we had the, I would say have the advantage or luckily we had a lot of experience on those efficient inference techniques because other models such as the PicaFlex is also really
was able to achieve such efficiency is also because we were our team was able to apply a lot of efficient inference techniques on both model side on inference and on more infrastructure side. Nice. I'd like to also talk a little bit about directing actors, because this is something that is really, really fascinating. Hi. Hi, both. How are you doing? Hi. We're both doing well, thanks. How are you doing? Yeah, we're good. How about you? Great. The scene we're going to do is a dramatic fight between the two of you. I think first we can rehearse the lines. But before we do that, can we maybe change the background to kind of like a kitchen night scene? Yeah, a kitchen at night sounds good. I'm ready to rehearse the fight. Let's bring the lights up, please.
It's a little bit too dark. Okay, great. let's do this line by line Alice your first line is you said seven I thought something happened to you you said seven I thought something happened to you Bob then you reply to her nothing happened I was just working nothing happened I was just working Alice you say back to him you could have called but you choose not to. You could have called but you chose not to. Bob, then you reply back to her, I'm doing all of this for us, for both of us. I'm doing all of this for us, for both of us. Alice, this is where you get a lot more emotional and you say,
there is no us if I'm always alone. There is no us if I'm always alone. And, okay, that's great. Now let's go into action. Let's do all of the lines. And I want you to, first, before you start saying your lines, face each other. And when you're done with your lines, Alice you should kind of grab Bob and then you know you should turn away and leave the frame And Bob you stay there Very upset And action. You said seven. I thought something happened to you. Nothing happened. I was just working. You could have called, but you chose not to. I'm doing all of this for us, for both of us. There is no us if I'm always alone. and cuts. Lindsay, with live direction,
you can tell an AI actor to move closer to someone, deliver a line differently, or change the set, and the actor responds, the video keeps going. How does that feel creatively? Does it feel more like editing or directing actors or kind of improvising with a cast, or does it feel like I'm doing something weird sitting at a computer? well like i like i was saying with the the real-time dragging and the way that there's just an immediate response that feels so human um it it really truly does feel different than typing into a box hoping for the best getting something back two minutes later being able to see the actors sort of acknowledge, listen to you, take something in, rehearse a line, play it back to you immediately. And then also remember compounding comments and feedback and take that into the final performance.
I sound like I'm like a super fangirling over our technology. And I kind of am because it is really kind of mind blowing. It's just so different as a creator than it is to do it the old way. The old way. The old way of using a text prompt. The old way of Q1. Right, exactly. The demo describes a rehearsal mode even, and how saying action then to turn that exchange into like a clean take. And I'm wondering, like, what can still change between a rehearsal and a final performance? I mean, I think just like any directing experience, and I'll also give this to Matan because he experienced it firsthand. You know, you can change your mind. You can have them try a number of different ways. You can land on one, tell them to stick with that, lose the others,
and also see a bunch of different camera motions, see different motions, see different gestures. It's just wildly different. Yeah. Nice. Matan, what was your experience with it? Yeah, I think there's something in the process, something in the way that, you know, we work with an act is kind of humanizing it back again because in a lot of senses we kind of lost that. We're like very, like everything about the creation process with AI is about controlling exactly what you want. And, you know, we become, you know, programmers of like, you know, the creative process. And something about it just, you know, bring it back to a very basic human level of just, you know, interacting and trying to get, you know, mold together what you want to achieve with, you know, an actor or a character.
Again, it felt like I felt two things the first time I tried. It was like, oh, this is definitely a glimpse of the future, but also, oh, this is also familiar, you know. So it was very interesting to see, I would say. And, like, you know, it's very different. Very different from any other. It's very different from any method of creation within AI that I've experienced before. That's awesome. Well, I understand two systems work together here. A script model that converts your direction into short-time beats and then a video model that's working with there that's helping perform them. There being, like, kind of multiple layers to this. I understand there's a script model, a video model, and a voice model because you're also speaking to it and communicating with it. And I feel like there's this group of different AI working together to build this thing.
Is that correct how I'm seeing it? Yeah. So, I mean, from a technology standpoint. Gotcha. Yeah. So, that's more like an architecture design thing. So kind of the model we developed, traditionally most of the models are just taking to text input, and maybe now with reference to text to video model, I guess, reference to video model, you're taking to, like, multiple references, right, like, in-edit reference, text reference, auto-reference usually. So for this, you're actually also taking to interaction input. Yeah. And then, so it's, like, it's more like a module model. model. It's like, in some sense, if we use applying a best word that people are talking about, it's more like a world model because it really models a different aspect of modality and a different aspect of interactions. So to answer your question, yeah, the model basically natively take into all those modalities and interactions. That's important as outputs. The agenda layer right now is mostly about more when you're
real-time directing. so um because the hope is that okay you're talking to actress right in the real time yeah so you might talk to different characters or different actors uh the hope is that you can like a very what we're hoping to achieve is that okay talk to the actor a i don't know sarah and after b i don't know like baron or something and then like um and then like you're able to tell oh sarah can you like stand a little bit closer to me or set you can guys stand closer or can you make sure you're more emotional when you're talking about a script so the agent later really comes into like regular for video models there's no like memory and there's no like concept about self or concept about characters so the agent like there is really helpful for that our goal is like okay so sarah when she start acting she'll remember i tell her to you know When she recites the line, she should be more emotional, for example.
And she should recognize when I say Sarah, Sarah is moving closer, it's not Darren's moving closer, you know. So the age player really, like, give the characters or give objects and characters in a scene a meaning and self-awareness and memory. So that's how it is, yeah. And I guess you still have to give them pretty specific direction, and just like probably to some extent maybe like you would with a prompt as far as making sure this actor understands, like, I'm talking to you here and I'm talking to you here. But, I mean, I guess you've got to do that in person too. I think I keep seeing this kind of a middle ground that's probably going to continuously get a little more like working with people. Exactly. The agent is really trying to, it's really like, what's fascinating about this is really about change of mental model of how you create videos. So very initially, it was mostly about prompting. So the mental model is really like, you have to actually prompt, okay, 35 millimeter.
I'm not even creative. It's probably Blinxing, but I know it's better. Like very precise. Oh, this is a muted tool. blah blah like they're very precise like description like of the objects and characters with agent it becomes like better because you can describe the intention of the video to create um but then with this new interface in that real-time interface we're developing is it become even a higher distraction of mental like way for you to think so you don't have to think about okay as a video architect, I think about people and environments. Yeah, I think there's two parts of it, right? The first one is writing your idea down, and then there's treatment and delivery. That's what a director does, in a sense. He takes a script, and he works with the actor about the delivery of that concept or script. So, again, coming from not from, because I'm not a script writer, but coming from directing, that felt, that piece that was missing, in a sense. Yeah.
And also it's a more, like, iterative and visualized process versus before. So, yeah, it's a more, in our sense, more intuitive. It's, like, you don't have to think about, oh, like, it's less about, like, okay, what kind of video you want to create. But it's more about, like, oh, like, should it be standing closer? Or you, like, visualize it and see, oh, maybe some lighting here. Or maybe the camera's around. Or maybe you should be a bit more emotional about it. So it's more thinking real-time, moderative, and also more iterative, just like how you talk to actors. This keeps bringing me to the idea of, like, when I think of live camera control and things like that that are really ambitious and that you all have implemented in this, are you making, like, kind of a world model type thing that is able to, I mean, to be able to see off camera? some of that has to, I assume, exist and auto-generate. And I'm wondering, is it generating this as it goes,
or is it kind of building the world in advance, and then you're able to just work the camera from there? Is that – I realize maybe that's a little more technical, but – Yeah, it is a world model. So, I mean, people have different definition world model. It is world model in the sense that it is real-time, it is interactive, it is modeling the world and it takes multi-modalities. And so, yeah, and then it's technically the model can do anything, like also the gameplay or whatever, but because we're really focused on defining the future of creative work again, so it's really a world model specifically for how people will be able to create videos in the future, especially like focus on, I would say, that use case. Yeah, but a lot of technology are very fundamental in a sense that can be potentially applied to other use cases, too.
Okay. Okay. I would say one thing about the camera director part. There's maybe a little bit different from the native experience of, like, directing on set, but it's also a huge unlock. Because when you're on set, you're trying to kind of control everything together and make it come together. so the camera and directing the actors that need to happen simultaneously. And in this process you kind of separate the two parts to like two phases where you first you know you get your perfect performance first from the actors And then you can go in once you lock that perfect performance and, you know, start, you know, seeing different versions of just like the camera work. And I think that's also very intuitive to camera work because when you think not about a director, but more of like a cinematographer is watching through the lens. and, you know, we direct the camera in real time while he's watching this scene. So I think that feels like a huge unlock, I think.
And you're not dealing with 20 takes of the same scene where you didn't have the camera where you really wish it was for this magical moment that's not going to happen again no matter how many times you try. Exactly. Exactly that. That's the exact unlock, I think. That's so cool. So as far as availability goes, I understand H3 Flex can be available via API, correct? Or is, I should say. Yeah. It's been out. It's available via our TK API Club. It's also open weights are available on Hug and Face. And it is newly available in our creation platform. Awesome. Yeah. Very, very cool. I also have a few questions I'm wondering about around the creative ownership end of things. And Lindsay, there's sort of a new kind of control here around this sort of stuff. And I wonder if just because AI enables me to bring in a photo of a person or generate a speaking performance or something, who should really own or govern, I guess, maybe is the question, that likeness, that voice permission.
Because I assume even if you're creating a character in much the way that if that was the image of an actual person, is that your person? Is that your actor if you've created them? Whoa. This is a question. Yeah, I mean, you could think of it as your own IP, right? If you've created something with AI, it's no different than if you've created it using clay, paint, illustration, you know, your own personal performance around the bunny that you've decided you are. Yeah, so for sure, if you've made this your character and, you know, we provide commercial use licenses for that in Pika, it is yours wow it's an act of creation right? yeah different
it crossed my mind as I was reading about this beforehand and I was like who owns the creative it felt like I guess it's creative in the sense that we think of if you made an ad for someone or shot a commercial or you designed a book cover Right. And I'm wondering, as a creative director here, how do you decide where the line sits between, like, a shared idea or ordinary creative references maybe? Do you feel like that's kind of blurry right now, or do you think there's any idea where this will go as a direction? Just with, like, AI in general, I know that there's just a lot of questions around ownership with training data, et cetera. But I think that the same standards of integrity and creative originality still hold for people who work in the industry, regardless of the tool that they're using.
So if I illustrate something by hand that looks exactly like another, a well-known artist, it's the same as if I pull in a bunch of references into AI and like try to replicate it. It's you really have to be the police. You have to police your own actions and you have to be aware of what is homage and what is stealing. and it really is in the hands of the creator to be aware of that. But all great creative work is born of inspiration. You know, it doesn't exist in a vacuum. So I think that people who, I mean, I'm getting a little, you know, soapboxy here, but I think that people who complain too much about references are sort of ignoring the vast history of our artistry and creation and how everything is informed by something else. I think Eric clapped in one set of music, there's only 12 notes.
You know, there's only 12 notes. And we've all been playing them for the history of music. And I always thought that was really interesting to remember. I have a question I'd like to ask all three of you before we go. Imagine we're having this conversation, let's say three years from now, So what do you expect will be the biggest difference broadly around making video from the standard today? Yeah, I can start. I think kind of the enact is kind of our vision of what maybe will evolve in the future. Because right now, video creation, I would say most people are still using, like, web, like, app-based video creation, which is more like prompting, maybe slightly easier prompting now, and templates, apps, mini apps. There's some people starting to use, like, agent, MCP, or agents.
And we feel like, you know, even with agent, it's still not the best creative process. It's still not the most accessible process and not the best process because it's still fairly technical. It is not very intuitive, like how people direct in person. You don't feel like visualized or like you don't feel like as much control. It's still a bit technical and also lose a little bit of fun of the creation process. Yeah. So we actually think that future maybe the creative process will actually transform from the agent to the real-time directing process we're talking about, which people are, the technology will immerse people to have the same intuitive, controllable, and visualized creation process like how can we work during a person today, except with, like, much less constraint. It will be much cheaper. It will be much less constrained. Like, right now, you cannot actually direct a dinosaur.
But in a real-time directing process, you actually can direct a dinosaur. You can see this all the time. I never even thought of that. Wow. But, yeah, so, like, you can direct the actors to really complicated actions that's not possible, right? And clothes that's not possible or direct very impossible characters. But they're in the same experience as how you would direct a real person. So you don't need to be at that stuff. But then that, I feel like, is very exciting. Wow. Maybe, I don't know, I think some things are elaborated more. the pond do you think who has some thoughts yeah I think you were spot on it's not just like how can we do it the same way we did what more can it unlock using this it shouldn't constrain you in any way it should do the exact opposite you won't need to hire a bear on set around all of your people
or things like that when you need animals that's wild and the animal handlers which are always their own special breed I've heard the story yeah I'm sure you have from my perspective I would answer the question slightly differently and say that I hope three years from now the types of people who are creating are different vastly different from what we see today you know we were talking before the show started about adoption of AI creative, both from the creation perspective and the consumption perspective in the United States, is not as high as it is in the rest of the world, and there's a lot of contention around it. And I think for creative professionals right now, the adoption has been slow, and there have been a number of barriers to that. One is just the upskilling that you have to do. I have to learn entirely new modes of doing the thing that you've always done well.
And now you're not maybe as good at it because you have to learn the art of prompting. And there's this equality issue where it's never quite what you want it to be. And then I think that there's this element of control and authorship. I think a lot of the outrage is that it feels too easy or you're not really the person who has produced the work because AI has done so much of that. And my hope is that things like Pika Enact and all of the initiatives that we're working on can return that feeling of creative ownership and craft to creators so that they feel closer to the work. They don't have to learn all kinds of new things in order to make. They can flip back into their old ways of creating and focus on making incredible things. Yeah, because having a vision for something in your mind that you want to create
and also having this set of personal tools it takes to go do that, there's a lot of barriers there. And I think what you all are working on really knocks a lot of that down. Demi, Matan, Lindsay, thank you all so much for joining me. This has been fascinating. Thank you. Thank you. Before we wrap up, where should people go to learn more about Pika and Enact and H3 and all of the other great things you all are working on? I think our website, Pika.art. Pika.art. Will be a great place to start. Cool. Well, to everyone watching and listening, thanks so much. We appreciate you being here. If you enjoyed today's conversation, please take just a moment to like and subscribe. You can also stay on top of all the latest AI developments with our daily newsletter, The Neuron, and join around 750,000 people who read it every morning. And we'd love for you to be one of them. And that's all we have for today. So farewell for now, humans.
番組の概要欄(原文)
<p>What if making an AI video felt less like writing a prompt and more like directing a film set?</p><p><br></p><p>Pika is experimenting with a new approach called Enact: interactive AI-generated scenes where a creator can drag an actor into position, speak to the performers, or rethink a camera move while the scene is taking shape. Instead of asking for another finished clip, the director participates in the process.</p><p><br></p><p>Pika CEO and co-founder Demi Guo joins creative leaders Matan Cohen-Grumi and Lindsay Brillson to explain the idea behind Enact, why these tools feel different from conventional AI video generators, and what their team had to invent to make responsive generation work.</p><p><br></p><p>We also get into the limitations: video quality versus speed, what an AI actor needs to remember from a director, camera control, access to Pika's H3 Flex model, and who owns creative work made with AI. The bigger question is whether tools like this can help filmmakers feel more connected to the work rather than less.</p><p><br></p><p>Pika: https://pika.art/Pika H3 Flex and product information: https://pika.art/blog/pika-h3-flex</p><p><br></p><p>For more conversations about practical AI and where the technology is going, subscribe to The Neuron AI Explained and The Neuron daily newsletter at https://theneuron.ai/.</p><p><br></p><p>Thanks to Alumni Ventures for sponsoring this episode and helping accredited individual investors access professionally managed venture capital portfolios. Explore their investment opportunities and learn more at https://www.av.vc/.</p>
出典・この記事について
- 番組
- The Neuron: AI Explained(The Neuron)
- 元の回
- Pika Enact: Can You Direct AI Video While It Plays?
- 配信日
- 要約作成日
- 公式サイト
- theneuron.ai/podcast
この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について
関連エピソード

なぜ FLUX 3 は画像・動画・音声を一緒に学習するのか — Black Forest Labs の Andreas Blattmann と Dustin Podell
Latent Space: The AI Engineer Podcast

【ライブ】パッチは間に合うか?AIが脆弱性を見つける速度が人間の対応を超えつつある
The Neuron: AI Explained

火山の上のAIから得た学び:エージェント時代の検証・物理AI・不確実性下の意思決定
Industrial AI Podcast

4年間でAIはあなたの記憶以上に変わった
The Neuron: AI Explained

生物学を制御する――Fabian Theis(Helmholtz Munich)が語るバーチャルセルとラボ・イン・ザ・ループ
Latent Space: The AI Engineer Podcast
AIがAIを修正するMIT研究、Mistralのオープンモデル、Google Genie 4
Mixture of Experts