AI効率を探る:Armが描くエッジAI、エージェント、フィジカルAIの安全性

Trying to get our Arms around AI efficiency

The Stack Overflow Podcast30分
AI効率を探る:Armが描くエッジAI、エージェント、フィジカルAIの安全性

AI効率を探る:Armが描くエッジAI、エージェント、フィジカルAIの安全性

The Stack Overflow Podcast

0:0030:22

要約

Stack Overflow PodcastでRyan Donovanが、ArmのAI・開発者プラットフォーム担当SVPのAlex Spinelliと対談した。ArmのCPUやGPUへのニューラルエンジン搭載によるデバイス上AIの効率化、ランタイムとの連携、エージェント向けのツール設計を語った。後半ではロボティクスなどフィジカルAIで重要になる機能安全(functional safety)と、開発者が責任を持つべきだという考えが示された。

この回のポイント

  • ●ArmはCPU設計のIP企業から自社チップ製造へ広がり、クラウドでもArmベースのインスタンスが増えているとSpinelliは述べている。
  • ●RISCベースのArmは非RISCに比べ電力効率が20〜30%高く、x86からの移行でコストが大きく下がると説明した。
  • ●NPUは各社で標準化されておらず、ArmはGPU内にニューラルエンジンを載せ、Vulkan等の標準APIで使えるSDKを提供している。
  • ●エージェントは通常のソフトウェアだが、ツール呼び出しやモデルとのやり取りが多く、既存の負荷に上乗せされる計算需要を生むと述べた。
  • ●フィジカルAIでは機能安全が重要で、決定論的な動作、機能の分離、10ミリ秒などの時間制約を守るフォールバック設計が必要だと指摘した。
  • ●規制や減速ではなく、ソフトウェアを書く開発者自身の投資と責任で安全性を確保すべきという立場を示した。

章立て

  1. Spinelliの経歴とArm入社の理由

    ドットコムブーム期の経験やGoogleでのTensorFlow、AmazonのAlexa基盤などの経歴を紹介。デバイス上のAIの価値に注目してArmに入ったと語る。

  2. Armの歴史とハードウェアでの効率化

    ArmのIPビジネスから自社CPU製造への展開を説明。SMEやGPU内ニューラルエンジンなど、デバイス上でモデルを効率的に動かす仕組みを紹介した。

  3. ランタイム対応とRISCの利点

    LiteRT、ONNX、ExecuTorchなどと協力し、最新ランタイムを使えば性能が出る状態を目指す。RISCによる20〜30%の電力効率やクラウドでのエージェント実行にも触れた。

  4. AI Portalとエージェント向け設計

    最適化モデルやツールを集めたAI PortalとMCPサーバーを紹介。エージェントは長いドキュメントや多数のコマンドも苦にしないため、CLIや公開情報の設計を変えていると述べた。

  5. モバイルGPUとニューラルエンジン

    発熱や電池消費を抑えつつレイトレーシングなどを可能にするSDKとデモゲームを紹介。NPUの標準化の問題と、GPU内ニューラルエンジンの狙いを解説した。

  6. フィジカルAIと機能安全

    Arm Total Design for Physical AIやDevice Connectを紹介し、ロボティクスでは機能安全と決定論的動作が重要だと強調。開発者が責任を持つべきだと締めくくった。

解説記事

Stack Overflow PodcastにArmのAI・開発者プラットフォーム担当SVPであるAlex Spinelliが登場し、ホストのRyan Donovanとともに「デバイス上のAI効率」をテーマに語った。ハードウェア、ソフトウェア、エージェント、そしてロボティクスの安全性まで話題は広がった。

ソフトウェア出身者がArmに入った理由

Spinelliはドットコムブーム期に独学でWebサイト制作を始め、その後GoogleでTensorFlowのプロダクト責任者、AmazonでAlexaのコアソフトウェア基盤などを担当してきたと自己紹介した。機械学習・ソフトウェアが専門の自分があえてハードウェア企業のArmに入ったのは、AIの価値が身の回りの物理的なデバイスに現れると考えるからだという。

Armは長らくチップのアーキテクチャ設計(IP)を提供する企業で、携帯電話のほぼすべてがArmベースだとSpinelliは述べる。近年はGravitonなどによりクラウドでもArmのインスタンスが増え、今年は自社CPUも製造した。開発者のワークフローに直接関わる立場になったことが、ソフトウェア面の取り組みを強める背景だという。

ハードウェアとランタイムの両面で効率を高める

効率化の手段として、Spinelliは最新CPUに搭載されたSME(Scalable Matrix Extensions)を挙げた。モデルをCPU上でより効率よく動かすための拡張で、iPhoneやMacBook、今後Android端末にも広がるという。クラウド向けにはスレッド数の大幅な増加でエージェントの実行を支える方針だ。

ただし優れたハードウェアもソフトウェアが対応しなければ意味がない。ArmはLiteRT(旧TF Lite)、ONNX、ExecuTorchといったランタイムの提供元と協力し、最新版を使うだけでArm端末の性能を引き出せる状態を目指している。NPUやGPUはベンダーごとにオペレーターが異なり標準もないため、翻訳層ではなく、ランタイム側が「そのまま動く」ようにする取り組みだと説明した。

またRISC(縮小命令セット)の設計により、非RISCと比べて電力効率が一般に20〜30%高いと述べた。これがスマートフォンの電池持ちを支え、クラウドでも移行だけでコストが段階的に下がる利点になるという。MetaのパーソナルエージェントMuseがクラウドのCPU上でマイクロVMとして動いている例も紹介された。

エージェントを「利用者」として設計する

Armが公開したAI Portalには、最適化済みモデルやランタイム、ツール群がまとめられている。15年分のチューニング知見や学習資料はMCPサーバー経由でエージェントからも参照できる。Spinelliは、開発者として人間だけでなくエージェントにも向けて設計していると語った。エージェントはドキュメントの長さやコマンドが300個に分割されていることを気にしないため、CLIの設計も人間向けとは変えるべきだという。

エージェントは通常のソフトウェアであり、モデル呼び出しやツール呼び出しを大量に行うため、既存の処理に上乗せされる負荷になると整理した。

GPU内ニューラルエンジンとフィジカルAI

モバイルではNPUが標準化されておらず、Samsung、Xiaomi、Appleなどで実装が異なる。そこでArmは、標準化が進んだGPU開発の流れに乗り、GPU内にニューラルエンジン(行列演算に最適化された回路)を載せた。Vulkanなどの標準APIでニューラルネットワークベースのレイトレーシングやアップサンプリングを、電池消費や発熱を抑えて使えるという。実証としてゲーム「Neural Dawn」も公開した。

ロボットやセキュリティシステムなどのフィジカルAIでは、約80社が参加するArm Total Design for Physical AIを立ち上げ、検証済みスタックの推奨を目指す。Device Connectというオープンソースの枠組みでデバイス同士をAIオーケストレーションでつなぐ試みも紹介された。

機能安全と開発者の責任

Spinelliが最も強調したのは機能安全(functional safety)だ。自動車業界では以前からの概念で、ソフトウェアが決定論的であることが求められ、最適化は予測可能性を損なうためコンパイラの最適化まで切る場合があるという。非決定論的なAIを扱うには、センシング・思考・判断を分離し、たとえば10ミリ秒以内に返答できなければフォールバックに切り替える、あるいは自動運転を解除するといった設計が必要だと述べた。

規制による減速には否定的で、他国は待たないという見方を示し、業界の自主規律と開発者による投資が重要だと主張した。品質テストやゲートを作るのは開発者自身であり、責任は人間にあるという立場だ。

まとめ

この回は、Armがチップ設計企業からソフトウェア・開発者エコシステムへ軸足を広げていることがわかる内容だった。日本の開発者にとっては、オンデバイスMLでランタイムを更新するだけで性能が得られる方向性や、エージェントを前提としたツール公開の考え方が参考になる。ロボティクスや組み込み分野では、機能安全の標準を学ぶことが今後重要になると示唆された。なお数値や製品の効果は発言者の説明であり、実際の評価は各自で確認したい。

文字起こし(英語・自動生成)

Hello and welcome to the Stack Overflow podcast, a place to talk all things software and technology. I'm Ryan Donovan, your host, and today we're talking about AI efficiency on all things devices. My guest for that is ARMS Alex Spinelli, who is VP of AI and Platforms. So welcome to the show, Alex. Yeah, thank you. Thanks for having me. So top of the show, we like to get to know our guests a little bit. Can you tell us a little bit about how you got into software and technology? I guess I have a little bit of a circuitous kind of entry into engineering. And I actually didn't go to undergrad in an engineering capacity, but I'm 53, 54 now. So I grew up in the first dot-com boom. My dad was an engineer. I had mini computers in my basement and bolden boards and those kinds of things. So I kind of grew up coding and building things.

During the first dot-com boom, I was someone I could code. I could write things. Yeah. And I had some friends who were like, hey, they were graphic designers. I'm like, hey, do you know how to build a website? Yeah. And I was like, I had no idea how to build a website. They're like, yeah, sure. Sure, of course. And started building websites when people just were first starting building websites. I remember those days. And then, yeah, like, what's this Internet thing? And, you know, got my first Linux box and built that from scratch. And I really fell in love with being able to deliver things that millions of people can take value from and use. And then did end up going to school and working through engineering and then an MBA and have had roles at Google, where I used to lead the TensorFlow group, product management, product strategy. the core software platform for Alexa at Amazon and a bunch of various roles in the early New York

finance and media circles. And so I've been working on AI and ML before we called it, you know, with the big data and text analytics for a very long time. And so it's pretty awesome to see what's happening now. Sure, yeah. The acceleration is crazy. I mean, I think we've been talking about various iterations of AI since the 50s or 60s. Yes. But today's AI is getting productionized, getting very much integrated into everything. And we're getting a lot of conversations as it gets into production about the sort of efficiency of it, especially in terms of the hardware uses as AI comes onto devices. Yes. How are you and folks at Arm thinking about that efficiency conversation? Yeah, well, I landed at Arm as a hardware company. I'm really a machine learning, software, large-scale technology person by background. But I very deliberately did that. And it actually plays into this efficiency conversation. And so I am a big believer in the power of what AI can do to improve people's lives, create opportunities.

There's going to be disruption, and we can talk about that for sure. It's a very hot topic, both in sort of employment and security and all those kinds of things. But really, I think it's devices that enter the physical world, surround us, is where I think that value is going to be had. Enabling developers to be able to bring their visions of AI and machine learning into the real world and do it in a way that is efficient. And we can define efficiency because there's probably 10 ways to define it. But it's effective, it's efficient, meaning it's cost effective, it's performant, it's secure. when I met the folks at ARM, I was very aligned with the way they were thinking about it. And, you know, ARM is one of those companies, you know, not everyone knows ARM. I mean, I think in the industry, we think ARM, of course. Yeah. But a lot of the development community is not aware of ARM. Yeah. I think, you know, ARM has largely been a chip designer. Right. And for more invisible spaces, right? You don't worry about, you know, this recording device that I'm using.

Right. I don't know what kind of chips it has. Yes, and so the company's history is intellectual property, so exactly what you're saying, the architecture and design of chips. We didn't make them, but they're ubiquitous. It's something like 99.x percent of mobile phones are ARM-based. Right. And now, if you look at the cloud with Graviton and our own chip, we now do make chips. So this year, we actually have produced our own CPU, and then we're just getting started in that space. Ubiquity is even growing. And most cloud instances now are ARM-based instances, be the Graviton and soon-to-be AGI CPU. And so how to bring AI and ML efficiently, again, in that broader definition, has become really, really important for us because now we are in someone's developer flow. And so that's why I joined, is to really kind of help figure that out. And it's hard. There's lots to do. Yeah. For sure. And the devices, that's where you sort of need the efficiency at the most. That's where you have the limited resource, especially as people start moving AI on device.

I know there's tradeoffs around quantization and performance and all this stuff, but how can that efficiency be enabled by hardware? So there's a few things. At the architecture level, ARMS worked really hard to bring increased capabilities into CPUs that run on devices. And this might get a little inside baseball for your average developer. Right. But there's something called SME, which is Scalable Matrix Extensions. That's actually in our latest CPUs that are in your iPhones and MacBooks and coming to Android phones. And that actually allows you to actually run models more efficiently on your CPU, on your devices. We are looking at radically increased thread counts for CPUs in the cloud, for example, that allow agents to run. because agents are common software, but they're incredibly multi-threaded applications, right? In this GPU space for mobile, actually, ARM is bringing neural engines,

so really efficient extensions in the GPU that actually are tuned to modern ML and AI. And so those phones are actually rolling out both in phones from Xiaomi and Vivo and soon-to-be Samsung. And so unlocking those operators effectively inside hardware is a big part of what we're doing. But you could have the best hardware if you don't have great software and you can't actually allow developers to take advantage of it. It's kind of a paperweight, right? Yeah. I mean, with any sort of hardware that enables software, you have to have the sort of direct interface, right? You have to be able to program directly to it. With traditional CPUs, it's like the SIMD. For ARM, how do people access the ARM chips? ARM has this awesome history of being a great citizen in the ecosystem. Super big commitment to open source, to bringing different competitors together to create some of these standards and really help developers get the most out of their hardware.

That's where we're trying to play. And so if you look at your core frameworks and runtimes for machine learning, and so there runtimes for example LightRT formerly TF Lite Onyx Executorch from PyTorch we really working hard with each of those organizations to bring the highest performance optimizations that can take full advantage of this latest hardware that I was talking about. So SNE the neural engines in our GPUs and making sure it's a lot easier. Today it's just to be very direct it's hard to deploy ML and manage ML across devices You have this alphabet soup of runtimes and backends. You have NPUs, GPUs, CPUs. NPUs from different providers are different. They have different operators. There's not a standard around them. And so we recognize that it's difficult. But hardware companies have their own perspectives, and they're trying to do the right thing for their consumers.

But we're working hard to try to create a software layer working across the ecosystem, on those different providers that can provide as much kind of cross-device capabilities, optimizations, and make it easier. And we're getting there. And it's still early days, but I think, you know, we're really going to keep iterating on that. Are you talking like a translation layer? Like, I know, like, the GPU providers have their own sort of proprietary. Yeah. No, no. So it's less about a translation layer and more about if you use the latest runtimes, making sure you don't have to do anything else to get, to take advantage of the latest hardware. So trying to help the runtime providers just work, right? And so that if you've taken the latest Onyx, you're going to get the best performance on an ARM, you know, ARM-based phone, right? And so that's the latest. It's interesting because I talked to a lot of folks who are reporting on the space and it's a little bit like we got your back developers.

But you don't need to worry about this because we've worried about it. But what they do need to worry about is, you know, really keep pace with some of the latest advances. That's the thing about ML right now is you have to build technology and your software stacks so that you can swap things out very quickly because stuff is advancing so fast that there isn't sort of a six-month plan for your software stack. You need to be able to kind of move pretty quickly because there might be a really cool advance that just came out and it's going to, we're going to have another one in another few months. Yeah. And, and, but the, you know, the software moves a lot faster than, than hardware does. Are you able to sort of future proof that runtime or have it enable new things? Yeah. I wouldn't say we're future proofing the runtime. I think, I think what we're trying to do is help a developer squeeze out as much performance as they can on that current hardware, knowing that those cycles do, they're a lot longer than software, obviously. Right. Yeah. Yeah. ARM came from a sort of different philosophy of chip making, CPU making,

where the x86 stuff has, like, big pockets of compute. And, you know, ARM traditionally had the smaller, I think it's the risk. Yeah, risk. Reduced instruction set. Yeah. Is that, you know, the reduced instruction set sort of feels like how the graphics cards, the GPU providers operate with, like, small bits of compute. Do you find that that has been an advantage for ML and AI? I mean, I think the ARM architecture and this risk-based approach has proven to be really important for the industry. And what the reduced instructions that allows you to do is actually increase efficiency, reduce power, demands. You know, if you compare an ARM compute engine to a non-risk-based compute engine, we typically run at 20% to 30% better efficiency in terms of power consumption. And so I think that unlocked, you know, our mobile phones and battery life and all that kind of stuff. It's a big deal.

That's why we actually went into the server market as well, especially if you think of the sort of compute explosion that's happening. Not just GPUs, but the CPU compute explosion, which is where agents run. Agents don't run on a GPU. The agent runs on the CPU. It's all a distributed, like parted out compute, right? Right, right. Yes, you know, the CPU market on the cloud now just exploded. Actually, if you look at Muse, which is really interesting, and folks should play with it, Muse is essentially a personal agent from Meta, and that agent is running on CPU engines in the cloud. Each one is like a micro VM, right? And so that reduced instruction set for cloud-based computing ends up being really interesting because you get a, and you rarely get this as an infrastructure owner, a step change in cost just by migrating from an x86 to an ARM engine because of that 20% to 30% power efficiency. And so that's been a really interesting advantage for us to lean into, knowing that we can bring the software capabilities overlaid on that efficiency play.

Yeah. So what are the software capabilities that you lay over that efficiency? Like the chip efficiency. Yeah. So the good news is, again, we're doing the heavy lifting and working with the ecosystem. When I say the ecosystem, I mean working with PyTorch, right? Working with the providers of the core languages, Java, Rust, Python, et cetera. And so we actually participate and build in a lot of the tuning and efficiencies into those systems. Now, as we move up the stack, one of the things we're doing is, and we actually just launched this. It's called the AI Portal, so if you can go to the ARM developer website. We've actually pulled together all the various software tools. We have a whole list of optimized models, optimized runtimes, and since we recognize that developers, a lot of our target is now agents. Those are the developers that we have to serve, too. Not just serving humans, we're serving agents. We have to serve them a little differently, and we can talk about that.

we've exposed a lot of that knowledge base through an MCP server. And so as you're side-saddling with an agent and doing development and trying to get the most efficiency, highest performance, best application onto an ARM device, essentially 15 years of tuning and optimization from our code bases and knowledge bases and videos and learning paths, all can actually be available to your agent. Right. And your agent can read a lot more data and content than you can. And your agent is fine reading all the docs. The agent doesn't care how long the dots are, how well they're written or whatever it is. You did say that the agents interact differently. Do you mean they interact differently with the sort of intermediary software that touches the chips? So really it's around the way they interact with tools. And so we've just released a bunch of performance tools as well. So ARM finally released a performance suite for tuning ARM CPUs, so our own toolkit.

We have a whole graphics, neural graphics suite of SDKs for actually doing some really interesting things on mobile and graphics that you could only previously do on desktop. So this is like ray tracing on a mobile phone. Pretty unheard of stuff. And then the tooling around those. And the interesting thing for us as we were designing these software stacks was, of course, there's always going to be room for experts, human experts. And I think human in the loop is super important. But for agentic workflows as well, you really need to think about exposing everything open source so that the agent can go and validate things. And then also CLIs command lines And if you think of like bringing user design into play thinking about the way an agent interacts with software on the command line is pretty differently than the way a human interacts. An agent doesn't care if you've deconstructed your workflow into 300 different commands if it brings

clarity. Where a human you want to limit keystrokes and typing. The agent doesn't care. You don't want to go to the man page every day. It's a very different way to really make it much easier for agents to kind of learn and interact and validate what they're doing. And so we've taken pains to do that. Yeah. When you mentioned the ray tracing on the phones, I was like, oh, man, how hot is my phone going to get? Right? Because that's another of the limitations of a lot of these devices is that, you know, the heat that the compute generates is right there in the device and can affect things. You don't have room to do, like, big blocks of things. Yes. How are you thinking about, you know, mitigating those sort of things? Yeah, so this is a super pool. So, and this is what's really great, being at a company, a rich IP design company, now building more, you know, full hardware solutions and now building out our software capability. And so ARM's latest GPU for mobile includes a neural engine inside the GPU. Okay.

And so that allows the GPU to do things it couldn't do before. And so what we've done is created an SDK to take advantage of that so that you can do super sampling, up sampling, and ray tracing, for example, without the problems that you're describing. So this doesn't kill your battery. It doesn't make your phone a heater. That's right. and it allows, like I said, this sort of like desktop-level graphics capabilities that, you know, just a few years, and we actually released a game to prove it. Okay. So there's something called Neural Dawn. You can go and look at this. You can download it. It's in the Play Store. Yeah. And your premium phones are rolling out with this new hardware. So it's a hardware-software pairing, and so it is only capable on, you know, the phones that support this hardware. But, no, it's pretty impressive. Yeah. And like I said, this is just, we're just getting started, and so we expect to see a lot more of this enter the market. You mentioned the word neural in terms of chips and software. Could you explain what that is and how that works in terms of the hardware and the software?

Yeah. The sort of root of a machine learning of a model is matrix math. And so you're leaning into things that are called neural networks. So that's kind of the basis of a lot of modern AI is taking advantage of neural networks in a variety of ways. Yeah, the transformer model is that. Transformer models are an example of that, but only one example of that, right? And so if you look at world models, they're not transformer models. They're a combination of several different kinds of models. And vision models are not necessarily transformers. You have diffusion models, which are image and video. And so all of those kind of underneath is this idea of being able to navigate, traverse, train, and then run inference on neural networks. And so that is, at its core, matrix math, right, matrix multiplication. And so CPUs historically have not been good at that. It's just not the way they've been built. They're much more kind of sequential. And so GPUs in the cloud, NVIDIA led the way here, have increased capabilities there.

But as you kind of skinny them down and they get smaller and you stuff them into a mobile phone, even there they're sort of not fully optimized. They happen to be better than a CPU, but they're not fully optimized, especially in terms of power. and power utilization. And so you had this proliferation of things called NPUs, which allow a mobile phone to run your graphics. So you're watching your videos and you're playing with your phone. That's a neural. And then the GPU is doing your graphics, and then there's an NPU that allows you to do your machine learning. Right. So computational photography, running an LM, et cetera. NPUs are not standardized. So coding to an NPU on a Samsung is different than coding to a different phone, coding to a Xiaomi, coding to Apple. And so what we're leaning into is the GPU because there's a much more standardized model for developing machine learning for GPUs. But it's too expensive from a power perspective to just run your full model there.

And so what we've done is put a neural engine, which is a matrix math optimized piece of silicon that sits inside your GPU, right, again, alongside that GPU. Right. And so that unlocks the ability to do things like neural network-based ray tracing using typical APIs, like Vulkan and other APIs inside your phone. So, again, complex. You've got to go unwind that. But, again, we try to do most of the heavy lifting there. Right. And package that in an SDK so that someone can take full advantage of it without having to do all that hard unpacking. Yeah. Yeah, I know. I'm glad I asked that one. No, because, like, you know, I've heard that the NPU is the neural processing unit, and I've also heard the TPU, the tensor processing unit, where, you know, a lot of these things we're talking about, machine learning graphics are doing a lot of very complex maps, like calculating a lot of numbers all at once.

Does any of that change with agents, or is this just sort of like how that works? Yeah, agents, it's interesting. Agents end up being additive, right? And so it doesn't change an agent. And this has been important for folks to realize, an agent is just software, like traditional software. So it's running in Python, C++, Rust, what have you. and it is calling out to a model, right? And it's probably calling out to a large language model for deep thinking and it might be calling to other models for various processes, categorization or vision or sensor fusion. But suddenly the kind of workload that an agent triggers is lots of tool calling, right, lots of command line calling, and it's very chatty with the models. And so agent workloads just are a massively additive workload to whatever you are already doing. And it's increasing the sort of compute demands on your device. Because they can move faster and just spit out commands all day, right?

They can spit out commands all day. They can run all night. They don't mind brute forcing things and trial and error. They can definitely run up your token budget. Everybody's a script kitty now. Exactly. Part of the efficiency question I've heard is sort of scheduling efficiently on resources. And I think that applies to both the devices and the cloud that you're working on. Are you thinking about software that does that in better ways? Yeah, so when we look at taking these cloud techniques and bringing them to not just mobile, but also physical AI, this kind of explosion of devices and robotics, A lot of that is not just trying to translate what was running in the cloud and kind of shoehorn it into devices, but rethinking the sort of frameworks, rethinking the SDKs and the different software stacks. And on the physical AI space, so this is robots and security systems. We actually just started Arm Total Design for Physical AI as a consortium.

In a month we have about 80 industry participants So this is folks from AWS Teradyne et cetera And what we love to do is work with them to essentially create kind of a good housekeeping seal of approval style This stack across from these, you know, 20 different providers is a recommended, secured, validated performance stack for this kind of hardware, for this kind of use case. And so that's the process we're now going through. Yeah. To help folks understand and recognize that, realize that efficiency. Yeah, because it does feel like there's a kind of Wild West moment with the physical AI, right? It's the Wild West, for sure. A lot of people are talking about just the orchestration layers or consistent standards in accessing the software on there. Is there a hardware software bridge that Arm's thinking about there, or is this like somebody's got a... We're obsessing over this.

So we actually released something called Device Connect that is a open source. You can go to ARM developer, look for Device Connect. It's on GitHub. It allows you to programmatically link and connect different devices through an AI orchestration framework. And so you can take advantage of small devices that might have a sensor, and they can participate in an application interacting with another device that might be smarter, more capable, maybe move something around. So robot armies then. Hopefully not armies, but helpful robots. And actually Anthropic just released something called the Model Hardware Standard in the same kind of space. So lots of thinking around how can we actually help software take advantage and interact across device and lots of thinking around how we can do that safely and securely. And so I would say we got really excited about the innovation we were working through. And then most of the time was around security and safety. And so it's super important.

Yeah, yeah. And I think like, you know, sci-fi always gets there faster. There's a cheesy Tom Selleck movie from the 80s called Runaway where somebody takes over all the like vacuum cleaners. Right, right, right. I remember. Yeah. Yes, I remember. So what sort of security concerns happen with robotics that are different? I think the general data security and access security, so those are, and by the way, in general, I don't minimize them. They're super important. You need to obsess over those things. I think for your average developer out there or even your advanced developer, there's a whole new world if you're thinking about robotics or physical AI called functional safety. And so this is anyone who's been working in the automotive industry knows this really well. It's around for quite some time. Don't run over people. Right. Don't run over people. But also your software has to be deterministic. And so this is not just typical safety stuff. This is you even turn off the optimizations and compilers when you're creating functional safety software because optimizations are not necessarily reliably predictable.

Right. And so maybe if there's anything to take away from this talk is if you're thinking about robotics, you know, cross device systems, physical AI, go learn about what functional safety is. Because there's a whole slew of standards and a whole slew of things that you need to do. And they're really important because if we're going to have big, heavy, strong, fast devices flying, moving around, like we don't want to hurt people. Yeah. And so this is a super important space to understand. And making those deterministic where, you know, most of our AI things are not. That is, yes. That's the problem, isn't it? How do we make something deterministic that's inherently non-deterministic? And the ways you do that, again, you go do the research, you actually split capabilities. So sensing and thinking, thinking and deciding, you kind of segment and split those up. You really think through fallbacks, right? So this decision needs to return in 10 milliseconds reliably every single time. because 10 milliseconds might be what is required to avoid hitting something.

And so if your model can't return, you need to know it can or can't return within those 10 milliseconds, you need a fallback that actually can make a decision. Or you need to be able to say autopilot's off. So these are the things that we really have to take very seriously. And by the way, the industry is. Yeah. Oh, sure. But there's a lot of learning. There's a lot to do. I mean, you know, if there's any, you know, societal guardrail, it is the threat of lawsuits. Fair. Fair. we definitely take safety and security seriously yeah uh but i think as a company we really do think we need to be careful with the sort of alarmism right and so we're kind of really encouraging the industry to sort of take policing itself quite seriously uh and you know i think we are of the opinion of other countries are not going to wait uh they're not going to slow down and so this is an area i think again to developers at large if we all take this seriously and we all kind of come up to speed on this. I think we can continue to move at pace and we can build in the kind of safety

and security concerns that are very, very real. Yeah. But I don't think it's a sort of government legislation and a slowdown that's required. I actually think it's an investment in these spaces. And by the way, that comes, we're the ones writing the software. Right. We've got to invest in it. Managing the agents. Like, I think we, it's kind of on our power to take this stuff seriously. And so I think we can do it if we combine it. Yeah. And, I mean, you know, AI, in a lot of these sort of publicized breaches, it's just that AI is a goal-seeking machine, right? You give it a goal, and they'll figure out how to do it. Correct. For robotics, do you think the sort of separation of concerns, FUSA, is enough to, like, stop the, you know, the universal paperclips machine, right? I am a believer in the benefit that AI can provide. And, you know, we will have bumps, for sure. I think it is – but we've had these powerful technologies before, and we make mistakes along the way.

But I've always – this is one of the reasons I love the industry. I love developers. We're problem solvers, right? Inherently, that's what we do. And this is where humans have to take their own responsibility and why we're not going to cede all the decision-making to the robots. And so I think it's something we can get behind. And again, I'm pushing this because we're the developers. We're the ones kind of, we see the quality tests. We see the gates. We're the ones building these protections in. And so it's in our hands. I think we can do it. I hope people do, in fact, take responsibility. Well, I have been Ryan Donovan. I hope you've enjoyed the conversation. I'm the editor of the blog, host of the podcast here at Stack Overflow. So if you like what you heard, please leave a rating and review on your favorite podcast provider. And if you want to reach out to me directly, contact me directly, you can find me on LinkedIn. I am Alex Sinelli. I'm the Senior Vice President of AI and Developer Platforms at Arm. And, yeah, I am available on LinkedIn as well.

All right. Thank you for listening, everyone, and we'll talk to you next time. Thank you.

番組の概要欄(原文)

Ryan sits down with Alex Spinelli, Senior Vice President of AI and Developer Platforms at Arm, for a conversation about bringing ML capabilities to edge devices, adapting software tools to meet the unique demands of autonomous agents, and the critical importance of prioritizing functional safety as artificial intelligence transitions into the physical world of robotics.Episode notes:The new era of Arm is focused on maximizing performance and power efficiency for AI workloads across edge devices and cloud infrastructure.Connect with Alex Spinelli on LinkedIn. See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

出典・この記事について

番組
The Stack Overflow Podcast(The Stack Overflow Podcast)
元の回
Trying to get our Arms around AI efficiency
配信日
要約作成日
公式サイト
art19.com/shows/the-stack-overflow-podcast

この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について

X でシェア

関連エピソード