
未来はPostgresにある(Changelog Interviews #688)
The future is Postgres (Changelog Interviews #688)

未来はPostgresにある(Changelog Interviews #688)
Changelog Master Feed
要約
Snowflakeで Postgres 部門を率いるCraig Kerstiensが、Postgresの「退屈なコア」と拡張機能エコシステムの強みを語る。DuckDBとIcebergをPostgresに持ち込むpg_lake、ETLの苦痛、Postgres 19の注目機能、AI生成PRがコミッターに与える負荷や脆弱性開示の難しさまで幅広く議論した。
この回のポイント
- ●Craigは、Postgresが安定したコアと豊富な拡張機能で成り立ち、拡張を通じて実質的にデータプラットフォームやランタイムのように使えると述べている。
- ●pg_lakeは約12の拡張の集合で、PostgresにIcebergカタログと組み込みDuckDBを載せ、S3上のParquetを通常のSQLで作成・照会できる。
- ●OLTPとOLAPの分断やETLは大きな痛みの源で、同僚の「ETLのEはevilのE」という冗談が紹介された。
- ●Postgres 19ではREPACK CONCURRENTLY、クエリプランの固定、GROUP BY ALL、グラフプロパティ対応などが目玉だとCraigは評価している。
- ●コミッターは約50人と少なく、AIが生成した低品質なPRの流入が負担になっている。Cで書かれ学習データが少ない点も課題とされた。
- ●セキュリティ脆弱性は、クラウド事業者と連携した責任ある開示とパッチ提供が重要だと語られた。
章立て
Postgres人気とその理由
毎年「Postgresの年」と言われる状況や、Herokuで早くから採用した経緯、人気の背景を振り返る。
拡張機能とpg_lake
Postgresのコアを退屈に保ち拡張で広げる思想と、IcebergカタログとDuckDBを組み込んだpg_lakeの構造を紹介。
OLTP/OLAP、ETLの痛みとDuckDB・Iceberg
行指向と列指向の違い、ETLの負荷とコスト、DuckDB・Parquet・Icebergの役割を解説する。
Postgres 19の注目機能
REPACK CONCURRENTLY、プラン固定、グラフ対応、GROUP BY ALL、UPSERTの改善などを紹介。
AIとオープンソースの保守負担
AI生成PRによるコミッターの負担、DBAの不足、CからRustへの移行議論などを話す。
テスト体制と脆弱性の責任ある開示
ビルドファームやコミットフェスト、Snowflakeの検証、ゼロデイ対応と公開の緊張関係を議論。
Snowflakeの戦略と今後
データベースミラーリングやpg_lakeの活用例、Postgres学習・貢献の方法を語る。
Supabaseなど新世代とまとめ
Postgres上に構築する新興企業を評価する観点と、Postgresへの期待で締めくくる。
解説記事
Snowflakeで Postgres の取り組みを率いるCraig Kerstiensが、Changelogのインタビューに登場した。Heroku、Citus Data、Microsoft、Crunchy Dataを経て Postgres に関わり続けてきた人物が、拡張機能、pg_lake、次期 Postgres 19、そしてAI時代のオープンソース運営の課題について語った回である。
「退屈なコア」と拡張機能の力
Craigは、毎年のように「今年こそPostgresの年」と書く記者がいると笑いつつ、Herokuで採用した当時は誰も賭けようとしなかったと振り返る。Postgresは課題管理ツールもGitHub上のPRもなく、メーリングリストにパッチを送る運営でありながら、30年以上改善を続けている。その理由は自分にも再現方法が分からないと述べている。
人気の鍵として挙がったのが拡張機能だ。BM25全文検索(ParadeDBやTimescaleのpg_textsearch)やPostGISなどがあり、コアは堅実に保ちつつ周辺で機能が広がる。聞き手は、これをOSやランタイムのようなものだと表現し、Craigも「データプラットフォームだ」と同意した。
pg_lake:PostgresにIcebergとDuckDBを入れる
Craigが「芸術作品」と呼ぶpg_lakeは、約12の拡張から成る総合拡張である。出発点はPostgres内にIcebergカタログを置くことで、その上でクエリするためにDuckDBを組み込み、SQL構文や関数の変換、GeoParquet対応、拡張間のバージョン管理までそれぞれ拡張で解決した。
結果として、create table ... using iceberg と書くだけでデータがS3上のParquetとして保存され、通常通りinsertやselectができる。クエリ時はDuckDBが高速な分析を担い、SparkやSnowflakeなどIcebergを読めるエンジンからも参照できる。Craigは、Pythonスクリプトを別マシンで動かす代わりにpg_cronで済ませる例も挙げた。実際に、Athenaなどで3か月かけたパイプラインをpg_lakeに置き換えた人の話も紹介されている。
背景として、行指向のOLTPと列指向のOLAPの分断がある。両チームは別世界で、ETLは本番DBに負荷をかけ、遅延やコスト、障害の原因になる。同僚の「ETLのEはevilのE」という冗談を引き、Craigはその苦痛を強調した。DuckDBは分析版SQLite、ParquetはCSVの列指向版、Icebergはその上に更新や削除を重ねるメタデータ層という説明だった。
Postgres 19の注目点
CraigはPostgres 19を「JSONB以来、約10年で最大級」と評価する。目玉は次のとおり。
- REPACK CONCURRENTLY:pg_repackやpg_squeezeといった外部拡張が担っていたテーブルの肥大(ブロート)解消を、ロックなしでコアに取り込む。
- クエリプランの固定:Oracleのヒントに相当する要望に対し、プランナーを回避するのではなく、特定クエリのプランを固定して突然のプラン変更による障害を防げる。
- グラフプロパティ(SQL標準に沿ったもの)、GROUP BY ALL、UPSERTで既存行を返せる改善なども挙げられた。
また、企業がOracleやSQL Serverのライセンスから離れ、Postgresに賭ける流れも指摘している。
AI時代のオープンソースと保守負担
議論はAIの影響に移る。Craigによれば、コミッターは約50人で、AI生成の低品質なPRが押し寄せている。フロントエンドならモデルで形になるが、Cは公開コードが少なく、データ破損のリスクもあるため事情が異なるという。DBAの数が増えない一方で開発者は急増しており、データベース知識のギャップが広がるとも述べた。
セキュリティでは、モデルが数十年残っていたバグ(PostGIS作者が見つけた15年前の小さなメモリリークなど)を見つけている例が紹介された。それでもPostgresは多くの企業や政府が使い、監視の目が多いため安心だというのがCraigの見方だ。CからRustへの書き換えはメーリングリストで定期的に話題になるが、現実的ではないとした。
脆弱性対応では、過去に深刻なゼロデイがあったことに触れ、修正のリリースをクラウド事業者が即日展開できるよう調整する責任ある開示の重要性を語った。Snowflakeも拡張の導入前にAIを使った自動スキャンを行い、メンテナに連絡し、修正パッチ付きで報告する方針だという。
Snowflakeの狙いと関わり方
Snowflakeにとってpg_lakeは土台であり、最終形はPostgresとSnowflakeを双方向にリアルタイムで複製するデータベースミラーリングだとCraigは説明する。スマートTV事業者が視聴データをPostgresに取り込み、Snowflakeで分析した広告の判断をPostgresに戻して低遅延で配信する例も紹介された。
関わり方としては、pg_lakeのGitHubで試す、pgsql-hackersメーリングリストで開発を学ぶ、pgsql-usersで質問に答える、コミットフェストでパッチをレビューしテストするといった方法が挙げられた。Postgres 19のベータをテストすることが、最も歓迎される貢献だという。
まとめ
今回の対話から見えるのは、Postgresが「単一のデータベース」から、分析やAI用途を取り込む拡張可能な基盤へと広がっている姿だ。日本のエンジニアにとっては、専用システムを増やさず運用を一本化する選択肢としてpg_lakeが注目に値する。Postgres 19の機能は運用負荷の低減に直結しそうだ。一方で、少数のコミッターに依存する体制やAI生成コードの流入、脆弱性開示の調整は、オープンソースを使う企業にとっても他人事ではない。使うだけでなく、テストや報告で貢献する姿勢が問われている。
文字起こし(英語・自動生成)
What's up, friends? Welcome back this week on The Change. Welcome to my good friend Craig Kirsteins. Craig leads the Postgres efforts at Snowflake, and he previously helped build data platforms at Heroku, Citrus Data, Microsoft, and Crunchy Data. Of course, we're back for the year of Postgres and why it might finally be true. Craig bet on Postgres back at Heroku when nobody else would, and he's been geeking out of it ever since. We discuss PG Lake, the open source extension that puts an iceberg catalog and embedded DuckDB inside Postgres so you can create parquet files in S3. So cool. We get into why Postgres 19 might be the coolest release in a decade with repack concurrently, query plan pinning, and group by all. And we also talk about the small group of roughly 50 committers who keep it all running and how a ton of agent-generated PRs are landing on their doorstep. A massive thank you to my friends and partners at Fly.io.
Computers for agents. Sandboxes, y'all, are not enough. Give your agent a real computer and get back to building. Learn more at Fly.io. Okay, let's do this. This episode is brought to you by our friends at Coder.com. You know what? Your developers, they have guardrails. The humans, guardrails, right? That makes sense. But your AI does not. How often are you running your AI dash dash dangerously? I think that most people are living a little dangerous. Some are sandboxed. Some are not. Most are not. And I'm here with Nicky Pike, Field CTO for Coder. So, Nicky, what is the focus right now for Coder? I mean, really, our focus and what we're doing at Coder is on the governance around AI.
So, you know, everyone right now, they're talking about, you know, AI agents like they're magic. But nobody's really asking where they run and what they have access to or what happens when one gets compromised. I mean, kind of give an analogy. You're not going to invite a contractor into your home and then leave and have them have full access to your safe and everything else on day one. You know, this is what people are doing with agents. We are wanting to, we are the infrastructure that helps you put guardrails and restrictions and governance around agents and what they're doing to you today. That's where Coder succeeds because, you know, we started out as a CDE company. There's no secret. We were a CDE company. We created remote dev environments, standardized workspaces. But what we realized is that we accidentally built the exact foundation for the AI age before the AI age even really arrived. So isolated, governed, audible compute environments, that's what we build. And that's exactly what enterprises need when their workforce starts, including agents alongside their humans. So, you know, for us, this wasn't a pivot. This was really we finally found the problem that perfectly fit with what we'd already built.
So if you're living dangerously, dash, dash, dangerously, non-sandboxed, you name it, Coder has AI governance for you, a centralized gateway to observe and to control LLM tool usage across your environments, authenticate model requests, audit every prompt, and contain agents behind a default deny allow list the smart way. Learn more at Coder.com. Request a demo once more. Coder.com. Well, friends, we are back with an old friend and a good friend. And I've missed you, Craig. So it's just been way too long. I think Postgres is more, is it always just a little more at the epicenter of what we're all thinking about?
I feel like it's always the boring but provocative topic for us. What do you think? Yeah, yeah. No, I think that's like spot on. There's one reporter I talk to that every year comes back to me. He's like, Craig, I'm sorry. I'm doing it again. and writes this article that it's like, this is the year of Postgres. And it's been like the year of Postgres for like five years in a row. It's like, all right, the same for Linux though, right? And I feel like it's finally, they're sort of both incrementally becoming the year of. But it's funny for me, having like been here for so long of like, you know, we picked Postgres back at Heroku long before it was cool. In fact, like no one else wanted to bet on Postgres and everyone was like, why on earth are you doing this, right? And it wasn't cool then. And it's like, you know, I have no misconceptions of what I do for work. I'm a nerd, right? I run databases for a living.
And so, like, you know, when my kids take me to school, I'm not going to win the, like, coolest, like, you know, hey, let's talk about your parents' jobs. Like, I'm not. You know, I don't know about that these days, though, you know. I think now the nerd dads are the cool dads. Maybe when we get to high school. I mean, for elementary school, I don't know that you're beating out the, But it's like Postgres became sexy. Can I say a database is sexy? I mean, you can if you want. It is America. But it's like we're here now. I mean, it's been fun to see this because investing in it 15 years ago, who else was there, right? And actually to have a small hand in it getting to where it is is exciting and fulfilling in that journey. And I think, like, Postgres itself deserves, in a way, even more credit. I think now that it's gotten so popular, so many people are latching onto it. And, like, I want some of that Postgres coolness. And yet, Postgres itself is just so freaking solid and good.
And, like, you know, to be fair, though, as devs, I was at a conference a few years ago. And we were at the speaker dinner. And, like, as engineers, we try to pretend we're so principled. And they were like, you know, as a Python developer, like, obviously I'm going to pick Python. I would never use Ruby. Who on earth would use Ruby? And yet they're basically the same language. Like, I know I'm going to have pitchforks coming for me. Yes, there are differences. I'm with you on that because I'm just immediately that person. So don't don't don't feel bad. I'm sure I'm sure there's going to be some pitchforks. I'm like, I am stopping this episode right now. He said Python and Ruby were the same. This guy knows nothing about technology. Matt's listening and just yelling in Japanese right now. And yet, like, you know, and it was funny because they went on and they're like, for example, Postgres for my sequel, first my sequel.
And this was 10 years ago. They're like, because I'm a good Django developer, of course I pick Postgres. Well, I mean, I'm a real engineer. I make good decisions. Why would I ever pick my sequel? And yet they're like. I care about my future. In reality, they were like, you know what? I actually don't know why I'd pick one versus the other. And it was really funny because the table got quiet because half the table knew me. I'm like, oh, because of concurrent index creation, transactional GDL, listen, notify, JSONB. They're like, oh, that's really cool. I should go learn about some of this stuff. And I think it's awesome to see that now the world is embracing everything that is in there. And we're learning. But also I think there's still a little bit of sometimes that, you know, again, I'm probably going to get pitchforks. So, like, you know, we're bandwagon fans. We think Postgres because it's the age of Postgres. And we don't always peel back the onion of what's so great about it. Well, sometimes story plays a role in that. Is that critical to you at all?
Like Oracle, MySQL, you know, Oracle Database, MariaDB, the whole division there to sort of, like, I guess set the ground at unease back in the day? Like did that, I mean you've been in the game for a while so was that a role for you? I'm sure, I mean there's no way it didn't help right? I think part of it was like it kind of became the default for Ruby developers on Heroku, right? Like that was a time, you know, very influential. Like rest in peace. Distribution of ideas, man. Yeah, definitely rest in peace Heroku. and like cool great and then it started to become cool with those features right and we back at Heroku we invested and wrote some checks for core development features right and hired those folks way way back and I continued to do it at Citus and Microsoft and as part of Crunchy Data and now Snowflag that it's like cool like want to contribute and make the core better but definitely you know
Oracle with MySQL and MariaDB definitely helped, but I'm often asked, okay, well, if I wanted to create a company and recreate what Postgres is, how do I do that? How is Postgres what it is? I'm like, I wish I knew. You don't, right? There's not an issue tracker for it. There's no, things don't get pushed up to GitHub with PRs. They mail patches to a mailing list. Do you know how to generate a patch this into a mailing list? I'm not sure I do. Right? I don't think I do know how to do that. And yet, it just continues on. Like, eventually, they mirrored the repo to GitHub so people could see it, right? So they didn't have to go pull it up. But, it just works, and it's the test of time. I wish I could say how you could repeat this, because by all accounts, this should not work. And yet, it really just keeps
getting better. And there's huge kudos to Like, you know, all of the core developers that believe in, they believe in Postgres more than they believe in most things, right? And want to see kind of the, see Postgres continue to live on. And it's over 30 years old. What's 30 years old besides Linux that we're talking about? Well, I think even when I've read your stuff over the years, the one thing that I keep hearing you say is extensions. keeping the core boring, keeping it predictable, keeping a core contract in place secure, fast, things like that. Things that really make it, like you said, sexy, the unsexy side of it, which I think is actually sexy to me. Those are features, right? Those are curves. And so I think about software like that. So you have this core, and then they call that user land. It's extensions, and then anyone can kind of make Postgres, their own version of Postgres, maybe even Postgres compatible. Okay, but whatever, right? So, like, what do you, do you not believe, or maybe you do believe, since you said it a few times,
that extensions is really where it is allowed to thrive because it doesn't force the core function similar to the Go language, to keep it simple, right? To keep the language simple, let things happen in user land that need to happen in user land, and let core be core and boring, in a way. yeah I mean yeah I love extensions I think they're the next what we should come back to in a few minutes probably is Postgres 19 itself I don't want to diminish the core and like man if there was ever a release that is just like wow to me it is Postgres 19 I think is the coolest release that we've had in maybe 10 years but to your point yeah extensions are absolutely amazing. And I think you look at people that, you know, there's now multiple different text search extensions, right? So that, you know, BM25 kind of text search can be in Postgres, right? This is kind of the core of what Elasticsearch does and makes it so great. You know, ParadeDB was the first one there. There's PG TextSearch, which is,
you know, fully open source from Timescale, which adds that capability, right? There is, PostGIS. On Accent. The list goes on, right? Yeah, PostGIS is massive. It turns it into this whole other... PostGIS is a fascinating one. Everyone's like, that core team has to talk to the other core team. They do, but they're also running on this parallel path. What you do for geospatial is so different than the rest of core Postgres. It's this massive extension. We open-sourced PG Lake back in November, which, oh man, to me, it's like a work of art of like, it's actually like 12 extensions in one. So PG Lake is like the overarching extension. But like we started with putting an iceberg catalog inside Postgres and I'm like, hang on. That's like a open file format, parquet files, like that's column based.
What's that? And it's talking to S3. What's that got to do with Postgres? and it's like, well, it's just a catalog. But once we have the catalog there, okay, great. Well, we need something to query this, so we embedded DuckDB inside Postgres. Get out of here. Yeah, it's turtles all the way down, right? And so, but hang on, DuckDB has a different SQL syntax, so we have an extension that converts from Postgres SQL syntax to DuckDB. But hang on, DuckDB has different functions, so we have an extension that does that. Well, what about the Geo Parquet files, right? How do we convert from GeoParte to, like, GeoData types in Postgres? So we have an extension for that. And so... I'm bothered over here. Now, we've got ten extensions that run with different versions. And what happens when you bump one version that doesn't work with another? So we have an extension... A cascade of change? We have an extension that manages the versions of extensions, right? Now, you're like, oh, great. I'm glad you love to create all this complexity. But the result is, like, I can literally now just go and query a parquet file and load it from, like, S3 into Postgres.
So I don't need the traditional, like, great, I don't have this Python script running on a box. It's literally just running with pgcron inside my database. And for, like, data engineering, they've had these sort of tools for a while. But, like, inside your transactional Postgres database, no. Like, it's always this, like, duct tape and glue. And, like, I don't even have to load it. I can actually just, there's like elegant pipelines that people are setting up now that like, great, just drop off this stuff in my data lake in this S3 bucket. And I just want to query the new files as they are there. Just shows up. Which is like, huh, I never thought about doing this before. And it's like, not today. What happens if you bring like DuckDV and Parquet into the world of Postgres? Like what changes there? And can you maybe, just for the uninitiated, kind of give a version of what DuckDV and Parquet do, like why those are queryable fast? Yeah, yeah, yeah. Let me back up, right? So, like, we live in this world, like, this 30-year-old divide of, like, OLTP versus OLAP, right?
And that's, like, database academic stuff, like online transactional processing, online analytical processing. The reality is like, you know, OLTP, operational databases, are row-based stores, right? They store a whole row together. Whereas analytical systems are typically column-based. And so what you do is you store a whole column so that you can summarize that column as the entire, you're not getting the whole row most of the time. It's like, how many, you know, how many sales did I have in this book from Amazon yesterday, right? Well, I don't need to know across all books. I just need to know, you know, some, all of this book, right? that's very different than I mean all the order details for all the books, right? For sure, yeah. So, you know, LLTP, row base, often single row inserts and updates and lookups, right? Like I log into Amazon and place my order. That's the order going into Amazon, right? And, you know, really fast kind of inserts, heavy indexing on them, right?
Like good old V-tree indexes in there or GIM indexes, like advanced indexing. But you want the query plan to be incredibly fast and, like, leverage indexes and then get your data back in, you know, a single millisecond, right? Like, this is just what Postgres does. Yeah. Completely different from the OLAP world, right? Which is, I don't mean anything in one millisecond. I want to know this report from Amazon.com, how many orders were placed yesterday, right? Like, I'm okay with taking that a few minutes. It's a lot of, you know, how you organize the data. Data gets batch ingested. It gets split into partitions. Like, this is the world that Snowflake lives in, and all these data warehouses, right? And they've lived in this world for a long time. These worlds are like Mars and Venus. Like, I've seen these teams, like, I've almost never seen these two different teams in the same meeting. I think they hate each other, right? Because one's running, like, the production operational system.
Now, the other one's just trying to do their job and give the analytics insights and drive the business and make decisions. But, man, they're kind of that. One of my colleagues joked that he's pretty sure that E in ETL stands for evil. Oh, gosh. Which I loved it because it's a necessary evil. We haven't gotten away from that. Help me unpack that one. Why evil? Give me more details on that. It's a source of – so when you go and scan all the data from a row based store, right, you're exacting load on it. And you're doing this batch pool, which it's not ideal for. And so you're often creating unnecessary load and oftentimes outages, right? So while you're doing that, your wall's backing up on Postgres. You've got latency in getting the data into your analytics system. It's costly. Moving data from one system to the other is a huge source of pain. I usually start talking to people about this, and they almost start crying. Yeah, I've been dealing with this for so long. but you can't be like, nope, I'm not going to move my data between these two systems because it's painful.
You can't just be like, I protest. Can you just throw more machines at it or more of more to it and sort of crush some of that or more compute? I guess it's more cost, right? More cost. Compute is so expensive these days, man. Compute not so much, but disks and RAM. RAM's climbing. I mean, all of it's climbing. RAM is three times the price that it was a year ago. I mean, that's just consumer level that I've got. I'm not purchase powering a data center, so I'm purchase powering a dev lab. So even in my mileage, the costs have risen three times. I mean, if you want to buy four terabytes of NVMe storage this moment, one single one, you're going to bust out $1,000 at least, maybe more, for a decent NVMe drive that's worth the IOPS and worth the speed. And yeah, you can throw more at it. You can keep throwing more at it and keep throwing more.
But it's like, you know, this is the pain we've lived with for a while. And so kind of getting back to the, like, okay, hang on. What's DuckDB, right? The most simple parallel I can give, right, is DuckDB is like SQLite, but for analytics, right? So you've got SQLite as your lightweight embedded transactional database, right? It's great for local stuff, right? I want to ship a small database around. I want to embed something. But SQLite, most people aren't saying, like, that's my production Postgres replacement, right? Postgres is a server process. You've got hundreds of your web servers connecting to it, high concurrency, massive scale, right? And so you've got DuckDB, which is closer to SQLite, which is great for my desktop analytics. Like, I'm just me, I want to do some stuff, right? More powerful than Excel. I like SQL, it's really good, and it's an embedded analytical database. So that's what it's good at. So it knows how to speak all of these columnar formats, right?
And so Partey is an open standard, just, you know, Partey file. Iceberg is metadata on, like, you know, think of if I copied out, you know, a million records of purchases from my database to a flat file, right? Like think of parquet as a CSV file, but it's designed for columnar analytics. Great. But what happens when I've canceled my Amazon order? Well, uh-oh, I've got to go write a new file now, right? And so where Iceberg comes in, which is kind of the new cool hot thing in the data analytics space, open file format, right? So it says, oh, well, here's this parquet file, but here's this update set, and here's this deletion set, right? So you can stack these together. So now I've got, like, the Postgres analogy is like, and this is the poor analogy, but it's like, oh, well, if I've got, like, CSV and then change sets of CSVs and change sets of CSVs, well, sort of now I've got a database that's live, right?
Yeah. Yeah, like some version of history, right? It's a version of change, as you just said there. But, yeah, that's interesting. Okay, so I haven't heard of Iceberg yet. Oh, man, so I got someone you should talk to next, right after me, for, like, deep dive on Iceberg and what it means for this whole world, right? And so, flat files, then, these Iceberg, they're just essentially two files. So Iceberg is the, so Iceberg is a catalog with metadata about the Iceberg, about the Parquet files, right? That basically turns it into, like, great, now someone can just go and connect to a iceberg catalog and know how to query it. All these data warehouse engines. Interesting. So open source file format that in the data warehousing world and data analytics world, you can go and query. So, okay, what's Postgres got to do with this? Why do we have DuckDB? Well, now Postgres can go and create these iceberg files. So now I can go inside Postgres and go create table, orders, and all I have to say is using iceberg.
And now it's automatic going to store the data in these parquet files in S3. You know, I can insert into it like normal. I can query it like normal. When I query it like normal, for PG Lake, it's going to use DuckDV to query it, which is going to be fast analytics, right? Now I can also go and connect this to things like, you know, Spark and query it directly from there, from Snowflake, from whatever things that can speak Iceberg, which most analytics products can. so now I just like blurred this line to wait, Postgres understands analytics can write out analytics and I have this gap, right? This like bridge now over this gap of like, well these are disconnected systems well now they can be shared, right? And now I have this, for me PG Lake is just like you know, the next question is like okay, that's really cool engineering but when do I use this, right? I'm just like scratching my head like I've never thought about this stuff before what are my options now? and this is just like one extension but like for me T.G. Lake is this like
Swiss Army knife I'm like yeah I've used the pocket knife in my Swiss Army knife I've kind of used the screwdriver like I'm curious have you ever used the corkscrew to open like when you're out camping are you just packing like two bottles of wine and like now I need my Swiss Army knife to open almost never the corkscrew and the toothpick are the two things in the Swiss Army knife that I'm like who's reusing that toothpick I would be reusing that toothpick It's metal Yeah it's metal I thought it was plastic I would assume it's metal The little tweezers there are metal Okay I don't even have that particular knife I was just following you assuming it was metal Because most knives or most Swiss army knives Are made of metal Well I mean everyone I've ever had All the other parts are So you've seen a weird part That you're just not sure of And how you'd ever use them but then you kind of find a useful one. But then there's also the obvious ones, right? Right, with the knife. You never know when you're out camping and there is someone with a bottle of wine
and if you have the Swiss Army. You're not carrying a corkscrew, right? But I've got it in there. One second, I have got a Swiss Army knife. You've got wine. We prepare ourselves. This is a camping trip success right here. Man, I'm looking forward to our camp out this holiday weekend. Right. So that's just one expansion, right? which to me is awesome if you're like, hang on, what is Postgres? Postgres has no business doing anything. That's a whole ecosystem world on a database is what you're saying. You can explore PGLake for a while. You can commercialize for the next year. Maybe even the next several parts of your years as part of your career. Especially as data becomes more and more centerpiece for everyone, really. And you didn't have to really go learn a new database or system or like it's still just Postgres. It's actually a Postgres extension post GIS, like geospatial or time series or text search.
Like take, like, you know, what about vector storage, right? Like, man, that world of extensions. And I think for me, it's also interesting because it is open source. You're not locked in, right? When someone's like, oh, I'm Postgres. And like, prove it to me, right? Like show me that you're an extension or you're actually contributing to Postgres versus like I'd you know I speak the wire protocol okay that's great until I need logical replication well well but do you really like yeah I do right like this is Postgres and I think it's for me from a philosophy perspective like one it exciting frontier but I think it also from a philosophy perspective It making the community better Like it giving back It better together how we win It's almost like with extensions, Postgres is its own version of an OS, like a runtime. Not quite an OS and not quite not a runtime. You can almost do, if you can build an extension for it,
you can do it on Postgres. And because it's open source and because of the way it's ran, that is a very interesting superpower because you can sort of like center your entire fleet, your entire team on one database versus many databases and bring those databases like Parquet Files or DuckDB or these unique formats that have very specialized places, but not necessarily you wouldn't think of them as a Postgres substrate, so to speak. But now you can because you have extensions and you can do those things. It's a really interesting power that it has. I think it's a – I've never quite – I've often said it's like a data platform, right? Postgres is not just a boring relational database. It's a platform. I think thinking about it is almost like an OS, like you're saying, like a runtime, might be actually a beautiful way of articulating it. Well, I mean, because just in that case there alone with PG Lake, you can expose yourself to something that you'd have to build a sidecar to, not an extension or something that was like a siloed database, which is the worst scenario to be in in any scenario where you're a developer.
where you're like, no, no, no, I kind of want all my data. Most of you don't want to play. Obviously, I want backups and different stuff like that. But operationally, I don't want to have to think about seven APIs or seven credentials that get rolled or who has that blessing or that grant. And you just get to, you know, you get all sorts of issues in that scenario where if you can roll in through PG Lake, having DuckDB and Parquet is that. And then S3, I mean, if you can, that's just amazing. Like that, I'm going to have to play with that. Yeah, like if you can do like 80% or 90% as good as a separate special purpose thing, but you don't have to operate it and run it and maintain it, like it's a reduction of costs. But I think more importantly, it's a reduction of pain, right? Like monitoring, you know, one backup, you know, one runbook, one fail time. I mean, one of a lot, but I'm sure you have an N plus in there somewhere, but not having to learn five things or three things or two things to get a job done.
You're just doing one. Yeah. I mean, I've seen some, it was funny. I was talking with one person when I showed them PG Lake when we announced it. And they're like, we were about to go live on this whole kind of pipeline using, I think it was Athena and some other things. They've been working on it for three months. They tested it. It was working well enough. They were planning to go to production in the next few weeks. They were like, man, I'm going to throw all of that out and use PG Lake. This is so much simpler. This is great. I just felt so bad because they've been this massive project of three months of engineering. I'm like, I'm sorry we didn't talk to you sooner, that we could have planned for this. And they're like, don't feel bad at all. I was so worried about that thing going to production and how I would operate it that I'm happy to delete code. They didn't care at all, but they spent three months that are just going to throw this out. I think there's a beauty in the simplicity of it, right? There's a lot of advanced engineering on it that's really, really cool,
but how you work with it, it's just standard old SQL. It's pretty straightforward. It's pretty, like, my favorite type of engineering is when you take all that advanced stuff, you do the hard stuff, but the end-user experience, I mean, this is back to the Heroku days, right? Get Push Heroku Master was beautiful. And I still haven't, I don't think people have recreated that sense, that simplicity, right? And so when you take that advanced stuff and make it so simple, right, which is my entire goal of what we're doing with Postgres. Yeah, something magical happened there. I mean, it was a different world, too. No one could run in the cloud anything. I mean, even to launch a Ruby app, to deploy something was the, well, I can run that here. And that was also the day when we began to do less of FTPing and copying to actually, you know, true CI, true CD, which is a whole different kind of thing there.
Well, I know you're on the tip, and you mentioned Postgres 19, which you recently penned a post on the Snuffly blog about. You can probably go there if you want to. I'm really interested in this extensions world just because there's so much you can do there, but I feel like we've obviously mined that quite a bit with PG Lake and whatnot. But certainly lots of cool stuff happening in these extensions world. I think the point there was really just like, this is what makes Postgres be so attractive to folks because you can build something like PG Lake or use something like PG Lake on top of Postgres. And it's a whole entire ecosystem within an ecosystem and then a runtime idea. Like that to me is what's always been attractive. with Postgres for me, and it seems like we haven't destroyed Postgres along the way. It's still thriving. Yeah. It runs well. There's lots of scenarios where production is challenging, and I'm sure you know that quite well with your background. You can allude to that, but what's a good next space, Postgres 19 or something else?
What are you thinking? Yeah, Postgres 19. I mean, we kind of transition easily from extensions to that. Like, you know, I read this post. I'd love for people to check it out. But it was funny because I thought it was too long as I was talking about all the features. And as soon as I published it, there were several folks that were like, you missed my favorite thing in the release. I'm like, guys, this is like a, this is a small research paper right here. Like, this is, you know, this was long. And they're like, oh, but you missed this one. Like, normally every post-pressure release is like, all right, performance got better. And we added this little kind of nicety and that sort of thing. And I have to go back to probably Postgres 12 when we added JSON support and we cheated. And everyone's like, oh, like, die no sequel. But like Postgres 12, we validated the text on JSON. No, no, sorry, not 12. 9.4. 9.4.
No, now I'm going back. Too far almost. 9.4 was JSONB, 9.2 was JSON. Okay. So that was almost 14 years ago. 2010, yeah. 14 years ago, we got JSON in Postgres. I have a secret, we cheated. Okay. We validated, it was valid JSON and threw it into a text field. And every headline that came out that year is like, Postgres is now a document database. And I'm like, well, I like the headline, but it's like, no, this is not like actual JSON compressed. There's no indexing for it. But like, man, the headlines are great, right? Like, hey, long-lived Postgres, it's now a document database. Two years later, we got JSON-B. I love my colleague Will describes it as like the B stands for better. It's, you know, a binary representation of JSON on disk. you can have like gen indexes so like man this is probably the biggest release in 10 years since
jsonb and one of the to me the flagship things is is repack and so postgres under the covers is a giant like a pendling log right and so you delete a record it doesn't delete the record and free up the space it just marks it as 30 and it comes back in and you know hey you write a new record can overwrite that, right? But that's where you have bloat. Like, you'd leave 100 records and new things are coming in. Well, I've got this 100 records of unused space, which is bloat, that's not going to get reclaimed immediately, right? Until you hold a lock and run a vacuum on your database. And so there were a couple of extensions for years, PG Repack and PG Squeeze, that would go in, like, under the covers, rewrite these tables, free up the vacuum. It's an online operation, but you need extra disk because it's doing all this magic. They're, like, somewhat risky to run. Like, this is definitely expert mode, right, of, like, all right,
I need to run repack because I have high bloat, but I don't want to take a lock and pause writes and run vacuum. And so, like, this is hardcore production databases, advanced, you know, management of them. But, like, repack just, like, all right, we're adding it to core and we're adding repack concurrently. So I can not hold a lock on that, and I can run in the background. And now this is just like going from advanced power user to, I wouldn't say the like, you know, 101 level stuff, but much more like 201, 301. Now I don't even need this extension that like was out here in the ecosystem, right? So we've taken that from broader ecosystem to in the core. You don't have to go grab the extension. It's just there. which man if you ever use PG Repack or PG Squeeze or dealt with kind of high bloat and tweaking vacuum like boom your life is just bad and this is one of those things it's like when you're having a bad day you're having a really bad like this is back to yeah I know what's going on
with my day we're not into you know cattle we're back to pets right like when we named our servers like yeah I know what's going on with that database like it's not some random UUID, like, I know those first characters in that UUID of that database to say, oh, this is the problem one, this is production, this is what we care about. If you're dealing with that, like, you're in a good territory, because you got some good data, and you got a good business going on underneath it, right? But, like, repack something into Core and repack concurrently? Like, huge. Like, that's, like, one of five things, right? Like, the thing I got called out for so much was, like, missing hints in Postgres. And so Oracle is well known for this. You can say, I want you to use this query plan. Postgres for the longest time has been like, no, the planner is probably better than you. Trust the planner. Trust the planner. It's not done. Now, what's fascinating is usually that's correct, except for in this weird case where some new data distribution changed.
So Postgres under Coverage is doing so much magical stuff. It records the number of records in the table, and it has heuristic on the distribution of that. So should it use an index or should it scan the entire table? You've got five types of indexes from Btree to Gen to Gist to Space Partition Gist to Brin indexes, and it knows when to use this one, when to use that one. Right. And so when you issue a query, it's creating this plan of here's how I go and access all the data. And it's usually really good at that. Now, as the data distribution changes and maybe like, oh, I need to take a new plan now. Well, if you've got a really mature production database, you may say, like, you know what, I want you to keep using this plan forever. Don't change. Don't surprise me. Right. Because there was a slight change in the stats or some new like spike of things. and you think all the data's changed and it hasn't, right? So now you can basically not go around the planner and manually say,
no, no, no, I want to tell you what plan to use. But like, no, I want to pin this plan to this query. Don't change in the future and surprise me. And on a large-scale production system, like often when that plan changes out from under you, you're surprised, you're firefighting, you've had a bad day. like one of those things that you know just keeps getting better as an operator of databases like it took me a long time to come to terms with like running products usually if you go ask customers if they have questions or feedback like they're like I don't like the color blue you use the roundedness on this button is terrible. Why on earth would you use that font? I was like, hang on, hang on. I just wanted to know, could you click the checkout button and did it work? I wasn't asking for it. Usually you ask for product feedback and you're overwhelmed, right? For probably
the first 10 to 12 years of my career, which makes me feel old, I'd go ask customers that I was running their database if they had feedback. And they're like, nope, I'm good. Now, in my head I'm like, okay, the only time I've ever done that was like, I was so pissed at Hilton for screwing up my reservation that I'm like, I'm swapping the Marriott, right? Like, the only time I'm like, no, I don't have feedback is like, I'm out the door, right? I'm like, I'm going to sleep at night and like, like cold sweat, like, man, I bet they're leaving tomorrow. Like, I can't, like if they deprovision, like I knew it, I saw it coming. But no, you don't want to think about your database. Like, it's like my bank. Like, I wanted to just run and work. Like, when's the last time you went into the bank and you're like, Teller, thank you for being in my bank today. Mm-mm, never. Yeah, the thankless job of being good, I suppose. Being good and being boring, you know? No surprises, no drama, no extras. And so, repack, hence, like, the improvements to Autovacum are huge.
logical replication when you're replicating from Postgres elsewhere just gets better and better and better. Postgres 19 is pretty magical there. And kudos to all of the Postgres core developers that just kind of keep doing this, right? They're not the matches of the world that we talk about. They're kind of there hidden, you know, one person on my team, Tom Lane, I mean, he, I love Tom. He just works on Postgres, and that's just what he wants to do, and just make it better. And Tom helped work on, like, TIF, the image format, then JPEG and PNG, and when I say worked on, like, co-authored the spec for libpng, for PNG, and then co-wrote libpng. He's like, hmm, I'm getting kind of tired of this image thing. Let me take a look at this Postgres thing. I wonder if I can work on that for a little while. and now he's like a significant like percentage of every Postgres release
and just happy doing it right like the thankless work of open source that man Postgres is continuing to power the world and that's awesome and it just keeps getting better and better and better at the same time like I think it's actually like we're getting cool stuff too Like, now there's, like, graph properties coming to Postgres. You're like, wait, hang on. How is that? Like, it's a SQL database. It's not a graph. Like, nope. Like, that's gotten approved as part of, like, the SQL standard is, like, graph properties. And now do we have, like, SQL, like, graph stuff directly in Postgres as a Postgres 19? Wow. Which is just, like, wait, hang on. Now I don't need a graph database, and I have text search extensions. Do I need Elastic and Time Series? do I need a time series database? And vector, it's really like deep dive on group by all.
Like, hey, if you don't want to group by every column, one of my personal pepies in writing SQL, so you can do like select from where, and you can go group by and you can number the columns. You say group by one, two, three, four, and it'll be like the columns from the select. terrible never do that it's one of my bad habits because it's like it's easier than retyping the column name um but for like readability for someone else and if they go and move one of those now it's completely changed like the yeah like the results right because you've reordered it um it's such a bad habit i think i wrote a a post on like my sequel bad habits not like my sequel but my personal SQL Bad Habits. I had too much fun with that name. But like Group I All, now you don't have to even put the columns, right? Like, oh man, that's really nice in writing SQL now. Like, Upsert, right? Insert and OnConflict update.
Now you can actually get the return. If you want to say, insert OnConflict, don't do anything. Insert this, but it's already there. Don't do anything. it would just return. But, like, as a developer, I totally wanted that row. Well, insert this or do nothing. But tell me the row that's there so I can do something with it, right? Right. Because it's, like, create or get. Well, now, like, your ORM doesn't have to then go to two things. It just, like, now that ties that down to better SQL efficiency. And it's, like, oh, this is actually all making my life better. at the same time back to that like the best database is one I don't think about right like I want to build apps like I want to build cool features I'm not thinking about geeking out on my database but like Postgres is that foundation that you can just like bet on what's fascinating for me is also that enterprises are coming along for now I think in the developer world we've seen this for a while right like you ask a developer like yeah of course Postgres
but now enterprises are like, why would I continue to bet on the high license of Oracle or SQL server? Like, Postgres can do all this. Like, we can make this long-term bet, and a fresh-addict school grad doesn't want to be working on a DB2 server. But yeah, Postgres seems kind of cool. Yeah. Well, friends, I'm here to tell you that my friends, and maybe your friends too over at the CNCF gave me a very generous discount to give to you. Use the code CHANGELOGPOD when you register for this year's KubeCon Cloud NativeCon in Salt Lake City, Utah. It's happening November 9th through November 12th, and I want you to go. AI agents, they're changing everything. They're changing what we ask of Kubernetes. Take identity, for example. When an agent calls an API, they run some code or it touches a database, who's actually authenticated for that action?
Well, Adele S., one of the chairs of the conference, tackles this very question in his keynote, the identity crisis. Who is the AI agent in your cluster? And Jeffrey Sika steps back with Cloud Native, built by humans, engineered to evolve. He's looking at how the open source community adapts to AI. And Lynn Sun asks, cloud native without limits, what can AI agents do for us? This includes agents that help debug and operate real systems. So if you're planning to attend, as I mentioned before, deep discount, generous discount. I asked them to give you something special and they agreed. So the code to use is changelogpod. Once again, changelogpod, all over case when you register. Head to kubecon.io. That's K-U-B-E-C-O-N.io. Select the North America event, check out the schedule, and register.
And use that code. It's a deep discount. I worked so hard to give you that. And they agreed, so use it. Once again, kubecon.io. This is a new world we're in, too. I mean, I think with a database that can scale like this and be so usable by many, and a database you could bet on, as you said, in a world where people are composing more and more components that would not normally be together, meaning just maybe in this case here with PGLake, not the perfect example, but somewhat of a good example is like you're willing to bring in a different database and parquet files. The same thing with maybe a Go developer. It's like no way would I ever be like TypeScript, okay? Like nah, okay? But you see more and more teams because of the speed and the ability of the models
letting you play in different lanes you would not normally play in or commingle languages and run times where you would not normally let them run and explore even. How is this changing maybe the development of Postgres? Do you know how much the – like what do you know behind the scenes at all? If any, like is the contribution of the advent of models, you know, making developers 10Xers, 100Xers, you know, the super engineers becoming super, super, super engineers kind of thing. like how much of that is playing into this Postgres world? Yeah, I think it's a really challenging one for something like Postgres, right? I think like there's sort of an inflection point right now where I don't know how it's going to pan out. And part of that is like, look, you look at the, you know, people contributing kind of AI sloth, right? And now for a small team of Postgres of, you know, less than 100 kind of people worldwide that are actually committing and contributing to Postgres itself, like as committers,
closer to actually 50 people of committers that have a commitment to Postgres. Now they're overwhelmed with PRs they've got to kind of sort through, right? And, you know, yeah, it's easy for, like, hey, I'm not a great front-end developer, and, you know, I don't know TypeScript, but now with a model, I can generate a pretty good UI, right? I don't think it's the same for C code, right? I don't think because the models are trained on the amount of code out there, right, The more code they can learn from, the better. There's not a lot of open source public C code that exists as a great model, right? So I think it's actually a challenging inflection point for the Postgres core team and committers and community itself of they're sort of overwhelmed with new people showing up with sometimes slop and how do we validate correctness, right? How correct are the models because, you know, on a UI, on a button misplaced, it's one thing.
Very different than, uh-oh, I corrupted my data, right? I saw a little bit on Twitter, it'll always be Twitter for me, that, like, a lot of folks operating, like, production pipelines haven't hardcore shifted to, like, yeah, I'm not going to look at the code and I'm not going to understand the code and, like, only AI, like, hey, I'm just going to, you know, orchestrate it and it's going to I think there was a you know a tweet I saw about like hey look how much Spotify like deploys it's like 7500 times per day yeah AI and everyone's like it still sucks yeah and I'm like and it's been broken more in the last like I saw so many folks be like this explains so much I'm like the app hasn't really changed it's often more broken now than it was a year ago oh my gosh yes and Yeah, the amount of commits and the amount of throughput for a team is not synonymous with the quality of the software. Now, it can be, but it's not in that case.
Okay, I'm not trying to be negative to Spotify engineers. If you're a Spotify engineer, I'm not downplaying your hard work. What I'm saying is that that doesn't contribute to being better software. I think you have different challenges that everyone knows about with Spotify that is probably above your pay grade or above your role that you control. So I'm not saying you in particular, whoever's listening. I'm curious on your take. Like, think of it like Postgres and your bank, right? Like, do you want your bank to go AI everything or like in reporting your balance and deposits? Or you want to – what's your take, Sarah? Like in the R&D, yeah, absolutely. Explore in the R&D, you know, like literally in RD slash wherever that's at for you. Like, whatever that confutes you for your organization, for sure. Innovate, explore, but don't do it in production. Don't do it with my bank data.
Don't do it with my, well, do it with my loan because maybe the bounce goes to zero. I mean, Joey, either direction, you can go up and down, right? Oh, yeah, it can go up. Like, oh, yeah, I accidentally charge you more interest. I think for, you know, certain things within the bank, like, you know, hey, show me deals and offers. Show me, like, use AI where you can find savings for me, right? They're like, hey, cancel AT&T and swap to T-Mobile. And it's like, cool, great. There's definitely an area there. But on my core database, I think I'm not there yet, right? And so I think there's, and again, it comes back to like these models are trained on the amount of code that exists. And Postgres is written in C. I haven't written C in a while. I did it back at university, but that's probably the last time. and a lot of people that have written C hate writing C and I've heard people are like oh but I really like Postgres C like the standards for it are so good it's almost unique that it's like oh I can understand this and follow it it managed to maintain that bar that I think like you know it but it going to be an interesting journey right Because you know we weren talking about AI like this a year ago
Really, we had this inflection point back in November, December timeframe of like, man, these models really have gotten good and it looks different. And, you know, it's, it's week by week, month by month on how it evolves. I have sympathy for the maintainers though, of the influx of like, new PRs coming in that they've got to manage and diffuse. And how do you – Sure, open source, that's for sure. I mean, like the landscape of it. I mean, I'm not sure what to think about the next few years of open source. Like we will still have more open source, and it will still happen. It's going to keep happening. But I think you're going to see more and more overwhelm, more and more demand, I would say. Like a PR is a demand for attention, right? You got a user out there somewhere who's got a problem, whether it's a SWOT problem or not, it is a problem. And because of the groundswell, we have more users to have more problems. And so you have more PRs because you got more people who are motivated to get their problems solved.
And, hey, it's open source, right? So let me share my thing. But it's also one of thousands in some cases, if not tens of thousands in other cases. Yeah, especially on the data side. Because I think you look like if you, we've been on this journey and I had a realization a few years ago that like, you know, boot camps were a thing, right? And people learned to develop and could, you know, code an app and literally go get a job, right? Like go to a boot camp for three months, come out, you're an engineer, you get a job. And 15, 20 years ago, you wouldn't make a change to the database without it flowing through a DBA. Like, they controlled the database, how it worked, what queries ran. Like, it flowed through them. They held the keys to the castle to the database. And then we had more developers saying, great, I'm not going to go through the DBA, I'm just going to spin up the database. And then suddenly the DBA's supporting it, right? Because, oh, this is in production. Well, now the DBA is responsible for security, patching, reliability, all that.
So they're now running all these databases that developers just spun up. they didn't get to control, like, was it a good schema design or what the database was. And, okay, that got worse for DBAs, but as we've had this growth of developers, which is awesome, I love it, right? Back to my early Hirofi days, like, we had this idea of citizen developers, and I think with AI, it's just everyone is one, right? Yeah. That's what I was trying to make, is that you have a groundswell of developers with problems is where we're at right now. and yet we've never had boot camps for DBAs. That ratio of developers but not experts on databases, if it was 10 to 1 before, now it's 100 to 1. It's about to be 1,000 to 1 of all these apps running that need database expertise on top. It's going to be an interesting time of what this means for the data ecosystem and how does it adapt. Yeah, I mean, I don't want to throw one more agent into the bucket, it but I mean DBA could be database agent right I mean we are in an era where we could literally
put an agent of some version of intelligence maybe I'm gonna get a pitchfork here in at the top of Postgres you know like the planner for example it doesn't have to be a good algorithm it could be literally intelligence like a a just-in-time review of the pros and cons of this plan versus B-treat whatever yeah yeah you know no I think it's a it's a hard and good question Because your point of like you said, well, do it on R&D, but not on my prod, you know, bank account. Well, for sure, somebody else should be doing this. R&D this. It is the future. It will happen. Yeah, I think it's when do we get there, right? How fast does the prod get there? How do we adapt to it? I think it would be interesting. You know, you can, from an organizational standpoint, you can do it in a lot of different ways. I won't speculate them all, but gosh, there's a lot of ways you can do that. And eventually it will get boring. Like right now, it's so provocative. Right now, it's so not boring, right? Like, every day you look at Twitter slash X, and it's something new. I mean, just yesterday, thank you very much, we got Fable 5 back, you know?
Just so everyone's clear, it's, you know, yesterday was July 1st, today's July 2nd. And the government shut down Fable 5 access. I don't know if you've played with Fable 5 much, Craig, but I'm curious to hear what you think. But, you know, this model is so different than others, and the way it thinks and the way it looks at things. And then, wow, it just drives me crazy when I'm so not a cybersecurity terrorist and it shuts down on me because it's like, no, no, no. I just asked you to look at the software, okay? I didn't ask you to do anything bad. I can't help it. Somewhere in the software I'm trying to do secure things, which is, you know, crypto or, you know, not like little crypto, but like cryptography on like a password or something like that, or hashing things or moving data or creating a, you know, content address storage kind of thing, whatever it might be. Those are cryptography things and cybersecurity related, but I'm not trying to build a digital bomb here, okay? But anyways, I digress. No, I think it's, you know, all of that has real impact for,
like we were saying, to open source, right? And how is it created? How is it maintained? Like, it's a fascinating time. I really feel for the, you know, I'm close to it, obviously, but, like, that Postgres ecosystem, you know, continuing to thrive, and how does it maintain that? Because it's such a small set of folks, right? And Postgres has such this fast footprint. So, I mean, you know, for me, a big kudos to all the committers on doing that generally thankless work, right? You know, you don't know who they are. Like, they're just out there doing the good work because they love Postgres and it's been the test of time. So it's like, hey, it just kind of had its 30th, you know, birthday. Here's to the next 30 years for Postgres. I mean, if I was a strategic thinker, which I think I might be a little bit, if I wanted to get access to some C code that I've never had access to, or I guess in this case, since it's open source, we do. But I would be curious about how some of the labs like Anthropic or OpenAI
are looking at a code base like Postgres and an ecosystem like Postgres to one, hand it the model to secure it, because that's the one challenge we're going to have extremely in open source is visibility. And so it's secure by default because it's so visible, but it's also so insecure by default, as you can see with state-of-the-art models now and scan and discern at record spaces that we've just never been able to do before. So I feel like open source is a good thing and a bad thing when it comes to the visibility of these things and cybersecurity. But I'd be curious to know how many memory leaks or bugs are inside of Postgres because of the choice of... C is a great language, but a lot of folks are moving to Rust for safety and things like that. So it's not easy. It would be basically impossible or an undertaking of major magnitude to move Postgres and the world of this from C to Rust, for example.
It's a fun discussion that pops up on the mailing list every couple of years. Like, oh, I'm just going to re-platform it in Rust. Like, good luck. Cool. Yeah. Have fun with that. See you next time. I don't know. Maybe it happens. You know, never say never. I mean, the models are getting complicated. Maybe, you know, they're getting impressive. Maybe they can go into it. If you give Fable 5 a good plan, it'll do some cool stuff for you, man, if it doesn't get stopped by the security stuff. I mean, it really can do some amazing things. I don't know that I even want to know the token cost of it, you know, to do the conversion like that. But, yeah, no, I think we've definitely seen as a community, like, legit bugs come up that have been there for 10 years, 15 years, 20 years. But I think because Postgres is so pervasive, so many people have been looking at it, been running their scans. They are contributing back, right? And so I trust it's out of a solid spot, right? And we're continuing to, you know, every new model comes out,
focus on scanning, running, where we can contribute. Large enterprises are making big bets, right? So if it's just one of those, there's a lot, you know, Amazon, Microsoft, Google, across the board, folks making big bets on Postgres. And it's not like it's a, oh, this is a 10-person kind of open source project that, you know, it's a library that I'm using that there's not, everyone's using Postgres, right? every Fortune 500 has probably a Postgres footprint. Governments are using it. It runs the TLD.org. The .org TLD domain runs on Postgres. It's going to have eyes on it. I trust that it's like, oh, we're not going to miss some obvious thing. It's battle-tested. It stood the test of time. But yeah, it is fascinating. You know, Paul Ramsey, who's like the primary maintainer of PostGIS and author of it, is on my team.
And he's like, yeah, I ran against, you know, I think it was Opus 4.6. And he's like, or no, I think it was submitted. He's like, yeah, someone found a bug in some code. It's been there for 15 years. Like, not massive bug, but like small memory leak, 15 years there. Like, he's like, that's been there since I probably first committed it like way, way, way back. I'm like, man, you've got to manage all that memory yourself, right? You're doing all that, whereas Rust is doing it for you. Go is obviously garbage collected. You know, these are things you don't have to think about when you write Go or Rust. Now, you wouldn't write Postgres. Maybe you wouldn't go, but you'd probably be better off. If you were starting Postgres today from zero, literal zero, probably Rust, right? If it was a ground zero kind of a brand-new project, Greenfield, you wouldn't write it in Go. I mean, you could, but it would be smarter to write it in Rust for the long-term sake of it and the safety of it and just efficiency of it. But yeah, I mean, you've got to manage a lot of that memory, all that memory yourself.
And so if you've got many developers, what do you know since you've employed and had as employees or, you know, colleagues in organizations you run, what do you know about you said battle testing this is stuff I don't have any clue about and I imagine because the contribution to Postgres is so distributed meaning it's not one consolidated place where Postgres work happens it sort of happens everywhere and integrates through the beauty of or like a beauty of Git what is it like to battle test Postgres like what is the test suite Like, where does CI run? How do we know? Like, are there certifications for part of it? These are things I just never thought about. But I do think about, like, hardening particular parts of a code base or maybe even thinking about deterministic simulation testing because maybe you can't do that with physical compute.
It's just a lot easier to simulate it. Is there anything like that that backs Postgres in ways? Has anyone contributed a large amount of compute to do deterministic simulation testing? a database is a prime place where you control time, network, etc. So I'm just curious about that kind of stuff. What do you know? Yeah, there's several pieces to it, right? And so the Postgres, non-profit Postgres Foundation has its own build farm, right? That a commit goes, spins off, it runs, comes back with a bunch. Postgres, you can download the debug stuff and run the build to debug. you know people run like it's C so running static code analysis against it's pretty nice right we back as part of Crunchy Data before we you know joined Snowflake we helped author like the SIG which is like oh here's how to secure this right with like the CIS benchmark is another one of like here's how to securely deploy this and like test for all of these things
right which are like all those security checkboxes like there's a whole process and kind of investment there um the other thing is like when a new release comes out like if you want to contribute to postgres and get your hands dirty the biggest thing you can do is test stuff when it comes out and so like there'll be like a beta tag of like new features and everyone's like we need testers more than anything else everyone's like yeah i want to contribute to postgres let me go write some patches i'm gonna write this stuff it's like actually need more people testing and so you can go and like sign up to like test review the code, evaluate the feature. And so there's basically like phases in a release process for Postgres where it's like, all right, this is the commit fest. All this is committed. I've reviewed it. I've tested it. I've validated it. I've looked at the code. I've tested it. I've run other kind of regressions against it or benchmarks. And I'm thumbs up on it. And so usually at a commit fest, that's where things like make it in or don't. I forget the timing. I'll, you know, maybe try to dig it.
We can dig it up and put it in like show notes. But, like, yeah, like, if you want to contribute to Postgres in a small way, like, hey, I don't know Steve, but I think I can read it and test some stuff. Like, jumping into a commit test is a pretty fun time to learn a lot. But, yeah, Postgres community has its own build farm, own CICV that runs, and then a bunch of companies do additional testing, right? Like, we do that. I'm sure Amazon does. I'm sure Microsoft does. Snowflakes is huge. What kind of battleground, battle tests, war room? I don't know. How extreme do you get when you name these things? What is that for Snowflake? It varies. There's a lot of this for us on the performance side, right, of what's this point release versus that one and major versions. Where are regressions? For us, it's a lot of the performance side is a big piece, but then it kind of does vary based on what we're doing as well. Yeah. We've also done plenty of scanning on the, you know, the other sides. I don't know how deep I can go on like what scans are running and where, but like we do have a lot of that and take a high pride on like that security side of like, how secure is it? Where are vulnerabilities in it? That sort of thing.
Yeah. I mean, I'm not actually exposing any crown jewels or secrets or anything like that, but is the internet reported back to the Postgres community as like a pulse, so to speak? I mean, like the way I think of it is almost like information and universities where they do labs and tests. Like you're kind of like a university with labs and tests. Sure, you're a very fortunate for-profit company. I'm not saying you're a university, nonprofit, or not so much profit. Anyways, what I'm trying to say is, like, I feel like Postgres is obviously this open source thing, and then you get the benefit of using it. So how do you give that kind of awareness back to the community about how much you trust Postgres? Yeah, there's a few pieces to it, right? If we, you know, we're going to, you know, say we're adding a new extension, right? First, we're going to do our own scan, right, with all of the models and otherwise in our own, you know, security assessment. Before, that used to be manual security assessment, right?
Now, it's heavily automated with AI and tools of like, all right, what are the vulnerabilities, right? Are they critical, high, medium, low? And we're going to actually, because usually I'm old. I've known these folks for a long time, right? It's a small community. I'm usually just going to call someone up and say, like, hey, we found these vulnerabilities. How do you want us to report them, right? Because sometimes it's like a well-established project like Postgres, and there's a clear process. Sometimes it's an extension that's one person, right? And they don't have a, you know, security kind of reporting line. And so it's like, hey, by the way, also, we're going to show up with full requests to fix this, right? It's not just like, hey, I'm going to file a GitHub issue and wish you luck. Like, we believe in being really good stewards of open source, right? Like, that's, to me, pretty important. And I think core to us, we have a team that contributes to Postgres itself. And so we're going to show up with that patch. We're going to ask, how do you want us to disclose this? And sometimes they're like, great, just file the issue, show up with the patch, and we'll, you know, if we need to make a CV out of it, we will.
And sometimes it's like, oh, no, this is, you know, this bad. We don't want to advertise it before we've got their point released. That's a big one, right? It's like you don't want to go advertising vulnerabilities before they're actually someone can do something, before they fix it out. Like, you know, hey, by the way, here's your bank account number. You should change it. I don't want to go publicizing that until you change your number, right? and so that's that balancing act of working closely with the community right of knowing these folks of hey there's this vulnerability how do we want to report it when good responsible CVE disclosure and then we've you know from the overall Postgres project itself like we've got core developers just contributing back and just continuing to invest in those and I have a set of folks on my staff that are we're paying them to work on Postgres and to make it better and contribute to it. That's good. So you do have ways. You know, what you just said there with the public nature of open sources
is really I think the rub will hit on, and I'm not trying to go back to that necessarily, but you were on that and made me think about that. So, you know, you said can we share the issue. Well, that issue is public. The real challenge with open source today and the lens we take with it is that we think we, and maybe this is true when you read the licenses and the spirit of open source, or maybe I'm just not grounded enough, but you have open source, but does that mean that all the work that is open source has to happen in the public? And in the example of security, the issues are public, so the discourse is public, and so the work is in the public. And so how do you have internal communications when everything is public? How do you have secure channels when all communications are in the public? And I feel like that's probably the tension and the change to come to open source is not so much will we keep doing it, it's how do we conduct ourselves to do it. Because if the issues are in the public and you can't disclose,
you have to have back channels, well, now you're kind of circumventing the whole entire system, the whole entire workflow of what you think is work. I think Postgres, like the source code is open. right? And like does the issue have to be you know, like maybe you publish it later right? But it's like you know, Postgres has had zero days in it not commonly but in the last 15-20 years there's been about two zero days right? And that meant like databases were publicly accessible there's like remote code exploits like privilege escalation. That's terrible that just means your data's open, right? Like that is the worst thing that could happen to anything right? Like great, you can go read all you know the bank accounts, the private data, your database being open to the world, just imagine that, right, for anyone to come and grab. Bad day. Now, how do you resolve that, right? Well, you've got to get the patch in. You've got to go do point releases. Those point releases also have to be available on your provider, right? So as a community, Postgres has now got to go,
okay, well, who do we know at Amazon? Who do we know at Snowflake? Who do we know so that they can be ready on day one to go and roll their fleet? Not that, oh, the new Postgres is released. Well, I wasn't expecting this. It's, you know, about to be the 4th of July weekend. I'm headed to the lake. Hang on. Like, I don't want this. Like, when do we release this? Are they ready for it so they can immediately patch? What if they have, you know, a week of downstream work once this hits, right? How do we keep our customers secure and work together, right, with all these people delivering Postgres? And Postgres has done it really well, right? And I think larger open source projects do do this, right? There's a way to do responsible disclosure. There's a way that they coordinate with key projects in upstream, right? If there's some core Ruby feature that's a bug that needs to coordinate with Rails, they know how to do that from a Ruby to a Rails perspective. I think there's probably something for smaller projects to learn, but it's also like a point of scale, right?
How do you scale that? because you do need those back channels. You need that coordination. But it does happen in Postgres, and I don't think it's anti-open source. I don't think the moment of vulnerability happens, it needs to be for the world to be known, right? Yeah, we need to get it fixed, and we need to do it in the best possible way. And there's a whole kind of aspect of security, open source, and responsible disclosure that I think is... it's not that it wasn't important before, but man, AI has really amplified that importance, right? When anyone can find these vulnerabilities, right, or if they know about them, take advantage of them, just the risk there is higher than ever before. You know, on that front, I'm not obviously an anthropic employee, clearly, right? I'm not high on the executive board there. I don't know what's going on behind the scenes when it comes to how they run their models,
but I got to imagine that the usage patterns of someone doing cybersecurity things or nefarious things might look similar to a developer protecting themselves, but I bet they look a heck of a lot different. And so I think the way that you prompt the model, and so maybe they're using open source models, and maybe that's how they're getting around a lot of this, but, I mean, that's why Fable was so protected because of what you could do with it. But I've got to imagine, like, from the model-level perspective, there's got to be things that you could do to detect how it's being prompted. And maybe that means you're prompt-sniffing everything. I don't know. I mean, I don't know how to solve that conundrum there even on its own. But the usage of the developer or the builder or the operator, whatever you want to call them, you know, prompting the model to get the results, whether it's in an orchestration system like Inductor or simply right in the CLI, whatever, right? There's a pattern that the person is doing to get the result back or to dig into something.
I can imagine that pattern is pretty clear for nefarious action. What do you think? Is that, am I off on that? Maybe, except for like pulling that thread. It's like, well, if that pattern is clear, can I, can I try to imitate someone that is like just a developer? that's just trying to do good local crypto and convince the model that's my path and then keep pulling that thread to see if I can find something. Hey, is that a backdoor in? I don't envy their position. They want this to be available to people, but they've got a balance of bad actors and what that means. I think at the surface, yeah, that makes sense. Then I start to pull that thread more and more. Well, okay, like where is the line? What's good? And as a bad actor, how can I make myself look like a good actor to then get as much out of the model as I can for my purposes, right? You know, my – and again, I'm not coming back to this necessarily, but like my fable is back for one day, you know.
I can't seem to get past anything security because I get so good at the security stuff. I'm like, well, that's kind of what I want to ask you about. And I kind of want to ask about, like, where is my application stack vulnerable? Fix it. Don't go and, you know, attack someone, but fix it so I'm not attacked. I don't understand that scenario. But, you know, it's a different world we're in right now because they're trying to protect it so much. I guess it's just that powerful. Yeah, it's just the nature of the beast in the moment, I suppose. Yeah, and I think, you know, to taste these things are moving, right? Like in a week, we'll have a completely different conversation. We'll be like, you know what? It's wide open. Like it's, you know, the pace of change here, I think at first it was exhausting. I think now it's getting back to being fun. It's like, man, okay, like I'm back to the early days of learning to develop, right? And building those first, you know, Ruby apps. And it's starting to feel fun again. Like at first it was overwhelming.
It's like, man, I don't have my creativity. I can't learn. Like I'm just prompting. But it's like, actually, no, now the limit is your creativity and how far you can push things, which it's flipped for me that it went from overwhelming to it's a fun time. But yeah, I hear you. It's like, man, I want to use this for the good. Why is it constricting me? I think we'll be, you know, it'll continue to evolve for us. Yeah. Yeah. Hopefully, because if not, we'll have to just – I think what will probably happen is that they'll just make fable or things that do that kind of stuff. I'd even happily give them more of my personal identification to prove I'm not an aference actor. Name it back to a real person. Am I literally a real ID, real kind of person that has a driver's license, that pays taxes, that I got a utility bill? You got to do these certain things to get certain credentials or capabilities.
Why not that? Maybe they'll do that sometime in the near future. I think we may have to go back to manually writing code, right? I mean, knowing stuff, we can't go back to that. Come on, man. I want to just wish it into existence. It's just so much easier. To OpenAI credit I was doing something with a login I was building a you mentioned ETL earlier I was building an ETL pipeline to move some data around and I had to let just say get access through just logging a password. I had to find my way in, let's just say. And opening, I was like, hey, this seems like you're doing things you probably shouldn't be doing here. And it flagged me. And I was like, oh my gosh, I've never been flagged before because I'm just trying to help my sales team get access to data here and pull the data here, transform it, move it over there, and all that good stuff on the ETA pipeline. And I'm like, hey, hang on a second here. I started defending myself. I'm just helping the sales team move data.
We have our data and our leads over here. We need to move them over there kind of thing. And to opening-ass credit, they let you submit a form to apply to get that level of, not like a certification, but you have to do something. I forget what it was, but it was enough to say, I'm real. I'm advocating that I'm trying to do things that are not nefarious. Don't flag me so much or flag me at all. And ever since then, nothing. So I was kind of happy about that. I wouldn't mind assuming a little of me to get a benefit. It's like security clearance away. It's good to know. I did a similar family. I went to Disney World a little while back. Making those dining reservations, it's like, man, those things just go like that. And so I'm like, all right, this is just running for me locally. I want you to log in. Like, there's no API for, like, what availability is. I'm like, all right, like, I want to, like, screen scrape and, like, text me when there's this new reservation available for this restaurant.
Like, because, you know, the kids want to eat Beauty and the Beast or whatever, right? And it's sold out. Wow. And then, like, hey, it's not available. So, like, I didn't get prompted and, like, told, like, hey, you're doing something weird. So it was probably back before they had more control. So good to know, the next time I run into it, I can just fill out a form. And I'm pretty sure they even link to it in the air, because I was, like, really, I was concerned. This was maybe, like, two months ago, and I was like, oh, my gosh. That's the first time I had this dark feeling in my brain, like, what if they took your clicks away from you, Adam? What would you do then? What would you like? I can't have stopped. I thought now promised like that full panic of like, uh-oh, are they going to... I didn't mean it, Sam. Don't take my codex away. Anyways, I really did have a panic. I'm like, well, hang on a second. I'm not a terrorist. I'm not doing bad. And they had a link to the form. I filled it out. It's all been good since then.
This episode is brought to you by our friends at Fingerprint. There is an interesting problem happening today. building products, you want to make sure things are as easy as possible for users. But you also need to know who or what you can trust. That's getting more complicated today with AI. AI agents are browsing, they're shopping, they're transacting alongside humans. Some of that automation is legitimate. Obviously, I use AI, you use AI, but some of it is not legitimate. And fraudsters, the bad guys, are getting more sophisticated. That's where Fingerprint comes in. Fingerprint gives developers device intelligence to recognize returning visitors and make better trust decisions. It combines more than 100 signals across devices, networks, and behavior to identify visitors with precision, even as fraud tactics evolve. And it goes beyond identification. identification fingerprint helps you detect bots ai agents proxies vpns and other indicators
of suspicious activity so now you can protect accounts prevent fraud and reduce unnecessary friction for the people who actually use your product and the developer experience is straightforward one api call no fraud detection model to build or to maintain and now with ai install you can add fingerprint to your platform directly from your favorite AI tool. So if you're building products for a world where humans and agents share the same internet, fingerprint is worth checking out. Head to fingerprint.com to get started. Once again, that's fingerprint.com. This episode is brought to you by WorkOS. agents aren't just writing code anymore they're accessing customer data they're taking actions across the tools we use and depend on so who decides what they're allowed to do that's a big
theme from work os and nit 2026 their conference for builders they previewed three products that show where things are heading first airlock intent-based access control for ai agents in a Live demo and agent worked across Linear, Snowflake, Stripe, and Gmail. Airlock denied a large refund, sent a smaller one to a human for approval, and blocked an email containing financial data. And then there's Atlas, a secure home for internal apps built with AI. Think authentication, data connections, credentials, permissions, and deployment. The stuff a prototype needs before it's safe to use at work. And then they have Passport. WorkOS is pushed towards zero sign-in. Enroll your Mac once, then open WorkApps already signed in. All three were announced as previews, but the direction is clear. As AI makes software easier to build, we need to get more serious about who and what can access it. I'm linking up the Ennit 2026 recap, so check it out at WorkOS.com.
Link is in the show notes. Thanks again to our friends at WorkOS for sponsoring ChangeLog. Once again, WorkOS.com. But man, wow, okay. So Postgres 19, the future is Postgres. We've been Postgres for life. What is the next big thing to come in, I don't know, what do you predict for the next year? So by December, give me some predictions. You got anything? I mean, I think a lot of what we're doing with PG Lake is pretty, it's early and new. like I'll come back to it that it's like it sort of blows your mind at first like Postgres shouldn't be doing this stuff I think you know AI is plugging into Postgres everyone's like what's Postgres got to do with AI it's like well it's the database that everyone's going to use under the covers right um I think your point like how do we have kind of like database agents like optimizing our Postgres because most developers aren't great at it right um but I think you know
we covered a lot of it. Extensions, Postgres 19, it's final, it's going to be in there, but it's not here yet. So I think that's the big thing. Development's already happening for 20. I think flexibility on data types, richness of text search, all those things just continue to get better. It's hard for me to say, oh, what's the big prediction? It's like, man, we're going to have some awesome new extensions come out that get released and Postgres is going to keep getting better and more performant and it's just going to be like, you know, what's the prediction for my bank? Well, I hope it just keeps my money safe. And, you know, Postgres is just going to keep doing what it does. It's like a flywheel in a way, right? You know, you have the core nugget, which is good, and then you have the core nugget that has extensions and you can make things that make it better. And those things make it better. and then more things make the things make the things make it better. The next thing you know, you sort of have this multiplication happening
and this flywheel happening as each layer matures. And like you had said, with PG Lake, you've got several components beneath it from table to engine to iceberg, you know, all the different components that make it up, all the different extensions that make up PG Lake. You know, that's really interesting. And getting better at that is probably what will happen for Postgres at large. What about PG Lake? What's a good way into PG Lake for someone? If someone's out there thinking, you know what, I've heard about this. That sounds really cool. Yeah, DuckDB and Parquet, sure, that's awesome. Data analytics, okay, cool. How does someone get involved with PG Lake? What can they do with it? How do they add it to Postgres? Yeah, search PG Lake, and it'll come up to GitHub, right? You can install the extension yourself, right? You can pull it down locally, get the tires. like it's available on Snowflake Postgres, right? You feel like, I don't want to build Postgres. I don't want to think about, like, how do I, like, it's on Snowflake Postgres, so you can just run, you know,
create extension PG Lake, and boom, you've got it all there, right? And you can kick the tires, see how that works. But yeah, you can also, you know, contribute to it as well. I mean, hop over to GitHub, throw a star on it. Everyone loves a GitHub star. See what's going on with it. Like, it continues to get active development. We released it back in November. but it was actually like two years of work of development from back in our crunchy David days so a lot of advanced engineering that then was open sourced two years later and we're continuing to invest in it and you know shoot me a note Craig Kirstein's on Twitter or I guess it's X now it's still Twitter to me, always will be I guess it's X now like send your feedback on what you'd like to see or you know hey have you thought about this like you know the crazy ideas that people are like well have you you know consider this or this seems weird or crazy. Like, I love it. Let's go build it. Mm-hmm. What about Snowflake? What's on the, is PG like the edge for Snowflake? You know, in what ways is Snowflake truly adopting?
And I know you mentioned developers being paid to invest in Postgres, but, like, what is Snowflake's big bet on Postgres? Yeah, the, so everyone that I think, you know, uses Snowflake also probably uses Postgres somewhere, right? And, like, we talked about this a while ago of, like, this divide of OLTP and OLAP and different kind of teams and systems and, like, bridging those two together. For us, PG Lake is sort of the foundation, but, like, the end perfect result is what we recently announced, like, database mirroring, where I can just, like, run one command and mirror my Postgres database to Snowflake and vice versa. and not have to think about CVC, not have to think about ETL. It's literally like click a few buttons in a UI and the tables are continuously replicated in real time. And you don't have to think about it. You don't have to worry about it. So now you've got your transactional system, your analytical system has the exact same data. I've got one customer that's a smart TV provider
that ingests all this stuff real time into Postgres. So you're viewing Disney Junior at home. I see you're in San Francisco. We're sending all this data into Postgres. Postgres now feeds that into, in this case, it's PG Lake. It could be mirroring. That Snowflake picks up and does all this analysis, and it's like, okay, we see your viewing history. We see where you're located. Now we want to, because everything's an ad platform these days, so it's a smart TV, so we want to show some ads. We know Disney on Ice is coming to Oakland next month, right? So we want to show this ad so your kids are like, dad, please, can we go? Like, it's right here. Can we go? And, yeah, I'm so annoyed at the ads on my TV because they're usually pretty accurate and good. But, like, so Snowflake, you know, they use then, like, Cortex, Snowflake's kind of AI layer on top to then say, what ad should we show? That goes from Snowflake back to Postgres to be that real-time single millisecond serving layer. And it's like, this is just, like, common data pipeline now with, like,
oh, yeah, I used Postgres before, but now I want to tie it to my snowflake experience. Cool, there's a lot of beauty and simplicity of these things I can build. For me, it's so, you know, I run databases. It's not cool. It's not the sexiest thing in the world, but I live vicariously through all these cool things people get to build. Like, oh, cool, this makes sense on this TV pipeline for advertising. I've got one customer that he surveys snakes in Canada. like, goes out, catches snakes, tracks the population of them. Like, so, like, two, three days of the week, he's out in the field catching snakes and tracking the population of them. And then he feeds this into a Postgres database using PostGIS. So, like, guy, part-time snake catcher, part-time, like, like, what's he? A collector. He's a censor. Yeah, it's like, I'm like, how do you, like, what is this, like, job that you get?
This is cool. I'd love to go to the job fair at the school and talk about, yeah, I catch snakes and I throw them in a database. And that's how I track all this and the population over time. So for me, I get to live vicariously in all these awesome apps and use cases people build. And so at the core of it, Snowflake is absolutely amazing for analytics. and now we want to tie that together of like your perfect data platform for like analytics, AI, data all together. Yeah, I mean, I'm a home labber. I've recently changed to saying dev lab versus home lab just because my lab is now just so developer. I mean, it always has been, but like it's so developer. And it's almost like when you talk to an agent about home lab, it's kind of a pejorative in a way to them. It's like, oh, there's no scale there. You're just doing toy stuff. It's like, no, no, no. You got to do Raspberry Pi. Shut your mouth, agent.
This is the developer lab, man. We're doing some cool stuff in here, okay? We're breaking ground. We're innovating. Anyways, I was just thinking, like, any given decent dev lab, sorry if you don't, should have, likely has a Proxmox box, which is a place you can, you know, it's a hypervisor. You can have stand-up virtual machines and LXCs and all that good stuff there. you know, I hadn't really paid much attention to PG Lake. It's about eight months old, right? This isn't a very old project. Maybe it's old in terms of development, but not old in terms of open source. It's relatively new in the last year. Like, it's basically a data warehouse in a box now. And so why not stand up a virtual machine with some disk that's hardened? Maybe you got TrueNAS you can back it up to and actually create a centralized data warehouse in your dev lab just for fun. Collect your data there. I'm looking forward to next time to hearing all the ways you analyze your data with PG Lake.
I'm going to. So I do a lot of stuff. Actually, I moonlight for fun, release for fun. A few folks of mine, I've helped them create some data pipelines, like ETL pipelines and stuff like that. And so I've got kind of a framework I've built that, you know, So uses Postgres, obviously, in some cases, but by and large, it's DuckDB and Parquet, all right? I mean, especially whenever I'm extracting. Extracting to a Parquet file with watermarking, like, that is the way. It's fast. You can analyze it pretty quickly. You can even use a Just file to kind of create a bespoke kind of like just-in-time API over the query. So rather than writing the queries all yourself, you just package them up in a Just file, for example. and it kind of makes it like this kind of fairly quickly an ability to sort of like iterate over data, at least that's where I've been playing at. But I don't really have a centralized place to kind of keep all my data at once it's gotten there. Like that's the next big problem actually is like, oh my gosh, I have all this data.
How do I harden it? Okay, I got to actually centralize it. And it's not just one tenant, it's several, like two or three tenants. so the tenant have to be isolated. Tenant one can't know about tenant two at all. It's just data. It's a warehouse to put it to. This is my next problem, really, and so maybe I'll stand up a VM on my ProxBox box and become a DBA for PG Lake. We've got one more DBA in the world. I love it. One more. Yeah, one more for sure. Well, Craig, thank you so much, man. What else can we – I know we mentioned PG Lake, how to get there, et cetera. Anything coming from that that you can talk about that makes sense now? What else in closing can we talk about? That's a good question. I mean, I think the biggest, you know, the other thing is like, hey, you know, there's a lot of people like, where do I go to learn a little more about Postgres, right? Like, cool, this is exciting. How do I, you know, a few things. One, the Postgres mailing list.
Like, man, you want to geek out on Postgres, the PG-SQL hackers mailing list? Um, that's where the development for Postgres happens. Like, you want to go learn? All in the public? All of it? Yep, I mean, that's, that's, I mean, I'm sure there's back channels, right? There's a couple of times, but like, but, but no, the PG-SQL hackers mailing lists, you want to learn about databases and get a masterclass, it's a great spot. You want to contribute and help others, the PG-SQL users mailing lists, right? Like, that's where users show up like, hey, I've got this query and it's not performing. Like, go help someone else out, right? Like, helping others in the great environment. If you really want to get involved, like, look at the commit fest. See if you can go review a patch and contribute in that way. And so I think it's, you know, it's a small set of people and a huge thank you to, like, actually commit and contribute to Postgres. Like, this is open source in its truest, purest form.
and as an open source fan, I love it. I want to see it kind of thrive. It's 30 years old and yet it's still going. And I can't think of any other things kind of like, man, what JavaScript framework has lasted, you know, more than two, right? Like we have a new one every so long. Man, give me back some jQuery. Is jQuery still around? It is still around. This version recently. Really? in the last year. Yeah, there's no release. Man, I want to go give jQuery a good old ride. But it's been a minute. But yeah, I think, you know, kudos to the Postgres committers. I'd love to see the community keep growing. And, like, there's so many ways you can contribute, whether it's, like, local user groups or if you want to learn more, I think it's, like, you know, people here love learning, right? Like, go check out the hackers mailing list and have some fun. Don't be afraid of it. as soon as Postgres 19 is out in a couple of months,
take the tires, take a look at the new features. Like, don't be afraid of your database. I think most people are. They're like, oh, no, that's like, that's scary. That's hard. That's complicated. Like, we're developers. We're curious. Like, dig in. Like, Postgres doesn't have to be scary. That's my biggest kind of takeaway for folks is like, dig in, have some fun with it, learn a little, and then if you find it exciting, contribute back. in any small way that you can. Yeah. The, you know, remind me how to get involved in mailing lists. So it's pgsql-hackers at postgrasql.org. I'll tell you. Yes. If you basically search for that, I think you'll come up to, here's how you subscribe. There's more than mailing lists. There's multiple mailing lists, right? And so those are the two, I think, the hackers one, which is where the action development happens and the user wants for debugging. But there's one for the dub, dub, dub
and announcements and all sorts. So go read the Postgres wiki on all the mailing lists. If you're like, man, I don't want to know how the sausage is made, but I'd love to help. I'm good at this or that. There's maybe one for you there. One more question for you before I let you go. You know, I'm a small angel investor in Superbase. and I've met some folks that are, I mean, I guess they're the futures version of developer, but they're still not a version of the past version of the developer that are now using Postgres. I'm like, what are you using Postgres for? And I'm like, or no, sorry, they're not using Postgres. They are using Postgres. They don't know they're using Postgres. Yeah. And I'm like, what are you using for your database? Are they like Superbase? I'm like, no, that's not your database. That's where your database is. Yeah. And I know now because they said Superbase, they're using Postgres, but they don't know. What do you think about this? like especially Superbase, they've had a major hockey stick with the influx and inflection point that we're in right now with database usage and stuff like that. Like having big players like PlanetScale and Superbase and Neon
and a few others like choosing Postgres, this must have made like your career magnum opus. Like just the fact that like we're now here and when you choose a database, you're not actually the developer of the future isn't choosing Postgres, they're choosing the thing that chose Postgres because they had good taste. What do you think about that? Yeah. Yeah. No, I think it's interesting to me because I see so many folks that are new. Like, you know, I love the Superbase folks. I love what PlanetScale is doing. Like, wait, DevCools are cool again, right? Like, man, we're back to Heroku and GitHub again. Finally, DevCools are cool again, right? Because for a while we went through this phase. We were like, man, as an angel investor, do you really want to invest in DevTools? Like, that's throwing some money out the window for a little while, right? And I was like, shut up and take my money, man. Just take the money. Get it out of here. It's your money now, Superbase. Go do something with it. And, like, you know, I think Superbase for me is the closest parallel is Firebase, right? Like, that's what they are, right? It's, hey, it's this easy layer for kind of APIs and tool kits to build your app, right?
More so than it is directly just a database provider. And so, I mean, it's great that they're building on it. And there's other people that are like, oh, great, I'm going to go build my startup on Postgres or around Postgres or extend Postgres. and you know it's exciting I think to see that I'm also kind of amused a little bit that so many people are like man I know Postgres now and I'm the Postgres person and I've been in this space for two years and it's like man I feel like the old gray beard right well I'm glad you come around because some of us have been doing this for a long long long time and we have to be welcoming to the folks we do have to be welcoming I'm excited More people means more software. More software means more things that software does. It just means there's just more, right? It's going to be, I mean, I'm trying to think about the negative, but it's mostly net positive. I just keep coming back to it. Sure, there's negatives, but it's almost all net positives. Yeah, but it's also like, man, building these products is hard, right, and doing this good is hard.
And so I think it's like there's a lot of value in doing this well, right? I think there are folks doing this really well. you've got to be solid at the foundations I don't want the new kid out of college founding a bank and giving them my money I kind of want the institution that's been doing this for a few years but a lot of those I think are pulling in core Postgres people they are really contributing back and I think it's important when you look at all these Postgres companies popping up how much are they digging into the Postgres world Are they becoming part of it or are they saying, hey, yeah, I'm throwing Postgres on my label because everyone loves Postgres. I think, you know, Supabase, PlanetScaler2 that are really like good deep engineering and digging into Postgres. And I think there's, you know, you've just got to kind of, hey, are they really Postgres? Are they making Postgres better? Are they contributing back? It's an important evaluation for me because otherwise I think like that tide changes quickly and that foundation may not be there.
Yeah Good word there Well Craig it's been a pleasure to catch back up with you again I always love talking about Postgres Postgres for life, for life, for life For life around here We need that shirt Yeah we just need like Endless for life's, I don't know I feel like especially where we're at now It's not going to change for me You know I will say Local dev I'm always with SQLite I've been dabbling with Terzo a little bit Because why not but once it's prod or production like Postgres hardened, backed up, true etc you know, can't why end of the way, and I definitely have to pay with PG Lake, I'm going to have to play with this yeah, no I mean anytime I'm happy to geek out on Postgres it's been too long, we'll do it again soon and yeah, it doesn't take much to line me up to geek out on Postgres for a little while.
Right on. It's been super fun. Thanks for having me. It's good having you, Craig. Thank you. Well, friends, it's been the year of Postgres for many years now, but I think this year might actually be the year of Postgres. What do you think? Are you Postgres for life like I am? Maybe you're not. Who knows? But it's always good talking to Craig. He's always excited about Postgres, and I love that. of course a big thanks to our friends over at fly.io the home of changelog.com and maybe the home of your agents learn more at fly.io of course the beat freak in residence break master cylinder bringing those beats nodding our head and I love it that's it the show's done we'll see you in the next one Win it.
Win it. It really whips the...
番組の概要欄(原文)
Craig Kerstiens returns to talk about the ever-expanding world of Postgres: a deliberately boring core, a thriving extension ecosystem, and pg_lake bringing DuckDB and Iceberg into the mix. We dig into the pain of ETL, what Craig is watching in Postgres 19, the impact of AI on open source maintainers, and why the database still has plenty of room to grow.
出典・この記事について
- 番組
- Changelog Master Feed(Changelog Media)
- 元の回
- The future is Postgres (Changelog Interviews #688)
- 配信日
- 要約作成日
- 公式サイト
- changelog.com/master
この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について