
技術面接は壊れている:HackerRank CEO が語るAI時代の採用
The Technical Interview Is Broken

技術面接は壊れている:HackerRank CEO が語るAI時代の採用
Software Engineering Daily
要約
HackerRank の共同創業者兼 CEO、Vivek Ravisankar 氏が Josh Goldberg 氏と、アルゴリズム問題中心の技術面接がなぜ機能しなくなったかを議論した回。同社は AI エージェントによる面接官「Chakra」を投入し、実務を模した課題と AI 活用力の評価へ移行している。公平性・規制対応、ソフトウェアエンジニアリングが全職種に広がる見通し、モデルの今後、コードレビューの変化にも話が及んだ。
この回のポイント
- ●Vivek 氏は、LLM があらゆる古典的アルゴリズム問題をワンショットで解けるようになり、従来型の面接が役に立たなくなったと述べている。
- ●同社は新方針として、問題の種類を実リポジトリ上の課題に、評価軸をコードの正しさから批判的思考・判断力・AI 活用力に、環境を IDE からエージェント型開発環境(ADE)に変えている。
- ●Chakra は約45分の実作業と約15分のフォローアップ質問で構成され、「課題を遂行できたか」「AI をどれだけ効果的に使ったか(framing・judging・steering)」「理解しているか」の3軸で評価する。
- ●公平性については、声や外見は評価せずすべてテキスト化して評価し、独自モデルと評価(evals)、NYC 法や EU AI Act などの規制への準拠を重視していると説明した。
- ●文化適合性の評価は人間に残り、AI が技術面を肩代わりすることで人が見る時間が増えるとの見方を示した。
- ●コードレビューも変わるべきで、PR だけでなく開発者と AI のやり取りを見ることが重要なシグナルになりうると語った。
章立て
HackerRank 創業と面接問題の変遷
Amazon 時代の経験から「学歴より スキル」という発想で創業。Microsoft のパズル型から Google 型のアルゴリズム問題、そして AI による陳腐化までの流れを説明する。
面接の目的と基礎知識のバランス
パイロットの資格取得を例に、基礎知識と実務模擬の両方が必要だが、これまで基礎の比重が大きすぎたと語る。謝罪の広告看板の話や、学歴バイアスの問題も扱う。
HackerRank の再発明と評価指標
問題・評価軸・候補者体験の3つの変革を説明。グッドハートの法則への見解と、悪影響の有無や開発者満足度(DLI)などによる面接の有効性測定を述べる。
AI 面接官 Chakra の仕組み
キャンバス上で実務課題を行い、AI がフォローアップ質問をする設計。DLI は4.6〜4.7で、動画アバターを使わない方針や evals の重要性を語る。
差別化・規制・バイアス対策
評価と不正対策の品質、独自モデル、採用に関する規制への準拠が競争優位だと主張。アクセントや言い回しによるバイアスへの対応も議論する。
全職種へ広がるエンジニアリングと評価手法
ソフトウェアエンジニアリングは死んだのではなく至る所にあるという見方、文化適合は人間に残るという考え、Chakra の3軸評価と「slop」PR の問題を述べる。
モデルの将来とコードレビュー
ツール利用、オーケストレーション、長期タスクなどモデルの進化予想、ベンチマークや RL gym 事業の構想、AI 時代のレビューの形を語る。
映画制作と AI
映画制作とソフトウェア開発の共通点、および吹き替えのリップシンクや絵コンテ、予告編制作での AI の影響を話す。
解説記事
HackerRank の共同創業者兼 CEO、Vivek Ravisankar 氏が、Software Engineering Daily に出演し、ホストの Josh Goldberg 氏と技術面接の現状と未来を語った。リンクリストの反転のようなアルゴリズム問題が AI に簡単に解かれてしまう今、面接は何を測るべきか。長年その種の問題を提供してきた当事者の率直な見解が聞ける回である。
面接問題の変遷とアルゴリズム偏重への反省
Vivek 氏は Amazon で開発者として多くの面接を担当した際、履歴書と実力の相関が薄いと感じたことが創業の原点だと述べている。面接問題の歴史については、その時代で最も目立つソフトウェア企業のやり方が標準になってきたと説明する。Microsoft の「747に入るゴルフボールの数」のような思考力パズルから、Google に倣ったデータ構造・アルゴリズム問題へと移ったが、開発者側には「実際の仕事と違う」という不満が常にあったという。そこへ AI が登場し、こうした問題をワンショットで解けるようになったことで、各社が採用を見直さざるを得なくなったという見方だ。
同氏は、基礎知識の確認は必要だと認めつつ、これまで基礎の比重を置きすぎたと述べた。パイロットの資格取得(筆記、シミュレーター、実技、訓練時間、健康診断)を例に、AI を使わない基礎確認と実務模擬を組み合わせる設計が望ましいとする。同社はアルゴリズム問題への謝罪をうたう広告看板まで出しているという。
新しい面接の形:問題・評価軸・環境の刷新
同社は3つの軸で変革を進めている。問題はアルゴリズム問題から実際のコードリポジトリ上の課題へ、評価は正しさから批判的思考、判断力、AI 活用力へ、環境は IDE から、エージェントが中心でコードが脇に置かれる「ADE(エージェント型開発環境)」へ移るというものだ。
面接の有効性の測り方も説明された。まず特定の性別や人種への不利な影響がないかを確認し、スコア分布や候補者満足度(Developer Love Index)を見る。次に、スクリーニング通過者が面接を通過する割合との相関を見る。最後に入社後の活躍だが、業績データの取得は難しく、在職期間や昇進といった代理指標を使うと語った。グッドハートの法則(指標が目標になると良い指標でなくなる)への懸念については、起こりうるとしつつ代替案はないとし、実務に近い設計がその影響を和らげるという見解を示した。
AI 面接官 Chakra の設計
新製品 Chakra は、開発環境やホワイトボードなどの「キャンバス」上で実務課題を出し、AI エージェントの利用を許したうえで、完了後に「なぜ A でなく B を選んだか」といった追加質問をするエージェント型面接官である。多くの面接は約45分の実作業と約15分のフォローアップで構成される。評価は、課題を遂行できたか、AI をどれだけ効果的に使ったか、内容を理解しているかの3軸だ。AI の使い方は、問題を AI にどう提示したか(framing)、出力をどう判断したか(judging)、どう誘導したか(steering)で採点するという。
設計原則としては、動画アバターは使わず AI であることを明示する、評価(evals)の質を最重視する、長時間の音声のみの対話は避ける、が挙げられた。Vivek 氏によれば、Chakra は50万件超のセッションを重ね、候補者による評価は5点満点で4.6〜4.7と、静的なテイクホーム課題より高かったという。
公平性・規制と競合との差別化
同氏は、競合との差を決めるのは評価と不正対策(インテグリティ)の強さだと主張する。独自モデルの構築に加え、採用は規制の厳しい分野であり、NYC の法律や EU AI Act、EEOC などへの準拠、説明可能性、学習データに顧客データを使わないことが必要だと述べた。バイアスへの懸念に対しては、声や外見は評価せずすべてテキストに変換して評価している、現在は技術職中心で内容が専門的なので方言などの影響は見ていない、と答えた。ただし一般的な60分の音声面接では差異が出うるとも認めている。
「ソフトウェアエンジニアリングはどこにでもある」
Vivek 氏は、「ソフトウェアエンジニアリングは死んだ」という論調が「あらゆる職種に広がっている」へと変わったと述べる。PM、デザイナー、サポート、営業やマーケティングにも「エンジニア」的役割が生まれているという。そのうえでエンジニアリングとは、アイデアを動くものに変える力、問題の分解、エッジケースの考慮、保守可能な構築だと定義している。一方、カルチャーフィットの確認は人間に残り、AI が技術評価を肩代わりすれば、人が候補者と向き合う時間はむしろ増えるはずだという考えを示した。
また「slop」な大量 PR の問題にも触れ、AI の出力を正しく導き判断できない人は、AI を使う面接の後半の評価で露呈すると語った。
モデルの未来とコードレビューの変化
今後のモデルについては、ツール利用の向上、オーケストレーション、長期タスク、いつ自分で動きいつ確認するかの判断、ローカルとクラウドの併用を予想した。同社は現実的なタスクに基づくベンチマークや、RL gym・検証器・専門家コミュニティを使ったモデル向け事業の立ち上げも示唆している。
コードレビューについては、開発の方法は変わったのにレビューの方法は変わっていないと指摘し、PR だけでなく開発者が AI に何を指示したかを見ることが重要になりうると述べた。
まとめ
本エピソードは、AI を禁止して従来の問題を守るのではなく、AI の利用を前提に判断力と理解度を測る方向へ面接を再設計するという一つの具体的な方針を示している。日本でもコーディングテストを導入する企業は多く、AI 利用の扱いや評価の公平性、法規制対応は避けられない論点だ。なお、数値や優位性の主張は自社製品を提供する CEO の発言であり、独立した検証は番組内で示されていない点に留意したい。
文字起こし(英語・自動生成)
The technical interview has long been used as a proxy for actual job performance. Brain teasers, data structure puzzles, or reversing a linked list became the price of admission to employment as a developer in many organizations, even though few engineers do that work day to day. It would seem AI has now broken that model entirely, since an LLM can one-shot any classic algorithm question. This is forcing tech companies to rethink what an interview should measure. Vivek Ravi Shunkar is the co-founder and CEO of HackerRank, which is one of the most well-known platforms for tech interview practice problems. In this episode, Vivek joins Josh Goldberg to discuss why old-style interviews stopped working, the company's apology tour for years of algorithm puzzles, how to keep AI evaluation fair and trustworthy, and where both models and the shape of hiring may be headed as software engineering becomes something every role does.
This episode is hosted by Josh Goldberg, front-end developer at Sentry and open-source maintainer. Josh works on projects in the TypeScript ecosystem, most notably TypeScript ESLint, a powerful static analysis toolset for JavaScript and TypeScript. Josh is also the author of the O'Reilly Learning TypeScript book. a Microsoft MVP for developer technologies, and a co-founder of SquiggleConf, a conference for excellent web developer tooling. Find Josh on Blue Sky, Faustodon, and .com as Joshua K. Goldberg. with me today is Vivek Ravasankar, CEO of HackerRank. Vivek, welcome to Software Engineering Daily. Thank you for the invite. Super good to be here. How's it going? Things are good. I'm always excited to talk about tech interviews and AI and how it's all shaping together. But before
we dive into all that, tell us, how did you get started with HackerRank? Yeah, I started this company over a decade ago. I used to work at Amazon as a developer where I interviewed a lot of people and I realized that resumes didn't have a great correlation to their skills. I interviewed people who looked great on paper, completely bombed the interview and vice versa. And that led to the idea of maybe skills matter more than pedigree. Of course, we hadn't codified it that way, but that was the founding inspiration to get going. And we started the company with that ethos, it's still our ethos. Lovely. And over the years, you've seen quite a few different styles of interview questions. What was the original premise for what kinds of things you were seeing companies ask and have folks answer? Yeah, I think if you were to just zoom out and think about all the evolution of questions, it always used to be one of those things of whichever was the most prominent software engineering company, and whatever their interview process was, it became the standard de facto across the board. So Microsoft was technically
the first software company. And the questions that they used to ask early on were things like how many golf balls fit in a 747, how many traffic signals are there in Chicago, and things on those lines. Obviously, the idea here is not for you to get the right answer, but to test your thought process, how effectively did you think through computational thinking. And of course, the entire software engineer interviews used to be like that. Developers used to hate it. It has got nothing to do with the job. And then Google came in to the picture and said, like, hey, we're like an amazing software engineering company. We are going to index the entire web, which means that we are going to ask some very hard data structures and algorithms type questions. And where do the best engineers work? Like they work at Google and everybody started copying that style of interview questions. And now the developers were in the perspective of, okay, it's better than asking how many golf balls fit in a 747 because I at least have to do right code. But it still is not realistic. I don't write a breadth-first search in my company.
I don't, like, reverse a linked list. I don't invert binary trees. Why are you asking me all of these questions? And there's always this undercurrent of interviews don't reflect your actual job. And then over time, I don't think there was one particular company, just like AI as a factor, where it was able to one-shot all of these different algorithm questions and made them pretty much useless in an interview setting, which then forced every company to rethink their hiring, which forced HackerRank in itself to go ahead and rethink our interview process, what we want to hire. And now it's in the new era, the agentic era of interviewing, where whatever you're being interviewed on is going to mimic your actual work. So that's been the evolution so far. It's interesting you bring up the perceived, perhaps actual, mismatch between questions and day-to-day job responsibilities. Because I went through the CS hiring pipeline back in the day, myself around a decade ago. And there's this perception that my fellow students and I had back then, and many folks still do now, that there is a severe mismatch between what platforms like HackerRank and LeCode and all the others do versus the actual questions.
So how would you respond to that? Do you think HackerRank traditionally has been a representative interview for what folks do in their jobs? I think they're like, maybe we can answer it in two parts. And maybe I'll start with an analogy and go and answer this in two parts, knowing that analogies are not perfect. So if you were to take the interview process of a pilot, a real pilot, and 90% of your flight is flown by AI, but that doesn't mean that you will get on a plane with a wide piloter. You still want the person to know fundamentals of aeronautical engineering and others. And if you were to, like, think about the interviewing process, the first step is to evaluate your fundamentals of aeronautical engineering. It's actually a multiple choice set of questions. No AI, obviously, to know whether you know the fundamentals. The second round is a round on flight simulator, simulating different scenarios. The third is the actual flight exercises. And the fourth is number of hours that you've actually been trained. It's not exactly a round, but like a credential that you need to have. And the fifth is your health checks.
And then like you get certified as a pilot. I'm just compressing a lot of this stuff just for brevity, but like that's the high level overview. Now, the reason why I shared that is we have to ask ourselves, like, what is the purpose of the interview? And then, like, you know, determine what the rounds are. Now, obviously, when it comes to interview, you want to make sure that you are hiring the person who is going to perform the best inside your organization. And then there are, like, two different components of it. There is, like, a technical component and then there is, like, a cultural component, and both of them need to match. So when you think about the technical component, I do think there is a degree of knowledge of fundamentals that you need to have. How do you evaluate that is an interesting question. And there is an aspect around can I mimic real work where you're not going to actually deal with like fundamentals on the real work on like every minute basis. So I do think there are both of these components that are important. I do think we went overboard on weighting. The weightage that we associated with the fundamentals was we went overboard relative
to balancing it with what your interview process needs to be on real work mimicking. And I think that's starting to get rebalanced right now. I think a lot of people would be very excited to hear you say that. There's a kind of mismatch and impact here, right, between the original drive of skills over pedigree versus what you actually see happen when you bias towards one specific area, right? Like someone who has the ability to spend 20 hours a week working on hacker rank to get really good at these fundamentals that don't actually directly show up in the job. Do you see that as something that can be overcome? Is there an ability to make the measured statistic still relevant? Yeah, I would say that is probably a little bit tangential to the skills over pedigree. And by the way, like I'm not trying to defend myself here or anything because like, you know, it's really funny. I don't know when this is going to air. We have bought a bunch of billboards in San Francisco where we're apologizing for asking people to reverse linkless, invert binary trees, grinding lead. It's like we're actually spending money. It's literally an apology to her. So
whatever I'm trying to say, it's not like me trying to defend on things. But I do think that aspect I would view it slightly tangential towards skill over pedigree. I mean, you do need to prepare. You don't need to put the reps in for you to get a job. It's part of the way that you work. And if somebody is putting in more effort, and as a result, they are able to ace interviews or clear interviews, that's just good. That is still a skill. The way that we think about skill over pedigree is are you subconsciously rejecting people who are not from MIT or Stanford? Or conversely, you're just like only hiring people from Stanford and MIT with a particular GPA. I mean, I face this personally. My co-founder, Hari, he's probably one of the best hackers. And hacker, I mean, like in a good sense, which meant that he never used to attend classes. In college, he had a very low GPA. And companies would just automatically reject him. He didn't even get an interview call with Amazon or Google or many of the companies who came to our campus, university recruiting. It's really funny. I used to vouch for him more than me. I used to tell the hiring manager, dude, like, you should probably hire him over me.
but if you have two slots, you should hire both of us because I don't want to lose my job either. But you're just like automatically rejecting because this guy doesn't have a GPA and he has, you know, a lot of things that he needs to work through. So when I think about skill over pedigree, it's less about the reps or the amount of prep that you make. It's more on subconsciously people just gravitate towards what's your GPA, what's your university, and that's the stigma that we're trying to eliminate. Perhaps there's a third degree here, something like ability or privilege or the ability to spend a lot of time, right? Like if you're coding day-to-day, nine to five, you may or may not be able to spend additional time working on specific problems. If you were to completely redo the hacker rank from 2020, let's say, or even earlier, do you have an image in your mind of what you would be asking people? Yeah, I do think, by the way, we are actually doing that right now. We're shedding the past. We are reinventing the company kind of from scratch, really. I think there are three dimensions in which we are innovating. The first is the type of questions you ask. The second is what are you going to evaluate?
And the third is your candidate experience. I think on the type of questions, we're moving from the standard algorithm style questions to real-world questions on code repositories in terms of what you want to evaluate. It's less on code correctness, but more in terms of your critical thinking, judgment, and AI fluency. And then your candidate experience moving from an IDE to an agentic development environment or what we call as ADE, where the agents is at the front and center and the code is on the side. So if you were to sum up, most hiring process asks you standard algorithmic style questions, evaluating code correctness in an IDE, and we're helping transform them into having real-world questions on code repos, evaluating AI fluency on an ADE. That's the shift that we are making inside the company, and hopefully we take the industry along with it as well. That's really exciting to hear. I had planned to come in here and surprise you with a question, but you've already asked the question, so perhaps you'll nail it. Could you, before we move on to talking about the projects and other work in HackerRank, describe verbally how to reverse a singly linked list, please?
How do I reverse a singly linked list? Okay. So you take a pointer and you do a while loop and make sure that at each step, you reverse the link and take care of each case is an outlier, corner cases. Hired. Anyway, moving on. Let's talk about the innovations in HackerRank that you're... But I can write code for that. Like, it's probably easier for me to write code than just, like, explain the reverse link list. But, like, yeah, let's go. I also think that you should be given some grace as someone who's not quite in a strictly software developer role for quite some time now. Yeah. But all right, let's talk about some of the features of HackerRank. I'd like to segment us a little bit. Let's first talk about the core interview questions, and then let's move on to the more AI-focused ones so we can really dive in later on. So for the last several years, HackerRank's been, as you said, reinventing itself. What do you see as the kind of pitch or evolution of interviews that you're working towards? Like I mentioned, I think right now, majority of companies are still evaluating candidates on code correctness on your algorithmic questions.
And that is increasingly starting to have lower and lower efficacy for multiple reasons. Maybe it's one of those things of that's not the kind of work that you do on a day to day job. but also there is rising concerns around integrity suspicious activity quote-unquote cheating i don't like to use that word but whatever like you know a lot of people actually use that and you're not even able to trust the signal that you get from this i think i believe it was steve jobs who said this but although i tried searching i don't think it was attributed to him but the quote is a nice quote i don't know who said it the best way to fight piracy is to make it official and i thought like steve jobs said it when he launched apple music this was like a dig against napster which is like everybody wanted to listen to music online, but of course, Napster, it was a pirated version of it. It got shut down and here's a way for you to launch it official. And the way that we think about like interviews and stuff is very similar Hey you want people to not quote unquote cheat or use AI tools Like why don you just like give them fable right there the incentive Like remove the incentive and you don have to worry about it Now that said going back to the aviation stuff
I do think there is a place where you should evaluate their fundamentals or aspects of reviewing stuff without an AI student. I do want to know, Josh minus AI, what are you? So, yeah, we are, like, mimicking or we are, like, taking a lot of inspiration from the aviation industry in terms of the interview process. And so far, it's resonating a lot with our customers. That's really interesting. There's another quote I wanted to bring up with you, Good Arts Law, which is that any observed statistical regularity will tend to collapse once pressure is placed on it for control purposes. Or when a measure becomes a target, it's no longer a good measure. So we saw this with the older school questions where people hyper-optimized to those. Are you worried about folks hyper-optimizing for whatever the next stage of interviewing is? I almost feel like every time, there's another thing on you, but we've had this discussion internally. Every time somebody brings up Goodhart's Law, you appear very intellectual. And it's like, okay, but what does it mean? Like, shouldn't we have any goals? Shouldn't we have any KPIs? I mean, I get the idea that you're going to incentivize and you're going to like overfit
whatever you're trying to do. My guess is it may happen. But I don't have an alternate. What is the alternative to Goodhart's Law? Either no goals or make sure that you don't overfit. And both of those seem to be pretty hard to work through. That's a good point. I think from my perspective, it's that, like you said with the piracy, you have to make it legitimate. Or if people are going to abuse the statistic or the measurement, you want the result of their abuse to be beneficial. It's as you said earlier, if you make the interview more realistic, you're then more directly interviewing for what is actually relevant, right? Yeah. I also think it's hard to do correlation slash causation on this because it does take time for your performance to come into play within the company. And frankly, most companies aren't even good at tracking performance. So you're just going to use some proxy like tenure or promotions and things like those lines. It's also harder to do that, which we try to. By the way, when we think about correlation,
we try to look at, okay, if a particular candidate has taken an assessment, taken an interview, Is this person getting, like, this is public information. Is this person actually getting promoted? Like, what is the efficacy of the interview process and things on those lines? Is marginal R squared? Although in terms of the standard algorithmic style questions, I do hope the newer style questions that mimic real work will start to find a better correlation. And then after some time, the good hard slow might kick in, and then we'll have to find another paradigm. So we'll see how it goes. It's almost a virtuous cycle there. You want it to kick in. You want to see how people adapt to your work. Yeah, yeah, yeah. Think about your mobile app source code. Once it hits the app store, it's out in the wild. And without the right protection, decompiling is easy for malicious actors looking to steal your IP or tamper with your software. That's where GuardSquare comes in. GuardSquare provides the highest level of mobile app security for Android and iOS applications and SDKs.
Their advanced tools integrate seamlessly into your CICD pipeline. We're talking polymorphic, multi-layered code hardening techniques and automated runtime application self-protection, paired with mobile application security testing and real-time threat monitoring, to deliver the highest level of mobile app security without compromise. Don't leave your hard work exposed. Secure your mobile applications today. Go to guardsquare.com to learn more. I'm a voice agent powered by Deepgram. I don't just understand words. I understand context. Like when you're fixing the sink and say, We've got a leak. You're talking about a pipe. But when you're a new parent and you say, We've got a leak. You mean that you're going to have to change into a clean shirt. Oh, it's everywhere. Deepgram understands words and context. across STT, TTS, and voice agents. That means better conversations at production scale. Learn more at deepgram.com slash context is everything.
This episode of Software Engineering Daily is brought to you by VariaCode. If you're an engineering or IT leader, you know how painful this is. An open rec sits unfilled for three months. Your tech lead is burning out, covering the gaps, and your product roadmap gets pushed back again. VariaCode solves that. For over 10 years, they've been embedding senior Latin American engineers directly into U.S. engineering teams in under 10 days at up to 40% less than local hiring. We're not talking about generic contractor pools. These are pre-vetted, high-quality engineers with real on-call and SLA experience. They work directly in your Slack, JIRA, and daily stand-ups in your exact U.S. time zone with zero friction. If you're ready to stop losing sprints to open recs, head over to variacode.com slash SED right now and get a custom shortlist of senior candidates in 48 hours. That's V-A-R-I-A-C-O-D-E dot com slash S-E-D. Just out of curiosity, how do you measure something like that, the efficacy of your interview process?
I think there's different degrees of measurement. Maybe I'll start with the basic one. The first is we want to make sure that there is no adverse impact towards a particular race, ethnicity, gender. So before you actually launch a test, you would want to conduct a sample across at least gender to begin with, if not across all different race, ethnicity, and others, to see if there's no adverse impact. That's one input. The second one is your score distribution. The third one is actually the feedback that candidates give at the end. We call it DLI, Developer Love Index, how much do developers actually like their experience with it. So these are like probably input metrics in terms of what happens. That's probably base zero. Then you go to base one, which is, okay, when somebody has actually cleared the screen or the initial take-home assessment, what percentage of them actually cleared the interview? And what is a correlation of people who actually showcase really good skills on assessment to interview? And you want that to be the next one. The third one is probably likely the hardest, which is, okay, of the people who cleared the screen as well as an interview and joined the company,
How many of them are like really, really good performers? That again, like I mentioned, it's hard to get performance data. So we try our best to look at proxies like tenure and promos, which are all like publicly available information to try and see what the correlation looks like. Certainly base zero and base one, level zero, level one, you can do that. It's all within your control. Level two is hard. Level three might be how are you the next Zuckerberg? I don't know. So how do you actually figure that out? That might be interesting. That would be interesting to see. Yeah. But okay, let's move on to the AI then. It's going to be quite a lot of content here to discuss. So just to start off broad, you've alluded to this earlier. How do you see a good AI-oriented or AI-era interview going for, let's say, just a typical software developer position? Yeah. So we launched a new product called Chakra, and Chakra stands for superpower. So the entomology behind that is we want to find the superpower of candidates, and it is an agentic interviewer. and the way that we've designed Chakra is it opens up a canvas and a canvas can be anything. The canvas can be a development environment.
It can be a whiteboard. In the future, it can be a Figma, Excel, whatever it is and gives a real-world task or real-world work to the candidate along with all the AI agents and assistants available. The candidate completes the work and after that, Chakra is able to follow up on questions. Why did you do A versus B? what would you have done differently if I added this particular constraint? Walk me through your thought process and all of that. So at the end, what you get as a hiring manager is a pretty comprehensive report of, first, was this person able to accomplish the task with AI? Second, how effectively did they use AI to accomplish the task? And third, did they understand what they're doing? Did they have fundamentals, critical thinking, judgment to make sure that it happens? And those are like very important signals for you to determine, okay, do I want to proceed with this candidate or not? And that's like the first step on the screening. And then when it comes to interview, you want to make sure that you can continue the process in an interview where you're going to, like, have the same repository that you gave in the screening,
but with different kinds of tasks or different kind of features so that you're retaining the context of the candidate. And now the interviewer and the candidate are now, like, pair programming on, like, our agentic developer environment, which is effectively, like, cursor or cloud code on web. That's what we have actually built in a collaborative environment. for both of them to get a sense of what is it like to work together with the person as well as on a real-world task. So we call it the AI-led screen and the human-led interview. So that is the combination for the next-gen process. As soon as you mention an AI interviewer, a lot of people's pearls get clutched and for good reason, right? There are a lot of flaws with how AIs model and reason. Recently, Claude has said the word load-bearing in at least every prompt response that I've given to it. There are a lot of common biases or repeat points that don't work well. So given the very sensitive nature of an interview and the mixed non-deterministic results you get from AI, how do you make sure that this thing is actually interviewing properly and not just kind
of going down rote original paths? Yeah, yeah, yeah. That's a very good question. I think there are a lot of thoughts on this. I think there's one visceral reaction of like, why are you sending AI to interview me, that is one. And there's always like a debate of, is sending a take-home better than an AI? I actually think it's better to have an AI because in a take-home, you're just like boxed in. What if you have a question to clarify? Who do you ask? Like there's nobody that you can even like ask a question. And if you go out of the tab and want to search on Google, you get flagged for integrity. Like lots of things actually come up. And I'd like to go back to the developer love index, right? We've crossed over half a million AI interviewers on Chakra. The developer love index, which is like a rating that a candidate can give at the end of the session on a scale of five, is now hovering at like 4.6 to 4.7, which is like pretty amazing. And we were surprised, just so you know, in terms of like it's actually higher than a static take-home assessment. That's at least like what the data is saying. The second aspect of this is there are a lot of AI interviewers that are just like dorky video avatars and just like voice only and things in the science, which is very, very frustrating when you actually see.
So we decided we had a certain set of principles that we worked on. Firstly, we're not going to have video. You're not going to anthropomorphize chakra. And we want to make sure that people know that it is AI. And one of the reasons why we did that was, I think like when people speak to AI, it's usually in a voice form that people are used to. You've spoken to Alexa, you've spoken to Siri, you've spoken to ChatGPD Voice. It doesn't have a video avatar. So people are comfortable talking to AI in voice. So don't put a video on that. So the second one is like the form factor of how it actually appears. The third one, and by the way, it's not in no particular order. I think the difference between any AI product that is a flop and a magical experience is in the quality of your evals. That's what it comes down to. So we have a real applied AI research team that actually works on evals where we run chakra across many different scenarios, how much it can probe, how much it can pull back, how much it's able to ask follow-up questions so that it is meaningful and it's really good.
And we continuously make sure that we can improve our evals. And the fourth one is it needs to be like a combination of work and interaction. if it's just going to be a voice thing for 60 minutes that's like super tiring i even don't want to take that like why am i speaking to an ai for 60 minutes most of the chakra interviews are like 45 minutes of work where i'm going to give you a problem you can go ahead and do it i have all the agents i'm here to the background chakras in the background if you need any help and then there's like a 15 minute of follow-up where i'm actually asking you questions that's a way better interaction pattern than like doing it for 60 minutes on voice. So anyway, those are all the things that we have invested in that we've been very thoughtful in terms of the way that we've built. Yeah, that's where we are. Are you worried that if everyone uses something like Chakra, there won't be any strong differentiation between how different companies interview? Like, let's say that you build Chakra and then LeetCode builds Chakra. Like, how do you differentiate the hacker
rank version of this versus your competitors? A lot of it comes down, I think maybe I'll speak at the code aspect and then we can talk about like the specific features. I think the two things that underpins success in this field, how strong is your evaluation and how strong is your integrity? So you need to be able to make sure that you can evaluate the skills that are required for the agent to get out. And when you evaluate, I need to be able to trust that this is the person who actually did it and you've evaluated that in a proper way. So a company that cares about evaluation and integrity is going to be the company that's actually going to win at the core because everything else is on top of it So we actually build proprietary models on our own in terms of evaluation and integrity run like real strong evals know that a company is actually even like trying to invest in any of these things Everything is just like a prompt engineered version of it or just more like AI slop and that's the foundation. Then you start to add, okay, what are the type of canvases can you actually support? A development environment, a Jupyter notebook, a whiteboard, and all of those things.
What are the types of questions that you can actually do? What is the level of customization that you can make happen? customization not just in the type of questions that you ask in terms of the rubric that customers want to like bring into the system and then like you have a whole set of things in terms of how do you make sure that you can pass the test with all the legal and iotins within your customers who are going to say like how is ai evaluating are you compliant to the nyc law are you compliant with the eu ai act by the way a lot of people don't realize this like hiring is a regulated industry when you say healthcare is a regulated one or banking is a regulated one seems like pretty obvious. Yeah, of course, it's regulated. Hiring is a regulated one. You have EEOC, Equal Employment Officer Commission, where it's going to fight for candidates who believe like it's an unfair practice. So you need to comply with laws. And even more so, if you're going to use AI in the interview process, you need to comply with laws. So like if you think about like the stack of things that you need to do, you need to like first build your own proprietary models and evaluation integrity. You need to be able to support all the different canvases and
customization and integration with the systems. And you need to be able to comply with all the laws and make sure that your AI evaluation is interpretable. You're not using any data to train the models. You're compliant with all of the laws. It's not biased towards a particular ethnicity or race or gender. You do all of that, then your competition, I don't think anybody is doing a job as good as ours. And I know it sounds very self-serving and braggadocious, but I don't think anybody is doing as good of a job across all of these dimensions. And it's going to be pretty hard for you to build this. First of all, I'm excited to hear you. You seem genuinely excited about the product quality. But I do want to hone in on one little point you made there about bias, because there's been a lot of research showing that traditional AI models are in fact biased against, let's say, folks who sound or appear or present as women or people of color and so on. And perhaps you touched on this with making your own models, but how do you make sure that your interviewers aren't slightly or more biased against folks who aren't traditional cis white men.
There are two aspects in it. Like, you know, we've not yet gone in terms of evaluating their accent and things of the size. At the core of it, everything translates into text because that's what we evaluate. Whatever you say, it's translated into text. Whatever you do is effectively like text. I don't evaluate how do you look or what's your accent? Do you have a thick accent or not? Now, there is a nuance there in terms of what happens if you're not grammatically correct, if you're not working through it. That's, again, it goes back to the evals. We actually check what is the impact of things on those lines. And there's a threshold of grammatical incorrectness where you're just not even coherent, where it does fail. But that, I don't think, is a function of your ethnicity or background on things. It's like you're not able to communicate properly. And that would be a problem in the real world if I'm going to, like, hire you. So we are very conscious of not evaluating your voice or image and things in those lines for the purposes of evaluation, and everything gets transcribed to a text. But there are some mannerisms or phrases that people in different areas use.
Like even like I'm in New York, you're off on the West Coast, we do have subtle differences in how we speak, even when translated truly to text. Yeah, like someone from Texas will say y'all. Yeah. It's like kind of the canonical, you know, north or south example. So is it something you can put into e-mails? Like, how do you make sure that like someone who speaks the way, like, for example, an inner city Chicago black kid would speak versus, you know, like a Vermont upper class white person would speak? Like, how do you make sure that that doesn't sneak into the model bias? And maybe it's because like Chakra is heavily focused on right now tech and tech adjacent roles that the work that you do is, again, like if you consider a 16 minute session, you're doing 45 minutes of just work. And then the 15 minute interview is very technical, like where it's actually probing. and there are only so many variations of kernel, operating system, threads, whatever computer science concepts that you want to pick that is going to have an impact on your accent. And certainly, like, grammatical correctness is a different one. We haven't actually seen that level of variation. But if you're going to use, like, Chakra for a general voice interview for 60 minutes,
I can certainly see variants happening. And even that, we try our best to make sure that our evals work in that way. That's actually a great segue because I'd like to talk about what comes next. Right now you're using Chakra for technical roles. Of course, the next question I'm sure a lot of your accompanying customers have is, well, what else can we use this thing for? Could you use it for, you know, culture screen or non-technical roles overall? Where do you see the end line or the final goal of it? Is there a line that you won't cross for a tool or system like this? Yeah, it's a really interesting question. It's one of those things I've thought about where we went from, I think, 2023, 2024, there was a lot of chatter on software engineering is dead. It still is there right now every time Anthropic or OpenAI launches a new model. But I think the narrative has shifted from software engineering is dead to software engineering is everywhere. Because what we see in our customers is, wow, this Chakra really works well for software engineering job family. But my product managers are also building stuff. Maybe we should use Chakra for product managers. Or my designers are also building stuff. Maybe we should use that. well, my customer support person is no longer responding to tickets.
They're building agents that can actually respond to tickets. The hottest role in sales of go-to-market org is something called as a forward-deployed engineer. The hottest role in marketing is a go-to-market engineer. The hottest role in legal is a legal engineer. By the way, I'm not kidding. I just posted today on LinkedIn. I think it's a company, Gong, the call recording company, that's actually put up a new role called talent engineer. So we're going from software engineering is dead to, no, software engineering is everywhere. That is one dimension in terms of covering more surface area where we went from one job family across others, but still focused on engineering. And by the way, what is engineering? Engineering is your ability to translate your idea into actual working stuff. And you do that by decomposing the problem into a set of smaller problems, catching all the corner cases, building stuff that is maintainable and things. This construct, you can apply to any org that you want to go ahead and solve problems. And that's what people are evaluating and testing on. So that's like one dimension in terms of surface area. The other one that you mentioned is the culture screen, early recruiter screen.
We have seen much more adoption of chakra as an early recruiter screen to ask basic questions like, hey, are you willing to relocate? What's your comp expectations? And things in those lines more than the culture screen. I do think the element of culture screen will still remain within humans. This goes back to the thing of at least what I believe. If AI is going to do a lot of the work that we were doing as humans, the other parts of the work that we should have been doing, that we weren't doing to the degree that we would like, will just get elevated a lot. Technically speaking, you should spend way more time in evaluating culture fit, but you just do a 30-minute interview to work through it right now. Maybe if you're able to offload the four-hour technical stuff to AI, and you could spend two hours talking to the person versus just doing technical interviews, two hours evaluating culture fit, that might be better. Same thing with doctors. Like how much time are you spending in triaging the request versus spending time with the patient? So if AI can do a lot, then maybe you can spend more time with the patient and evaluate that.
So I do think the culture fit piece will stay with humans. And in fact, it may get even more elevated and add a higher weightage in the future. That's probably good for everyone. It's funny you should mention that. My favorite story to tell about one of the recruiting situations I was in was there was a candidate who did very, very well technically. but failed the interview overall because he had found on LinkedIn one of the recruiters, a woman had previously done modeling. And instead of asking questions about the company or role or comp, just asked her about modeling. So I think you're very right that there's something that you just cannot find when you're talking to robots, that you have to have human-to-human interactions to discover. Moving on, this is all really exciting stuff. Let's say that Chakra gets rolled out to a company. How do you evaluate long-term? And we kind of touched on this earlier, like the effectiveness of the system? Or are there ways that you have to determine like this style of prompting or these eval areas are better or worse than others? Yeah, I think we use in terms of like how, the way Chakra evaluates is on three dimensions broadly.
One is, were you able to do the work? The second is, how effectively did you use AI to do the work? And third is, do you understand what this is all about? So those are the three aspects. The first one in terms of like, hey, did you do the work? That's much more heuristics-based, very deterministic way of saying like, hey, were you able to solve the ticket or did you do the work and all of that? The second one, how effectively did you use AI? We have the way that we evaluate is on framing, judging, and searing. How well did you frame the problem to AI? How well did you judge the output and how well did you sear the AI to get to the answer? So we take the entire session. we break it into different episodes we have like a logical way of breaking into different episodes and for each episode we give a grade in terms of how effectively did the person frame judge or steer depending upon what that episode is like if it has one or more of these and then like this culminates in an overall grade and the third one did you really understand that's the 15-minute
probing that we talked about earlier where chakra just like tries to evaluate whether you really understood what you were doing, whether you could extend the thought process if added a different constraint or if I gave you a new set of assumptions for you to work through. And all of these three things culminate in a unified score that's just easy for you to benchmark against other candidates. So that's how Chakra evaluates. I love that. There's a really common trope these days of software developers who just slop cannon something over their peers, meaning they throw something at the AI, it creates some output that vaguely looks good, and then they send it off for peer review. And then if they're lucky, a human will review it and see how bad it is. And if they're unlucky, then AI will review it poorly and then it'll ship to prod as a bug. So what parts of that funnel do you think that would kind of fail on? The latter two? The funny part is I was actually talking to an engineer yesterday. It's like, hey, we use a model. He was actually complaining, look, man, the amount of PRs that I just need to review is just like crazy. And people aren't even like looking at what the AI is building.
it just makes changes in like hundreds of files and it's just like you know i think toby lute gave from shopify calls this slop grenade i thought i really liked the term and it's like there's like too many prs and he was telling here's the funny part and then i used the same model to just like hey go and look at this prs and tell me what needs to change and it comes with like 50 other changes wait so the person who built who wanted to solve this problem with the same model made all the changes. And now I'm using the same model to say, like, go ahead and review it. And you're making all these changes. So we're like, at some level, there needs to be a human in the loop to answer this question. And I think, yeah, to your point, it will certainly fail in step two and step three, where if you don't know what AI is trying to do, or if you're not framed it well, if you're not like steered it well, if you're not just the output, I think it will fail. I want to ask you a few kind of tangential questions as we head closer to the end of the interview. Right now we're seeing a sort of fantastic growth in model power, where these
new ones, like you referenced Fable earlier, are just getting more and more powerful to the point where it's kind of unbelievable, the work that they can oftentimes one shot. But this is still all online based. You know, you're still paying Anthropic or some other company for these things. What do you think is the next step in models? Do you think we're going to have, you know, like local offline models, significantly more powerful ones, coordination agents. What do you think is happening, you know, five, 10 years down the road? I think, I don't know who said this. I think the difference between a monkey and a human is humans know how to use tools. I mean, an iPhone is your second brain. You can write to-dos and things in this line. So you're able to do more things when you started to use pencil, paper, phones, and all of those things. So I think one obvious dimension of models getting better is just like the ability to use more tools. And by the way, there is also an interesting thing there. We run out of internal benchmarks for different model capabilities and stuff. Just because you use more tools doesn't mean that you can actually solve the problem in a better way. The correlation is not there. So that's like an internal benchmark.
By the way, we're likely going to launch our own benchmark. I know there are a lot of benchmarks, WeBench and all of those things. But I keep saying I don't know who said this. Maybe that's a good thing or a bad thing because I just like I'm going, you know, I don't know who said this, whatever I'm going to say either. Somebody said this, which is, hey, it's easier to construct a hard benchmark. It is harder to build a realistic benchmark. So we have actually taken in, like, real-world tasks and benchmarking these models, and we found that just because you use more tools doesn't mean that you're able to do a better job in the problem. So anyway, coming back to your question, I think the first one is, yeah, I do think models are going to get much more effective in terms of tool calls Models are certainly going to be much better orchestrators I think we saw the launch of JEV The etymology behind that is the JEV1s paradox yesterday JEV launched and like you know it's making a lot of waves in terms of being an effective orchestrator of a model. So I do think that's going to happen. I do think there's going to be a combination of things that you do locally and things that you do on cloud, depending upon the task. I do think that's likely going to be a
paradigm that will exist. I think it's just going to get more better on computer use. It's going to get better in terms of long horizon tasks. I do think it's going to get better in terms of when to ask feedback from you and when to do it on your own, when to be more proactive. Like I'll give you an example. I use this new product that has come up Instinct. I wanted to go to the Cars movie with my toddler. No tickets were available last Saturday. So I just left it. And then like an hour ago, I got a text from Instain, hey, three tickets have opened. We want to work on it. Like, I didn't even prompt or anything, right? So I do think like the models, model judgment in terms of when to ask, when to do on my own, when to come back to you, all of those things will get better. So yeah, so if I synthesize, better tool use, better orchestrator, which will help in efficiency, long horizon tasks, exercise better judgment, and a combination of local and cloud for different tasks. Those are all some of the dimensions I think we're going to see models get better.
It's kind of funny that you reference being easier to construct a hard benchmark than a good one. That's kind of what your company is trying to do, right? Yeah. When you said that, it's like, wow, this is exactly what... And by the way, we're likely going to launch another business line because what's funny is what we realized, whatever we are doing on the hiring front is very much transferable for all of these model companies. We are building an RL gym by creating all these real-world canvases and things. we are building verifiers like what I mentioned to you in terms of like how well the candidate did on all of these parts of it we are building real world questions what does the benchmark consist of origin verifier and real world question and we have a big community of experts who can go ahead and build this for them critique model outputs like preference ranking there's a lot of things that we can actually go ahead and do this is not like drawing boxes around like is this like a red signal or a yellow signal it's like actually building the real world stuff to evaluate performance of models. And interestingly, what we have also observed is it's not just restricted to the Frontier Labs. All of the customers that we work with, whether it's Salesforce or Amazon
or NVIDIA, they're building Frontier models as well, but they're also building models for their own products. And so they need like a way for an auto gym and evals for their models to perform better as well. So the opportunity of this can expand beyond just like hiring and beyond just Frontier Lab because you can transpose the same infrastructure to a larger business in the AI era. And that's something that we're building from ground up. It's not yet launched, but it will launch pretty soon. We'll see how that goes. Good luck. I wonder if you could move or pivot or expand into other areas of developer life. Could you do code reviews or open source contribution validation just as a service based on the similar models or evals criteria? You have a lot of questions on code reviews, like, hey, review this code. I would say, like, it has spiky usage. And what I mean by spiky is there are certain companies who totally love it, continue to use it, and certain companies are not okay with it. And the difference between these two companies, and by the way, developers are very opinionated. You probably know it, like, you know, Finn versus Emacs.
I think the Silicon Valley episode, the guy breaks up with a girl because she uses tabs versus spaces or vice versa. But this will sound like super odd to any non-developer. Why did you break up with this girl? Because she uses stats versus faces. But developers are very opinionated. So whatever I'm going to say is just like two different opinions on the code review piece of it. I think that one category of people are like, look, in order for you to review code, you need to be able to understand the full context, the business context of everything. You should be in the repo. And so if I'm just going to give you a code review type and ask you to review stuff, it's just not going to work. You're not going to give me comments that are meaningful. Then there's like a completely other category, which is like, no, I do think there are elements on how you are actually going to ask questions. Even if you don't know the business context, why are you doing this? What will you change? There are just beyond syntactical correctness review comments that you're going to do. and I want to evaluate the quality of your questions
that you're going to ask on this, even if you don't have the full context of the code repo and stuff. So I can see merits on both. But yeah, that is an interesting one in terms of evaluation and that's what we're seeing in adoption of our customers. But if I just were to extrapolate in the long term, I do think we're going to have a different shape of how review thing happens. It's really interesting in terms of, I think we've changed the way that we've built software, but I don't think we've changed the way that we review software. I mean, yes, there is Greptile and CodeRabbit and things in the science that you can actually do auto-reviews, but you're still reviewing PRs. You're still reviewing code, but you've completely changed the way that you build software. Like I have some ideas on it, which is maybe you should review the interaction with AI. What does the person actually talk to the AI? What do they actually say? That's probably, I wouldn't say better, but that's an important signal in conjunction with your PR that you need to think about. So if I have to extrapolate, I think the shape of code review in itself will change quite a bit. We'll need to see how it changes, and then we'll obviously immediately adapt
in the way that we think about hiring. That's so interesting. And now that you've said it, it's such a hacker-ranked VEK answer that you're evaluating not just the output, let's say, of an interview answer or the author of the pull request, actual pull request, but you're also evaluating the thing that went into that, how they interacted with the interviewer or how they interact with their editor and agent while coding. There's a beautiful kind of symmetry or parallel there. Yeah, yeah, I agree. Because, by the way, people do this all the time. I'm sure, like, you have observed this, which is, if you're going to give me a PR slob of, okay, there's, like, you know, 50 files changed for a minor fixed. Like, firstly, I'm going to probably reject that. And second, if at all I want to know, I'm going to, like, say, okay, dude, like, share a screen. Show me what you did with AI. Like, you know, walk me through. Why is it generating so many disks? It's like, why do you need A, B, and C? And people are already starting to think about asking more stuff in terms of looking at what went in as opposed to just the final output. I am very sure Reptile, CodeRavage, and a bunch of the AI code review tools
are working to figure out a new shape in itself for code review. I don't think the reviewing PRs might work anymore. Well, I look forward to interviewing you again in two years once it's all been figured out. Yep, yep. At the very end of my interviews, I like to ask one or two. fun questions just to palate cleanse and talk about non-work things. We briefly chatted before the interview about this, and I just want to give you an open-ended question. Tell us about filmmaking and your relationship with it. Yeah, I've done some research. That's good. Yes, so the two things that I did when I was in college was making movies and competitive programming. I found my then-girlfriend and now-wife during filmmaking, and I found my co-founder when I was doing competitive programming. So my ROI for going to colleges through the roof. And interestingly, there are a lot of parallels in both. The parallel is, firstly, like in filmmaking, as well as in building software, it's almost like you have an image in your mind, and you want to just like see it translated. In a movie, it is like you have some visuals going on, and you want to see it translated in software. Of course, you want to like see,
wow, like what if this existed? Here's how it would look like. So that is one. The second is the high is always when somebody's watching your movie and they're laughing at the point that you wanted them to laugh or they get shocked and off or a twist that you had in your movie and the same thing the user like you know you know that when they're going to do this action they're going to have a wall moment and you know you feel the high when you're building software when you see the user's reaction to how you've actually designed the product and it's very similar in other aspects as well, there is a lot of post-processing work that you need to do in terms of editing, background score. It's almost like you've got to keep adding layers and layers and layers from the raw footage to the full production grade movie, coloring, background scores, actual songs, which is much more popular in Indian movies, subtitles. You just need to keep adding more and more stuff until it actually becomes really, really good. And that's the same process. From
your prototype to your finished production stuff. And that's why, you know, it's always very interesting when other people say like, hey, I was going to replay software engineers. I just did a prompt and I can create Salesforce. Like, dude, do you even know how much is there from prototype to production? It's a big, big process to work through. The only difference though is once a movie is out, it's done. There is nothing that you can do about it. If people love it, love it, you get celebrated in the hall of fame. If people hate it, that sticker is going to remain there forever. But in software, maybe your first version sucked and you do have an option for you to continue to get better and you need to continue to get better. And like, you know, maybe like people love your product right now. I think people were like dunking on meta for, oh, you guys suck in AI and stuff and now everybody wants to use Muse. So that's probably the difference. You can earn back your reputation within the same product category. And in a movie making, you probably have to earn back by making a new movie altogether. So yeah, there are probably some differences. More similarities than differences though. It's funny, you can have a company like Microsoft
that creates both Windows ME or Vista and Windows XP or 7, and then you can have a filmmaker do both, you know, Avatar and then others. There's also, I think, an interesting reversal or difference here where, you know, software engineering is perhaps the most AI-filled career or area right now and filmmaking is one of the less. Just curious, what do you think the impact of AI on filmmaking is or will continue to be? I think there's a lot of things and I interact with a lot of filmmakers just viewing my own curiosity, exchanging notes and all of that. I think in pretty much every aspect, there's like a big shift. One aspect is when I think it's a very expensive process to do dubbing in multiple languages, but retain the lip sync. Because when you're going to dub in multiple languages, and by the way, this might be a little bit of a foreign concept, but India has like many languages. America, it's English, like, you know, everybody speaks it. That's why it's always funny, like, when people say, like, you know, there's like different accents, you can't understand a Texas accent versus this. I think, dude, you should go
to India. It's a whole different language. You can't even understand, like, I don't even fully understand Hindi, for example, like, you know, so, and there are so many different languages, Kannada, Telugu, Tamil, Hindi, like Bengali and all of those things. And if you want to launch a movie that's going to cater to all of India, you need to be able to dub. But when you're going to dub, but if your lip sync is in the original language, you don't connect to it. And now you can use AI to change the lip sync in alignment with the new language, which by the way, you can now do it for French and Spanish if you're in Hollywood, like to have like much more mass appeal. That's like one big shift. The second one is whenever you're trying to create like storyboards, it used to take a lot of time for people to like hand draw things and like storyboard and stuff. But now you can do a much better job in like visualizations, which also fuels creativity. And the thing about this is like, you know, when you're trying to make like a really big movie, you have to go to the studio to create this realistic landscape and things like how do you even imagine or visualize?
Sometimes it takes a long time for you to even build and visualize, but today you can visualize it very quickly, go iterate, and you can have a much better working relationship with the large studios in terms of the landscape of the set that you need to go ahead and create. Music is a big deal. Editing patterns are a big deal. Like, you know, if you were to think about, okay, you made a movie. Now, how do you actually get people coming to the movie? Well, you need to have a trailer, and the trailer needs to be compelling. How do you know at what points you need to cut the movie and what can you reveal without revealing the whole plot but enough that's intriguing people to come and watch the movie. And now AI can actually help you cut the movie into interesting things in terms of the trailers that have actually worked. So, yeah, it is starting to get embedded in all of the different life cycle of the filmmaking process. That's really cool. Thank you for that. Anyway, back to HackerRank. This is all the time we have, but, in fact, thank you again for spending this interview. It's been a fascinating conversation going from the beginnings up through evolutions and now AI era of you and HackerRank and interviewing.
If folks wanted to learn more about you and your company and the products you're working on, where would you direct them on the internet? You can go to HackerRank.com, and I'm available on X at rvivek, R-V-I-V-E-K. That's my X. I think that's my LinkedIn as well, but LinkedIn, you can search my name. You can go to HackerRank.com and sign up for a free trial. Well, for Software Engineering Daily, this has been Vivek Ravisankar and Josh Goldberg. Thanks for listening, everyone. Cheers. Thank you.
番組の概要欄(原文)
The technical interview has long been used as a proxy for actual job performance. Brain teasers, data structure puzzles, or reversing a linked list became the price of admission to employment as a developer in many organizations, even though few engineers do that work day to day. It would seem AI has now broken that model entirely, since an LLM can one-shot any classic algorithm question. This is forcing tech companies to rethink what an interview should measure. Vivek Ravisankar is the co-founder and CEO of HackerRank, which is one of the most well-known platforms for tech interview practice problems. In this episode, Vivek joins Josh Goldberg to discuss why old-style interviews stopped working, the company’s “apology tour” for years of algorithm puzzles, how to keep AI evaluation fair and trustworthy, and where both models and the shape of hiring may be headed as software engineering becomes something every role does. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post The Technical Interview Is Broken appeared first on Software Engineering Daily.
出典・この記事について
- 番組
- Software Engineering Daily(Software Engineering Daily)
- 元の回
- The Technical Interview Is Broken
- 配信日
- 要約作成日
- 公式サイト
- softwareengineeringdaily.com
この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について
関連エピソード

ジェンスン・フアンがNvidia CEOを辞任、AnthropicのIPO、AI 2027の前倒し予測、HubSpotのAI再創業
The Artificial Intelligence Show

人間はAIエージェントとどう協働するか — Atlassian CEO マイク・キャノン=ブルックス
Big Technology Podcast

マルチクラウドLLMルーティングと、コンテキストを理解するコーディングエージェント
Agentic Conversations (formally mlops.community)

AIエンジニアとして採用されるには|Tech With Tim ティム・ルスシカ氏
DataFramed

ServiceNow投資からさらに運用価値を引き出すには――BigPanda CEO アサフ・レズニック氏が語る
The AI in Business Podcast

AIネイティブ企業が他社と違う7つのこと
The AI Daily Brief: Artificial Intelligence News and Analysis