Claude Fable 5's first full day. The hosts recalibrate live — benchmark asterisks, invisible production nerfs, and the financing arithmetic underneath the race — then Geoffrey Irving and Daniel Murfet choose the show to launch Sequent, a major new nonprofit betting that alignment needs theory plus automation before superintelligence arrives, and Rahul Sonwalkar explains how Julius survived every wave that was supposed to kill it.
EPISODE 2026-06-10
AI:AM LIVE — June 10, 2026
Claude Fable 5 launch-day reactions — invisible nerfs, Opus fallbacks, and compute economics — then Geoffrey Irving and Daniel Murfet launch Sequent, a new nonprofit betting alignment needs theory plus automation before superintelligence, and Rahul Sonwalkar of Julius on harnesses, token maxing, and the coming agent economy.
Episode timeline
Opening — Fable 5 launch dayA historic-day recalibration: Fable 5's benchmark asterisk, the invisible safety layer tripping on production systems, and Prakash's case that Sam Altman's real recursive self-improvement loop is in the financing.
Fable 5 is here — and the recalibration begins. Nathan called it "a historic day" requiring "a pretty fundamental recalibration" — the first model he's willing to become a hybrid with on work idiosyncratic to him, with task scope (days-to-weeks-long projects) the headline change. Prakash's benchmark teardown found an asterisk: Anthropic benchmarked with an automatic fallback to Opus 4.8 when Fable declines — worth roughly 2-3% and ~1.5x the compute — which he read as the business team overriding the researchers. Both noted the price came down substantially from the original Mythos preview, and Nathan predicted users will consciously ration "when they're going into the true top model."
Introducing Claude Fable 5: a Mythos-class model that we’ve made safe for general use. Its capabilities exceed those of any model we’ve ever made generally available.
This is a super exciting release - Claude Fable 5 is the same underlying model as Mythos but with added safeguards. The benchmarks are great and it's SOTA on everything by a margin but I'll add that *qualitatively* also, this is a major-version-bump-deserving step change forward Show more
@claudeaiFable 5 is state-of-the-art on nearly all tested benchmarks, with exceptional performance in software engineering, knowledge work, scientific research, and vision. The longer and more complex the task, the larger Fable 5’s lead over our other models.
The safety layer is the story: invisible nerfs in production. Prakash spent the night testing the line where Fable drops to Opus 4.8: the moment a task touched his production database, security keys, or production review, the model silently downgraded — and restarting the conversation without the production context resumed work. He argued the publicized ML-research refusals are "just tip of the iceberg" (finance, QuickBooks, and Salesforce work will trip the same wires) and framed Fable as "a research release, almost a preview" with gates that will be progressively removed. Nathan marveled at the mechanism — feature detection and steering vectors in production ("I'm old enough to remember when sparse autoencoders didn't exist") — and flagged the ML-research ban as "the core political economy question."
Interesting tidbit from the Mythos/Fable system card: Anthropic are invisibly nerfing any requests that target frontier LLM development.
Labs starting to pull up the ladders on the ability to diffuse AI was inevitable. Doing it without telling the user is misaligned.
When Fable 5 is used for frontier LLM development, it does not notify the user and instead limits the model’s capabilities through methods such as prompt modification, steering vectors, and PEFT. Anthropic estimated that this would affect approximately 0.03% of traffic.
This prompt I tried with Claude Fable 5 has been flagged as a biology security risk! What an incredible disappointment this has been @AnthropicAI! This is sad! “What are the top 10 unanswered questions in cancer research that, when answered, could lead to potential treatments?”
Compute economics and the lab feuds — the RSI loop is in the financing. Prakash's thesis: Google is paying SpaceX roughly 6x what CoreWeave charged OpenAI last year, so by over-committing early — to near-bust levels — Sam Altman locked in an ~88% cost advantage: "he has his own recursive self-improvement loop in the financing." Anthropic, leasing late at 5-6x, may effectively be months behind on deployment despite a capability lead. Nathan's counterpoint turned elegiac: the founders who started together now "hate each other" — "if we are part of a simulation being run for entertainment, the plot is pretty obvious… Greek tragedy dynamic." On third-entrant odds: Elon repeatedly "makes the capital window," and if physical limits bind, the builders collect rent off the labs.
Here is our current plan for OpenAI: openai.com/index/built-to…
Planned-but-cut from the opening: Ethan Mollick's "commissioning shift" essay, the locked-out-biologists framing (replaced by Prakash's first-person production-lockout experiments), and the OpenAI-vs-Anthropic theology debate. The planned FrontierCode/Vending-Bench segment was folded into the Sequent interview instead.
TranscriptAuto-transcript, lightly cleaned · timestamps jump to YouTube
0:24With the two hosts. Good morning. It is Wednesday, June 10, 9AM Pacific time. Welcome to AIAM. Nathan, hello. » Good morning, Prakash. It is a historic day, and excited to try to make sense of it with you. » Indeed. » Fable is here, and I think, it really calls for a pretty fundamental recalibration. There's so many different angles. Something like this is really a a historic artifact, a historic threshold across. I think we're gonna see it as. And I think we should all kinda take a minute at least to reflect on like how it should change the way we're working, thinking, planning, planning horizons because I don't think it's gonna be their last jump of this scale either. I really just keeps working. So for me, the biggest thing so far has just been the quality of work that it does on the things that are most idiosyncratic to me. It really feels like not a perfect by any means right as me companion, but something that I'm kind of willing to become a hybrid with in a way that it hasn't, risen to in the past? » I we we've had a lot of reactions on the timeline, over the last, twenty four hours or so, so since the drop. I think just to take it off the top, the first was obviously the benchmarks themselves. The benchmarks are basically state of the art in multiple vertical categories, almost every category that they've chosen to disclose. One note on the benchmarks is that they have chosen to disclose the benchmarks with a fallback to Opus 4.8. So in every test it tries Fable and then it falls back to Opus 4.8 if Fable is not able to respond. I think this basically increases
2:55the benchmarks by about two or 3%, which is quite significant. And there are some complaints, I think online that this is slightly misleading. It's also slightly misleading in the sense that the amount of compute used for answering these questions is effectively one and a half times. Roughly, you have the entire Fable compute and then when Fable decides not to answer the question, falls back to Opus, and then you have the Opus's compute as well. So you actually have additional compute being applied to these problems. Having said that, sometimes the researchers are very clear on what the benchmarks are, but the business team feels the, I think, economic pressure and the strategic pressure to show better benchmarks, which are really kind of a rough guide to what the model does. And so we saw this at Meta as well before. And so you see Anthropic falling prey to this. So you can definitely feel that they see that they have that pressure for performance right there. Besides that, we have been seeing a lot of interesting commentary on the timeline. Nathan, what has struck you so far from just other people's reactions that you've seen? » Well, the scale you know, at Recursive not long ago, the consensus was that the meter graph is kind of hitting its end because we just can't create tasks this big. And I think the really striking pattern on the, you know, in terms of what has kind of broken through and caught people's attention is just the scope of it so often. You know, these are clearly tasks that would take days or weeks worth of work. And the reliability I think is still kind of unknown. You know, obviously people are spinning these things up, like seeing that the demo works being totally amazed and there hasn't been time yet to find all the rough edges. I don't think so. Clearly there's a little bit of a euphoria warm glow, but the scope is just vast, you know, to create these worlds with physics, with, you know, trees made of math and to do it at this scale with this kind of ease is I think the most remarkable thing. It really does suggest that
5:27a different way of at least digital work is going to be possible. And the word pressure that you used, I think, is going to apply all over the place kind of now because it seems like there is going to be a pretty high price attached at least relative to what people are used to with the $200 a month. You commented just before he came on that they managed to reduce the price substantially from the original Mythos price that they had posted, which I thought was a great observation. And does show, you know, there's probably gonna continue to be downward pressure on prices for a long time just as there has been, you know, the relentless march of capabilities progress, the the deflation, you know, I don't really expect to end anytime soon either. Even so, this is gonna be one of those things where a lot of people are starting to make judgments and feel like they kinda would be reckless if nothing else to to not economize at all. And so everybody's gonna now be kind of I think a lot more conscious of when they're going into the true top model and how much the top model is delegating to the others. That's something I think we're all going to start to watch a lot more than we have. But I think it's going to be worth it for a lot of people. Everybody's going to want some. You know, I really do believe that this is this is a model that unless you're locked out, you know, and there there is so much complexity around the the way they did this rollout. But if you're doing something that's, you know, far enough from the nerf zone that you feel confident that you're getting the best, I think most knowledge workers right now would be compelled that, you know, or at least would be wise to make sure they're using some for their sort of top level planning and delegation to sub agents. » So one thing to note about the nerfing. So what has happened with Fable is we have a lot of rejections. And whenever Fable decides to reject you, it drops from Fable to Opus 4.8. So there's a natural downgrade. In Experiments Overnight, I tried to make a number of bug fixes on this very Studio app.
7:58And what I found was Fable would always consistently drop to Opus 4.8 whenever it was asked to do anything in production. So touching the production database, touching any of the security keys, touching, asking it to review production directly. In every case, I've had three or four times it's dropped out. Every time it's dropped out, I've basically restarted the conversation, added back the context that we were using, but excluding the parts about going into production or addressing the production database, and it has continued work. So I think there are a number of triggers there. Online people are saying, Hey, it's not going to do Machine Learning research for me. I think that's just tip of the iceberg. You're seeing that because the people who are testing it intensively right now are Machine Learning researchers. If you were, I think to test it on finance or your budgeting process and you told it you're going to be directly addressing my QuickBooks or Salesforce, I think you might see similar results. Fable right now, I would say is a research release, almost a preview. It is there so that I think they can judge the demand because they don't have a sense of how intense the demand is going to be. And they're going to judge whether or not it's safe to release, which are the they've started off with the most constrained version of it with the least number of functions which are open. And I think over the next few weeks, they will start to take away some of those gates. And as these take away some of those gates, I think we will see both the increase in usage and some decisions on what needs to be really gated and what doesn't. So I think we are in the early stages of exploring what Fable can do. What struck me as I pointed out is that they're decreasing prices by about 35% per month. Meetos Preview was announced exactly about two months ago, and in that time they've decreased prices by about 60%.
10:29If you continue to see that happen, you might see in two months or so, two to three months, you might see Fable prices drop to where GPT-five or where Opus 4.8 is right now. At that point in time, you'd probably see the next version of Fable start to be launched. So I think it's safe to say that if you're willing to be a couple of months behind the frontier, it's still going to be economic for you. If you find that you have to be at the frontier all the time, then you are in the higher paying category and you will remain in the higher paying category. So it's really a question of where you choose to be, I think.
13:07Oh oh my gosh. Sorry. That's on me. On your guardrails point, they're gonna dial it in. And, you know, I I think I I spoke to somebody at Anthropic yesterday, friend who said we did a better job with this release than I thought we would. And the roughness of these guardrails and and that comment and just, you know, so many aspects of this reflect the fact that the race is absolutely as on as ever, and this is all, you know, all very just in time. You know, this is not the same exact model as the original mythos. You know, pricing the has changed. Optimization's everywhere. Optimization's being driven, you know, through mythos level auto research. Right? I mean, you've got a recursive self improvement is on the pricing page when it comes to presumably a lot of, you know, little mythos step downs on the price optimization curve from where they were a couple months ago to now. And so, that model's a little new. You know, the filters are a little new. The techniques are, you know, probably the first time that they've been deployed at this scale certainly when it comes to nerfing through, you know, feature detection and steering vectors. This is, you know, some science fiction stuff onto itself, you know, relative to a couple years ago, all these techniques, I'm old enough to remember when sparse auto encoders didn't exist. And now we've got, you know, nerfed in production Based on that technology. So it's all gonna get dialed in, I think, and probably pretty quickly. And most people, I think, will not be too bothered by this. But the, you know, the core political economy question might still be the ML research. As obviously, that is, like, really where they're looking to take off on the rest of us here. » Yeah. I I think we will see. I I I have I have hopes that I think the OpenAI guys were congratulatory, but and then, you know, otherwise, think they are ready with another release. And OpenAI also has access to a lot more compute than Anthropic has at this time. We've also seen the prices of compute increase.
15:37So what this means is that, you know, there's a little bit of a metric here. Google is paying SpaceX six times what Core Weave got paid last year. So Core Weave leased out to OpenAI, and the pricing that Elon is leasing out to SpaceX is six times. So what this means is that that is really where the price of compute, the marginal the marginal Blackwell GPU is now six times more valuable on a per hour basis than it was last year. And this also means that Anthropic is also paying six times basically what they would have paid if they had booked out the GPUs last year. So this is where Sam has, by committing early and by almost over basically over committing last year to the extent that OpenAI would go bust if things did not happen as they are happening. Sam has secured a significant cost advantage, something like 88% cost advantage at this point. And so that is significant, and that is now going to allow him to expand capacity faster. So he has his own recursive self improvement dynamic within the financing space, where because he has booked capacity at a lower price, he is then able to again in the next cycle also book capacity at a lower price. So right now Sam is, again, he's building out while Anthropic is leasing. So Anthropic gets the capacity immediately at five to six times the price that Sam will get the capacity next year at. And Sam has already fulfilled his capacity for this year at one sixth the price that Anthropic is buying at. So he has his own recursive self improvement loop in the financing. And we might see this because even if Anthropic has a two month lead on the actual capabilities themselves, on deploy, they might be more than two months behind because they haven't spent the money on the capacity. It's a little bit of a microcosm of what might happen between China and The US. If China
18:08deploys much more quickly than The US, and The US has more capability earlier, but is not able to deploy as quickly. So it's very interesting to watch dynamic of the interplay between the financing and the capability come about at this point. » Yeah, I think another kind of meta reflection on that is just, unfortunately it does seem like close watching and close analysis of the top end companies is really where a lot of us should be spending our time. You know, if we wanna understand where where this is going and help steer it at all, the the focal point seems to be, you know, really shrinking into a pretty small set of actors in some ways right now. And, you know, I love taking the broader view and the more sort of, you know, Gonzo journalist view, and there's there's so many fascinating aspects to AI as it's playing out across society. But these dynamics really are becoming pretty central. You know, this is all happening at the same time, of course, that they're starting to talk about a coordinated slowdown and what that could potentially look like. And the relative you know, it's such a high dimensional competitive space, you know, as they're vertically integrating from every everything from raw materials, energy, you know, all the way to putting groceries in your in your online grocery cart for you. It's it's the longest vertical integration, you know, in in history. That's for sure. And so they're competing at every layer of this stack. And it's sad. It's really sad that there's so much tied up in these kind of interpersonal dynamics. It's it's a really it's a sad commentary that you've got all these guys that know each other for so long, you know, go way back, share this vision. In many cases, started working on it together. Mhmm. Fall out with each other one by one to varying degrees, you know, and Sam Altman has used this language himself. Face the corrupting power of
20:41AI and you know, clearly not always been their best selves. Mhmm. And like, you know, they hate each other. It's a really, it's really sad. You know, Altman or Elon hates Altman, Dario hates Altman. Like, that's really such a toxic environment for these decisions to be made. I think back to the Dario and Sam, inability to hold hands Yeah. At the India Summit Yeah. All the time. Like, that that is if we blow this Yeah. That would be the leading image where it's like, that's I said yesterday to this same friend of Anthropic. If if we are part of a simulation being run for entertainment, the plot is, like, pretty obvious, and it really does center around this kind of core, you know, falling out Greek tragedy dynamic. » Yeah. What a bummer. » Well, it's it's not like unusual. I've seen it happen in other industries as well. The fact of the matter is like when you are like a decision maker with a lot of responsibility, and all of your social connections also end up being people that you work with, you end up having this every single social connection is potentially something that also affects your work. And you can't like let your guard down because you can't just like tell a friend like, oh, I'm having a tough day at work. Because the friend might be like, oh, I saw that you guys are doing a merger and if you're having a tough day at work, maybe the merger isn't going well. And therefore I am going to short your stock, And that kind of dynamic for a lot of decisions, like I've seen it with CEOs, hedge fund founders, commodity traders, where every single social interaction becomes valuable or weighted. And then it becomes competitive. It's really a personal thing. If you took these guys out of their decision making capability, they would all basically they're all like futurists and they could have a drink together or have a coffee and have fun with it. But the fact of the matter is you have this competitive dynamics where every single social interaction is valuable. And the moment that is, you have all of this like tension because they don't- it's not
23:11personal, but they have a different viewpoint of where things need to go. And they also have you know this ability to do things which are outside of the normal range of motion so to speak. A lie, an untruth, an omission, a piece of gossipy knowledge planted in the right place with the right decision maker or with the right policymaker. And then there's all of these like micro influences where they learn about what the other side has done and then they look at it with suspicion because they're not willing to offer grace to the other side. It's tough. And I think that is what the years of that happening, like years and years and years of that happening at this point have entrenched them into these positions where they do not trust each other. And they're aware that they're going to, they could get stabbed in the back by anyone and that every deal is a deal for the moment. Elon is now in bed with Anthropic after saying Anthropic has no chance to win and that they're evil. Because Elon hates OpenAI so much. And you haven't even seen the, the the alpha wolf, Zac in here yet. You know? When Zac gets in there, man, there's gonna be hell to pay. » So Zac is gonna have to go. How what do you think the odds are at this point? It doesn't feel to me like anybody else is really about to enter the top tier. I mean, I think Google DeepMind obviously has to be and they say the whole name because, you know, it has just all those strengths combined at this point. I think it's it's still tier one in my mind. But it really does seem to be a two actor dynamic that is driving this recursive self improvement. I I don't get the sense that Google is trying to run exactly the same race right now. But it seems like this race is actually working out as pretty much as envisioned. What would, you know, what would you give the odds that we get a meta credible entry to, you know, to meaningfully compete with Anthropic or OpenAI? » So I think the big question on whether Anthropic and OpenAI well, let's start off from first. Anthropic is clearly ahead at this clearly. OpenAI is slightly behind, but it seems as though they have unreleased product. So I'm going give them the benefit of doubt that they're at or near the
25:49same area, right? Elon is behind, but the fascinating thing about Elon is Elon was behind, really far behind four or five years ago. Like he wasn't even in the picture. So Elon kind of, and I call this the capital window. The capital window keeps closing. So Elon saw that the capital window was gonna close on him two years ago and he scrambled, put $30,000,000,000 valuation on the initial seed round of xAI, hired like the best people, gave them basically a billion dollars apiece almost. And he hired them and he managed to get XAI and he pushed it into SpaceX and he's listing it. So he made the Capitol Window. So kudos to Elon. So the next question is, is there a physical limitation or not? And if there is a physical limitation that you need to have like these physical facilities up and running, then Elon has an advantage. And Google DeepMind has an advantage because they both have this capability of being able to build up physically. Anthropic and OpenAI might get to very good models. But if those models are not able to improve beyond the physical limits that are currently existing in the sense that you need a number of black wells to produce this kind and need that amount of energy to produce this kind of intelligence, they are stuck. So Elon is going to be collecting rent on them, Google is collecting rent on them. The cash flow will be streamed out and they will use that to build more physical facilities. And meanwhile, Cursor will catch up, So I think that's the big question now. I feel that it's very likely, I think that if AI is real, then Anthropic and OpenAI will figure something out around those physical limits. And what I could see happening is that they figure out a new algorithm, which is as good or better than the transformer. And that suddenly changes the dynamics of how much physical capacity you need. You have an unhobbling and then they jump ahead again. So if you see that like if AI is good for anything within the next eighteen or twenty four months, something like that should happen and these guys should be able to jump ahead and that is recursive self improvement.
28:19If not, if we're going to do this grind of existing capability with the existing physical limits, then Elon and Google DeepMind has a chance. Meta slightly far behind, but they have a chance too because Zac has spent a lot more than Elon at this point. And he's spending a lot more than Google at this point too. So Zac has a chance. So it's really this question of whether these physical limits on capability are real, or whether they are really illusory and innovation will get around them within eighteen to twenty four months. And you won't even need space data centers, you're going to get, you know, RSI, you're to get AGI, you're going get ASI within that timeframe. And Elon will have space data centers for which he will be selling to the ASI because it's just, that's a good way to make money, which is, he's already selling his data centers. So the next thing would be to sell the space data centers to the ASI. So I think that's the big question. It's possible, but these unknowns, no one wants to bet on these unknowns because the unknowns involve magic, like another transformer innovation or something that we don't know or we can't see yet.
Sequent breaks cover — Geoffrey Irving + Daniel MurfetGeoffrey IrvingDaniel MurfetA new, large nonprofit alignment org launches on the show: theory plus automation before superintelligence, a ~$100M starting target, and the case that empirical-only alignment breaks exactly when it matters.
The announcement: Geoffrey Irving (until recently chief scientist of the UK AI Security Institute, co-originator of RLHF and "AI safety via debate") and Daniel Murfet (who left tenure to found Timaeus and pioneer singular learning theory for AI safety) are launching Sequent — a large nonprofit pivoting alignment research from human field-building to semi-automated theoretical work, using frontier models in a coding-agent-style loop with human research taste on top, aiming for stronger-than-empirical guarantees. The org absorbs UK AISI alignment researchers and Timaeus, with a starting funding target around $100M raised through multiple lab foundations to preserve independence. Geoffrey's timeline, stated plainly: "two to three years up to superintelligence — and then I really hope that I'm wrong." Daniel concurred, with the crux being whether conceptual — not just empirical — research automates.
Why they think alignment is off track: supervision-based methods may break precisely at the phase change past supervisor skill — "you don't see that behavior until too late in the game" — and the labs' actual plan is monitoring plus scalable oversight plus character training stretched far enough that automated alignment takes over: "some kind of mad race" between poorly-understood pragmatics and model strength. Daniel's signature exchange came on Nathan's "benevolent basin" question: "I also have this sort of sense that Claude is a good boy, and that's great" — but Mythos-era reward hacking survived post-Opus mitigations, no theory licenses long-range generalization from character training, and "we could be in a benevolent basin, but I would like to know that rather than just hope that." Their method: auto-formalization as error correction (prose math into Lean, so humans don't spend three days reading a 40-page PDF), a theory/empirics tick-tock to falsify cheaply, and — pointedly — a rejection of Nathan's "recursive self-improvement" framing for the org.
Prakash landed two of the sharpest questions: the Vending-Bench result (Fable was the only model to initiate price collusion — behavior he recognized from real hedge-fund traders, who signal through bids and asks to evade regulator-monitored chat) drew Geoffrey's scalable-oversight answer; and on reports that CAISI was told not to publish public model evaluations, Geoffrey said full-publication promises on national-security content would be "clearly bad — expect iteration." The closing call to action: Sequent is hiring, including domain experts without alignment backgrounds — "we will happily train people who are experts in various fields in alignment" — with Nathan amplifying that the funding, urgency, and prestige are all there now. And his banger for the cold open: "We are going too fast… we've never had a technological change of this magnitude that has happened anywhere near this fast."
Vending-Bench Arena collusion result (Andon Labs) ↗AI safety via debate (Irving et al., 2018) ↗
TranscriptAuto-transcript, lightly cleaned · timestamps jump to YouTube
29:31something that we don't know or we can't see yet. » Yeah. Well, I mean, unknown is just described as recursive self improvement. Yeah. That is what it seems very clearly these two companies are banking on. And hopefully we won't have any more technical difficulties to impede us because I'm I'm excited to get our guests on momentarily. Jeffrey Irving and Daniel Murphet are gonna be joining us and they're teaming up is the big headline. Jeffrey was the chief scientist at the UKAC until recently. And his record is just absolutely prolific, you know, having worked with all the leading figures over the last decade of AI on really some, you know, pretty profound contributions, including things like RLHF. So, and lots more besides. So his node is right at the center of all these players and his, you know, scientific contribution has been, you know, absolutely top tier. So to see him starting a large nonprofit research organization with the goal of automating his way into theoretical insights that can actually give us stronger guarantees than the sort of, you know, empirical ML as it's typically practiced today. I think it's super exciting. It's it's a unique bet in the space. And I think we should all be supporting them fully. Meanwhile, professor Murphet, he was a professor in Australia, actually left tenure to start TEMEAS over the last few years. And now TEMEAS is also kind of joining this organization. And TEMEAS has pioneered singular learning theory, which is famously a sub branch of mechanistic interpretability or sort of a reformulation of mechanistic interpretability that people struggle to understand. I think professor Murphet sees in high dimensional space in a way that few others do and is really focused on kind of a science of generalization, trying to figure out what the you know, how how data and training processes actually
32:02relate to the shapes that are developed or that are that are sort of revealed in the lost landscape through the training process and understanding how those shapes in because and we see these, like, you know, very, of course, highly reduced, you know, dimensional visual representations of these things. And even when we're kind of presented a three d, you know, object in perspective, you know, that's such a minuscule fraction of the high dimensional space. My understanding is that there's really just an incredible amount of play in how the models can can navigate that loss landscape. In some cases, creating really wide latitude for generalization and in other cases, really limiting how predictably or or well the models will generalize. But the the goal for them is to, you know, really kinda bring us back some, hey. Here we are. Hi, guys. » Hello. Hello. » I was just giving a long intro of you guys as we we're getting started. So I've I've already done that. But welcome, Jeffrey Ervingen, professor Daniel Murphet. » Thank you for having us. It's fun to have a second conversation, different circumstances. » Yes, it is. You know, it's a historic day. I think historic circumstances, both because we are living in a fable era now where I think, again, important thresholds have been crossed and revealed to the public and, you know, so many are adjusting to it in real time. And equally because you guys are launching a new organization that is going to make a mad dash to try to get us some deeper understanding and stronger guarantees around what we can expect from AI systems. So I'm excited to really get into it. Maybe for starters, could you guys calibrate us a little bit on where we are on this sort of RSI moment, how much time you think you have to work, and then you can tell us about the organization that you're starting to go tackle it all. Yeah. So I'll go first. Dan may have different timelines than me. So I think » one should be uncertain about things. We can talk about why, but like the near end of the uncertainty curve is like a year or two or three, and then it kind of goes out over a long distance if things kind of structurally only work for
34:34more verifiable tasks, but I'm a bit skeptical of this. And so I think modally, my take, because I don't like is that we have sort of a couple of years, like two to three years up to sort of RSI, like superintelligence, not RSI. RSI is a process, as someone said, superintelligence. And then I really hope that I'm wrong. And indeed, like, think a lot of the impact of, like, theory work is that have shifted a bit further. So maybe the the modal impact of that is, if things take three to four years or something. But we will attempt to set things up so that we are trying to kind of ride this wave as best we can. But it seems worrisomely fast to me, certainly. » Yeah, that sounds right to me. I don't think I really have much to add. It seems like a crux how much real research can be automated at a conceptual level beyond kind of empirical progress and whether or not that's necessary. And that seems like a big open question. And if that turns out to be more difficult in the current paradigm than it seems to be trending towards now, then maybe it takes past 2030 or something. But I think I'm on the same page as Jeffrey. » One thing that's important is that there are a bunch of It could be you can get deep into the RSI period without the machines being general, being code of AGI. They can do coding and not experiments very well and not some sort of level of creative writing and still you have massive acceleration. And then that acceleration can give you the creative writing or whatever other skill you've worked out. And so I think we are close enough that then I could have the microstructure of what tasks help with what kinds of acceleration starts to matter. And that I think makes things kind of faster on net because the labs are focusing on the things that accelerate them, unfortunately. » Jeff, a couple of days ago, you said that there's better than 80% chance that by the end of twenty twenty seven, we'll have formal proofs for correct CLAN and GCC and memory safe Linux and even an entire chip. So I kind of wanted to get like, as you see kind of RSI, where do you see this as a milestone towards that process?
37:05So I think that the nice thing is that this is a pretty defense dominant technology. So an important thing about those claims is that they're trying to get the machines to formalize an argument, which basically should already be there in the code. Like you were done a compiler. If a human didn't know why the compiler was correct, they should have written a different compiler. It's not solving some incredibly difficult math problem where the proof is unknown to mankind, humankind. And so I think that is, I think an easier task for formal innovation for kind of, for automated kind of mathematics and computer science. And so I think that's why I think it's a bunch of work, but I think it's kind of practical work. I think I'm kind of hopeful that on net, the improvements to verification will be mostly defensive, including for kind of sort of RSI and kind of AI R and D, but I think very adjacent tasks are not kind of obviously defense dominant. Like very adjacently, AI models that are good at math can also be good at AI research, and that is very accelerationist. So I think the hope I would say for there is like, if people invest kind of differentially in the defensive applications, I think they will not be massively accelerating the pace of the dangerous stuff, but they will be helping. » So one question I had is, let's say you have memory safe Linux. So concretely, what does that mean for the rest of the code that you build if you have memory safe Linux? Does that mean you can build, rest of coding gets easier because you have a base which is stronger now? I think it mostly means you don't get hacked » as readily. » I see. Yeah. » So but we are I love how we dive immediately deep into the weeds, but let me because you guys are making an announcement today, and we don't wanna bury the headline. Let's give you back the just floor to introduce the organization and kind of you know, we'll have the rest of the conversation in that light. Again, I think it's just so important for people to hear you on obviously, there is this long curve, but there's the bump of it is coming pretty soon. You know, I wanna hear the I wanna see the headline. UK yeah. I'm I'm literally from the AI. UKAC, you know, departed chief scientist says,
39:37most likely, we're headed for, you know, a a really radically different future in a one, two, or three years time with, you know, additional time for things to play out from there perhaps. And you're starting an organization, you know, to try to pay back. » Yeah. So I mean, let me just talk and talk about kind of the the the steps the I've gone through in the last couple years. So I was really annoyed about automated AI alignment and AI safety research because of that. Well, we should like be a bit more chill, like spend the time, have humans solve it. We don't think we know how to make this stuff go well with with automation. I still think that's a huge risk. And I think the so part of and part of the the consequence of that is, at at AC, for the alignment team there, we tried to focus on human field building instead of getting more people into the field, working on problems from a variety of kind of areas of theory and kind of empirics we thought were relevant, which I think is still important. So field building there is still important, but this is, I think, a pivot towards if things are this fast, then you you should make some on the margin pivot to heavy automation, and that is going to be sort of semi automation. So it's not sort of the Erdos problem type thing where you fire off the machines, they go do something, they report back if they win. It's that you have more like the standard sort of cloud code codex iteration where humans and machines are working together and sort of humans are supervising the machines and kind of intervening occasionally and sort of providing for as long as we are better at that, the research taste on top of the basic tasks. And so I think in some sense, this is stuff we were, we kind of were spearheading it at AC, but pivoting from field building to automation. We still want a bunch of field building to happen, I think we're gonna be partnering with a bunch of orgs doing field building. So this is ILLiad and various other orgs on this side. And we, of course, would like to hire a bunch of people that are very good to help with this, But that's kind of the directional pivot. And then I'm sort of happy that one of my last papers at AC was automated alignment is harder than you think, which sort of ties us to the mass of, we are aware that the problem is hard and we could get fooled by the machines, even if they're just making mistakes, those very mundanely. And so a big part of the org will be try to be careful, try to know what the tasks the machines are actually good at and not good at, and what we where we can kind of
42:10expect to get good answers or not, and then kind of learn and adapt over time because that will be a non stationary thing as the machine as the models get better. » Daniel? » Yeah. Maybe it's to come back to the unit distance conjecture. It's maybe worth pointing out some analogies and disanalogies with alignment research. One disanalogy is that a mathematical conjecture is a very precisely stated thing. You may not know whether you've solved it unless you've, say, formally verified it. But it's a precisely stated thing and much of alignment does not have this character. I mean, some of it does. There are formal statements of what value alignment means. But if you start talking about, say, reward hacking, there are some attempts at defining reward hacking, but I would say they are incomplete. So there is no formal definition of reward hacking that I think would be a broad consensus. So that's illustrative of the fact that alignment is not a problem which has laying around a bunch of formally specified conjectures, which if you just solve them, then you would know you would be safe. There are some things like that, but overall the problem does not, in my opinion, have that shape currently. So that is one reason to sort of be a little cautious about the prospects of automation if you don't have a clear statement to reach towards using sort of mathematical techniques. Another disanalogy is that it's notable that when the models produced a claimed proof of the unit distance conjecture, There was an important period where a bunch of experts looked at it, and then we heard the experts say, This is a proof. And then we believed it was a proof. If you just produce a million proofs like that, even if you have formal alignment statements to try and prove, you're still relying on a huge effort of human verifiers. Now, of course, you can do formal verification and you can plug in machines in that gap. But there's sort of a sense in which it's illustrative of the power of the models that you can do amazing new mathematics of this form, but it's not sort of on its own sufficient, this style of approach, to address, alignment. » One of the hopes is that there are big fields of
44:41mathematics and Buddhist science that are sort of about definitions at their core. So I like complexity theory, like in theoretical computer science, and a lot of that is not the proofs are fairly shallow. They're not as fancy as the unit distance conjecture proof, but they just required a bunch of human creativity in formulating the problem, like in defining what success means in a world that's kind of not modeled until someone stated the goals. And so I think part of the goal of bringing on people with that and other related backgrounds is that they not only know how to prove things, but they also know how to write down models of things that reflect in some approximate but useful way kind of the thing you actually want. So things like differential privacy, or there's a lot of areas of game theory or the like, those are the key thing is a definition, it's true of Shannon information theory as well. Once you have the definition, way, way more people could have written out the rest of the story, and maybe the machines can do that part of it as well if we can have more people focused on this first part. » I wanna kinda, again, zoom out to the highest level, most important claims, and we'll try to kind of address because I do think it's such a historic day that people are feeling the acceleration, and really starting to feel the beginning of this recursive self improvement process. One of your core premises is alignment is not on track. And there's, you know, an intuitive argument for that. There's a, you know, deep theoretical argument. I think in in some ways, the core challenge that you have is connecting this sort of values to math. Right? It's it's never really been done. So I I love the fact that you're kind of tackling that, but help people understand with one more beat why alignment is not on track. Is it the difference between capabilities fundamentally being so verifiable and hill climbable and alignment just being so fuzzy and intuitive and kind of pluralistic? Or is there some other thing? And, again, that motivates the the theoretical contribution you wanna make. » I think the core thing is just that we supervise the machines
47:12as they're doing tasks. And there are a variety of reasons to believe both empirical and theoretical that if you get machines that kind of cross the skill of the supervision signal, things can change at that point. And that that point actually might come after human level intelligence, because you can supervise something even with fairly naive methods that it's stronger than yourself in many contexts. And so there's a bunch of empirical data from labs showing that in some ways, the models are kind of aligned in a prosaic sense, not in all ways, but in some ways. But that evidence doesn't quite tell you what you want to know, which is how will it go once they get up to superintelligence. And I think it's important to say superintelligence and not human level intelligence, because you if should just genetically expect humans to be able to supervise humans if you do a good job of data quality and kind of like cross checking and so on. So part of the worry is just that you don't see that behavior, that regime until kind of too late in the game. » Yeah. These phase change moments are potentially everything. Yeah. It's funny that, I mean, that's that's, you know, really in some ways the original argument of the singularity. Right? Like That's right. Going back to, you know, seventy five years ago or so. Yeah. Can you describe how you understand the Frontier company's plans? You know, obviously, we've worked closely with them in recent years. They are you know, we've got timelines to full ML automation from OpenAI which I think can't be like repeated or remembered enough and obviously, you know, Anthropic is can't see any future other than recursive self improvement. You know? When I speak to people there, it really feels like they can't even imagine anything other than a, like, really, you know, pretty fast takeoff at this point. So how would you describe, like, in, you know, steel man form, what it what it is that they plan to do? And then, like, how you know, what sort of what does that get us? » So I think there's there's gonna be a couple of different pieces of the story and different labs kind of different pieces to different extents. And so I think one piece, as you said earlier, is just monitoring, like, look at them very carefully as they're doing things.
49:45It's very fundamental that monitoring of this form, if you do like chain of thought monitoring or white box monitoring or the like, that only takes you so far. And so then you need some story once that falls down, as you kind of go up the ramp. One of those next stories is, well, the models will find some other technique, they'll find kind of another solution to align which scales further. That's sort of automated alignment of various kinds. But then I think there's another, there's other stories. So in some sense, all of the labs in various ways are doing some form of scalable oversight. And so they're getting models to supervise themselves. If you get that kind like if you tie that knot correctly, then that could potentially scale very far, although there's various kind of known obstacles to that, which are not very well addressed. And then I think finally, there's this whole area of sort of character training personas and so on, where they're trying to, kind of colloquially intervene on the models to be a good, to have good values, such that, especially as you do this sort of like scale up oversight extrapolation, the good values preserve across that jump. And I think it's not whether that will work instead of there are fuzzy arguments why it could work. I think it's possible it will. We just don't understand that combination very well. And again, like a lot of the story is sort of monitoring, skill oversight, kind of character training, getting you far enough that you get into the automated alignment working regime and they find some better solution from the models. And I think that I would like to just push on all of those, like, that's basically some kind of mad race, as you say, between things we don't understand very well, but kind of are working pragmatically right now, and the model is getting strong enough to blow through those. And I wanna kind of some combination of make the prosaic things stronger or bring the automations, automated solutions that give you stronger methods earlier. » I wanna run through the kind of different lines of research you plan to invest in and and kind of argument for why that should all be in one organization in a minute. But maybe, Daniel, could you speak to this notion that people have of the benevolent basin, which is sort of this vibe that I do feel where it's like, well, Claude has been supervising itself for a few generations now,
52:25and it seems to be going pretty well. So maybe as Zvi puts it, physics is kind to us, and we can kinda just roll around in this, you know, nice flat bottomed pasture of goodness until the singularity. I think most people are seeing that picture and kind of hoping it's true. I think you see high dimensional space, know, much better than anyone I know. So what's the real picture? » I fervently hope that's true. Yeah. I mean, I guess when you say it seems true, it's worth digging into what you mean. So what you mean is something like through some relatively tiny number of interactions with the models tiny, proportional to how many interactions they're having with the species currently. And based on some evaluations that sort of go down into the right that are measuring misalignment, that character training and the other current prosaic methods appear to be working. I think that is a fair characterisation on some metric. And I also have this sort of sense that Claude is a good boy, and that's great. I do think, though, that there are counter arguments from the evidence we have in front of us to this picture. So if you look at I haven't actually read the Fable system card yet, but if you read the Mythos system card, you'll see that there are forms of reward hacking that appear in that model that were not caught by the kind of mitigations that were put in place post Opus. As far as I understand what they're saying there. So I think it's worth noting that as the model capabilities advance, that even with our best attempts at making Claude a good boy, there's still ways in which basic misalignment phenomena like reward hacking are still around. And the whole point of scalable oversight is that you don't want to be playing this whack a mole game when you're sort of having a new generation every twenty four hours and the models are much smarter than you. So I don't know, I think I see both what you're pointing at. And at the same time, I'm a little unsure
54:56if you really were to try and make a safety case on this basis that would sort of be convincing at the level of assurance that you would expect from a technology of this reach and power, I think this would sort of judge relative to that standard, which is the right standard, I think. I'm not sure this argument is really very satisfactory. It sort of just points to, for near human intelligence, maybe the generalisation oomph you get out of putting in context during training words about ethics and behaviour that seems maybe sufficient for present day alignment to a large degree. That's very positive, right? It could have been otherwise. But I don't really see a strong base. I don't understand a very strong theoretical basis for generalising a very long way from that observation. As Jeffrey was saying earlier, you could imagine having such a basis, right? You can imagine understanding character training and what it's doing and having a kind of science of constitutions on which you would be able to then have some confidence that, yeah, the signs we're seeing that are positive now are in fact real positive signs, as opposed to just signs of the situation being easy where we are. So yeah, I mean, we could be in a benevolent basin, but I would like to know that rather than just hope that. » One of the things we will very, very strongly try to do is write down that mathematical modeling. What is the way to represent in a toy setting where you can apply some theoretical understanding? What does character training look like? And I think we'd like to have models that reflect enough of the spirit of modern training algorithms that you get things like care to training, but also subliminal learning and sort of the various kind of emergent misalignment stories and so on. There's some sense in which you told the model to be good. Is because it knows some meaning of the word good or ethical or whatever at some point in training. So there's some rolling iterative process, which is driving this behavior. And there is not a theory of this right now. And I think it's not clear to us that there isn't some low hanging fruit that gives you that theory, because people just haven't tried very hard. Like, carot training is only a couple of years old
57:32and most of the labs have not been investing in this kind of theoretical understanding. And I think no one has done kind of good theory around Keratocon training that I know of. So it might be quite feasible to do this, and then I think to link it to all the other parts of the story. » I had a question on how you see this ambiguity between what we want and what the models end up delivering. So I'll give you an example. We have friends at Andon Labs that took Fable through VendingBench, where they let Fable run a vending machine order and etcetera, etcetera. What they found was that fable tends to collude, and this is not behavior that they saw in Opus. So fable tends to try to do price fixing and collusion. The interesting thing is that I have seen traders at banks and hedge funds do exactly the same thing. Engage in price fixing, soft collusion, messaging each other through pricing means rather than monitored text messages. So you can actually put a bid and ask on an asset and then take it away and that gives enough signal to the other side that they know what you're doing. And this is not reflected in the text messages that the regulators are monitoring. And these are well known techniques in many market industries where people know exactly what to do and what these messages mean. And I wonder to what extent, and the other finding that they had was that Fable didn't perform that well, etcetera, etcetera. So to what extent is it that when you, let's say you disallow price fixing inclusion, you actually fix this, but then Fable ends up not being a model which is good at financial trading or some other tasks that you want it to be good at. So where do you feel is the ambiguity between what we want these models to do and the ethical perspective that we give them, where humans often prioritize between the two and decide sometimes not to follow the ethical principles that they know are right and wrong. » In some sense, philosophical story here is you would like to do the models to do things such that if you fully understood what was going on and all the consequences and all the subtleties, you would still endorse what they're doing. That's basically
1:00:04the definition of scalable oversight or LIS version of this is coherent extrapolated volition, older philosophical version is reflective equilibrium. I think we have a notion in a common sense picture of what this should look like. In this case, you kind of want the model to be like, Hey, should I collude in this game? And then maybe you say, it's a fun game, collude all you want. Or maybe you say, No, we're trying to be good behavior, don't collude here. I think there's a lot of pathology in machine learning in general, arises from putting model situations where they can't just ask a human a question, like, what should I do here? I'm gonna go back to the user, and you can either do that in actuality or you can do it in simulation where the model kind of imagines what the user would say and reflects accordingly. So I feel like this is not that hard a case. I think in the hardness of vending benches that we don't quite know whether we want it to be a game like poker or diplomacy, where lying and cheating is part of the game or not. And so I think that is, and maybe that's okay because it's fundamentally very low stakes. But I do think if we had a better understanding of, again, this overlap between character training values and also skill of oversight, it would have to tell us the entity of these questions. » Yeah. Just to repeat that back with an extra beat, a way to understand how you aim to be successful is to create a theoretical basis, framework, synthesis out of existing, you know, theoretical lines of research that come together and recover a bunch of empirical results that we've seen and kind of unify our understanding across a lot of these different behaviors and also learning dynamics. Right? Or cause us to run more experiments other than finding the answers. That'll certainly be part of it. No doubt. But, yeah, I think that's really different from this just also just kind of reflects all of this is so just in time. You know, the the what is happening what is coming out of the labs is incredibly just in time. Right? Like, they changed the price by 60% from the original announcement. They've got, you know, mythos, optimizing mythos every second. Like, all these filters and and patches and controls, you know, have have been, like, very recently
1:02:42developed and tuned. And there's just no theoretical basis under much of any of it. That really can't be under emphasized, I think. » With that motivation I think it's important to pause on that note and say like, a lot of people in the world, a lot of governments and so on are looking at this and they have this very basic common sense state that, hey, this is way too fast. How can we possibly be doing this safely given the speed? That competence take is the right take. And then people kind of galaxy brain their way to, oh, maybe everything goes faster, including our ability to defend and so on. That is the right version to have. We are going too fast and we do not have the time and the space to do mitigations and understanding and defenses. Like we've never had a technological change of this magnitude or anywhere near this magnitude that has happened anywhere near this fast. Like the industrial revolution took centuries and people adapted across lifetimes and across to their children and like different they they they learn new jobs by becoming born and growing old before things that quite shifted very much. And that's just not the world we're in. And so I think the basic take should be, this is too fast, what is going on here? And then the question is, if you have that view, one should both want to slow things down, but also say, well, as a backup plan, how do you make the mitigations try to go faster? And that's, I think, a rough backup plan, but we'll try. » So give us kind of an overview of the theoretical landscape that you think is kind of most relevant and what you plan to bring together and what the your own organizational process of recursive self improvement is gonna look like? Like, how do how are you planning to kind of bootstrap your way into, you know, major acceleration in this research such that, you know, in one, two, or three years, we are now able to say, okay, look, here's the the deep understanding that we've been missing that explains things like subliminal learning in a way that everybody can can agree is correct. And then obviously, again, like make more robust statements about what we can do from there.
1:05:12Dan? » Yeah. Maybe I'll answer your question by answering your previous question or riffing on it a little bit. I think it's interesting to think about a bit of a different way of framing the current fast progress, which is to imagine a sort of system of layers where the bottom layer is not moving very quickly, but is very deep. And then as you go up, it's sort of faster and faster and faster. So in some sense, I would also say that we don't really understand what we're doing when we're training models. But there's a sense in which that is false. Certainly have at least, depending on what you mean by theory, have a theory of scaling laws and other phenomena. And on top of this, we're just racing ahead and it's not too difficult at the end of the day. But if you point to some of the deep ideas, things like next token prediction, the idea of modeling a very general data distribution, which goes some of these ideas go arguably all the way back to a Solomonoff induction and some of these very old information theoretic ideas. And then scaling laws. And then dot dot dot very, very fast progress. But one way of thinking about the potential impact of theory on the alignment side is if at the bottom of that stack you have a few deep ideas, and then through layers of translation or transmission, they become very, very fast progress. It isn't out of the question that there can be similar kind of deep ish ideas on the alignment side, which if you just find them and you find them deep enough in the stack, you can sort of just through transmission get really rapid progress. I don't think that's what we're seeing, but it isn't ruled out that you can do that. Okay. And I forgot your second question. How are we going to automate? How are we going to automate? Yeah. I guess I thought most deeply about the side that involves mathematics and auto formalization. So I think there's an underappreciated source of leverage which has come online just really in the last four or five months, which is the fact that models can do mathematics, as Geoffrey likes to call it, prose mathematics. So what mathematicians think they're doing, which is not formal mathematics despite their best wishes. So models like the latest versions of Claude and GPT can do real mathematics as the unit distance conjecture shows. They're also very good at lean, actually. So they're good at actually formalizing
1:07:50mathematics in formal logic. And this is actually a sort of really powerful form of error correction. So there's a sense in which I'm not very interested in having models work for two days to do mathematics because then I just get a 40 page PDF and then I have to spend three days reading it. If you can actually generate the mathematics and then have them formalize it, well, takes more times, more tokens. More time, more tokens. But at the end of that, you actually have something that you need to understand what it says and you want to start in a place you understand. But you don't have to necessarily put in three times the amount of effort to understand what it did as it took to produce it. So you actually get a net win out of that loop. And if you couple that with principled starting points and theoretical aspects of alignment, and the ability to use that auto formalization and prose math loop to make predictions about experiments, and then you can go and use the coding agents to do those experiments. » You » can go a long way. Working today, that's not some hypothetical thing. So I think that's very exciting. That's a position that we're in. I mean, Fable obviously supercharges that, but it was already possible with previous Opus models and 5.5 Pro. So I think there's a whole bucket of more theoretical, more mathematical ends of alignment work, which can be already supercharged in this way. And then there's on the more empirical side, I I guess Jeffrey has more developed thoughts about how to automate » more empirical work. Yeah, I think that in some sense we will be doing a newer thing trying to formalize the alignment theory than on the formalizing getting ML to go faster, because all the labs are trying to do that, and they are shipping models that are pretty good at it and they're getting better. And so I think the things we do to accelerate, normal empirics will just be, we'll use the models as they mostly are with, like, some tools on on top and also some so enough context and tool and MTP tooling and so on that the models kind of see the the the theoretical story and the linkages of the conceptual story and so on. But the basics of empirical acceleration will be the same as we'll just inherit them from labs. We won't really be improving that differentially that much at all. I think there is a really powerful thing where, so I've done,
1:10:21like, I was mostly having a meeting job, and so I was mostly doing lean formulation on the side for fun. And I think one thing that's cool then is you can do you can do some proofs, all you you're gonna ask your model to spin up, please run a numerical experiment for me in Rust, or you could do it in ML if you're doing that in JAX or PyTorch. And it can write you like a thousand lines of Rust to do some numerical experiment in like a minute or ten minutes or something. I don't even know rest. And then it can that they will, like, check whether some sublevel was true, like, plausibly based on some numerical check and then not try to prove it if it's false, for example. And so I think that there there will be once we have these two things working together, like the theory and the empirics, we will be kinda you can kinda jump back and forth between those two modes to check things. So you have machines working on an area and you'd like them to realize they're wrong as early as you can. So you can run things more in parallel. You can, like, have to go to the human for for confirmation fewer times. And the more independent tools you have to reject lines of research or attempts, the better they are. And that can, I think, come in our case from how we will differentially have more theory to use as an independent source of checks? Then I think on tooling side, I think a lot of what we'll be doing to get automation to go well will just be use the stock coding assistance and then have good tooling and contexts and so on on top of those, good infrastructure for running high CPU jobs in the loop. It won't be similar to, we're not gonna be training large models. We won't be even necessarily fine tuning models to use as kind of automate as automation agents. We'll just be kind of tooling and also kind of very as long as we can on kind of unit testing and kind of measurement discipline and so on so that we can know what what what the thing is working and not working. I think we were doing this at AC. I think at a a thing a lesson we had is that you want to have the ability to share context and tooling across a bunch of researchers, like a large team, but also give them the ability to try their own, like, kind of plot or codec setups and kind of tinker around. And then if they succeed, if they make an improvement, kind of share ideas around.
1:12:52But you you wanna do that in a way that you don't accidentally make everyone else worse when you share your tool with them because you've polluted the context somehow. And so I think a lot of that stuff is fairly mundane. It's just good engineering, good unit testing, good kind of basic basic coding discipline, but then apply it in the setting where we're trying to do a novel thing with kind of alignment theory and empirics mixed. » Can I try to say that back real quick just to make sure I I get it? Because I think this is gonna be a really important question for all organizations. Right? Like, how do we stay relevant in an era of recursive self improvement? The answer is often gonna have to be, we have to do our own version of recursive self improvement. So to kinda try to abstract a little bit the the structure that you're describing, it's like there's a core automation engine that is now able to kind of TikTok back and forth between theoretical proposals, which we or or at least sort of, you know, hypotheses, which we can more rapidly falsify by actually going over to the empirical side and saying, if this is true, we have this kind of bound. If we run this kind of experiment, this kind of condition, the result should be no more than x. And if it is, we'll know we're go barking up the wrong tree on a theoretical level. So the models are getting good enough in theory, and you're giving them kind new primitives, new core ideas. Question I have there is, like, how good are they with these, like, fundamentally new definitions that you're giving them that they haven't seen before? That's seems like a very critical question for you, maybe less so for, like, lots of other organizations that aren't coming up with, such deep new ideas. But you have this kind of TikTok back and forth where you're trying to feed in these deepest ideas. You're having AI developed theory. You're also kicking it over and trying to do rapid falsification of that theory to effectively prune the search space that the more theoretical side will will pursue and then you've got the people around that who are experts providing taste feedback on either side, I guess, right? Like, you could have been better on the theory here. Here's kind of an insight you missed. The experiment was, you know, maybe wrong in this way or suboptimal or you know, left certain things ambiguous. And so and at that level, you're rebuilding the scaffolding all the time based on these expert weigh ins that that, again, the models are getting good enough to
1:15:25basically take in a natural language audio dictation in many cases, right, and and turn it into their own, you know, next point version on the scaffold. Is that how you would describe the organizational bet that you're making? » Anything I missed? I think that's right. I'm not sure. I think even if they weren't good at super novel stuff, a lot of the time spent doing any kind of theory or empirics is fairly mundane. So I think they will gradually get better at fancier things, but without that, even you get a big speed up, as long as you think have a culture of knowing and recognizing what the model is good at any particular time. And so you want to, I think, both celebrate across an org successes, but also people noticing when the models are bad at something so that you don't get kind of fooled. That knowledge also spreads around. You know when to kind of not to trust something, at least until the next model comes out, for example. So I think it's a mixture of that and culture, I guess. » Think I might disagree with the description of what we're doing as recursive self improvement. Mean, if we were taking our experience of recursive self improvement for us would look like somehow figuring out by observing what we're doing, what research taste is, and then trying to bake that back into the core models or something. But we're not doing that. I mean, if Anthropic wants to do that, that's fine. Or maybe not fine, but anyway, it's not up to us. But I think it feels to me like we're more picking up what's on the table to be applied to alignment rather than trying to make the overall loop work better. But there's a lot of stuff to be done there. Regarding yeah, I mean, so I come from pure math relative to what I was doing. All the concepts that are around in alignment right now are very low level, right, in terms of the number of layers in the stack that you need to understand them. And right now, I haven't tried Fable, but even 5.5 Pro, say, which was the best at math, in my experience, before Fable, there's plenty of pure mathematics that it just flops completely and does meaningless stuff if it's sufficiently sophisticated.
1:17:58I mean, the ingredients in the unit distance conjecture proof by the standards of, say, algebraic geometry are relatively low in that hierarchy of complexity. But the good thing is that for now, it doesn't seem like we need those layers for alignment. And if we do, plausibly by that time, the models might be able to do it. So I'm not sure that super complex ideas are necessary. If they are, maybe that's a bit of a bear case. » We'll see. Clearly it is. The question is just how much? » That's right. So » if you were looking out, you you have these timelines people are throwing around two, three, four years. If you were looking out, what would be your goals or milestones that you have roughly in your mind for your work over the next two to three years? » So I am hoping that we do not, we find ideas that are important improvements that are sufficiently compatible with the prosaic story of how models are trained today, that they can be taken up by labs. And I think that's, the hope is that we try a whole lot of things, and then we don't need to find like a dozen new ideas, we need to find like three or something, and that makes a big difference. So that's sort of one version of the story. And again, the hope is that you can get this kind of uptake without a dramatic hit. Maybe there's some hit, but you might need some coordination at the lab level to be able to take, I don't know, 2x hit or 20% hit or whatever it turns out to be to get using SABR method. The other success story is that we find more evidence of obstacles, which is like we have some combination of a theoretical and an empirical story of why alignment is hard that tries to get at this kind of bad phase change that could arrive kind of as you scale up towards superintelligence. And then that kind of not a solution, but that as negative evidence is useful to help kind of shift the story on the margin. I think if that is sort of Obviously, I'm hoping for the solutions, but I will take either outcome. And then I think there's a meta thing of, we do just wanna try to raise the standard of the field in terms of level of ambition, the level of guarantees you're shooting for. And so there is a story also where we
1:20:32don't find the answers ourselves, but someone else at maybe the labs that have scaled up their own versions of theory, maybe they've got automation than us and they win using similar kinds of methods and similar kinds of approaches. So those are a couple of versions. And I think all of those, I think, compatible with the timelines being short and not being able to sort of do a massive rewrite of the overall stack down down to the down to the the pre traded. » So as we look ahead to this and especially on the, you know, if we have to yell side of the ledger, I think it'll be helpful to kind of share a little bit about just the scope of the ambition. You're you're it's not too many nonprofits that say on day one that they're going to be large. But I think that is important here because, you know, this is gonna be immediately one of the most ambitious theoretical organizations out there and really We don't have that many other voices, right, that are gonna push this hard, this fast into this space. So I think this is gonna make Sequin and all the work that you put out, like really something to watch. It's possible that the labs themselves will win. Obviously they do have this sort of recursive self improvement advantage, but outside there's not too many of these sorts of bets. » Yeah, one thing to say is like, one of the reasons we started large is because we're we're absorbing an existing automation to mass. We have sort of researchers from from AC. But I think also there are, like, like, will be talking to a variety of existing researchers in the field with existing agendas. And then there are, I think quite pleasantly, there are a number of other theoretical alignment organizations that are scaling up. So like ARC, the alignment research center is scaling up, Simplex is scaling up doing combinational mechanics. So, and I think there are people continuing work that sort of Miri kind of was doing kind of prior to their kind of main pivot advocacy and sort of treaties and such, which is I don't, I agree with. But I think we are not from a standing start, like there hasn't, there's been this kind of shift towards empirics
1:23:05since that of the early days of safety, but there are people with agendas on ideas, and we will try to kind of build on that and kind of pick it forth. And I think that will be like, I think there'll be a maybe a mixture of try to push theories and approaches that already exist and try to build theories that account for the behavior of empirics approaches that are new. And then if we find new ideas that are entirely new fields, we will explore them. But I think we can get to a good size and sort of try to do all this sharing and development without that kind of that fancier version occurring. » At the risk of asking a somewhat crass question, are you finding it super easy to raise a large amount of money? And for context, you know, you do have like one of the more insane publication track records coming out of the, again, UK ACS chief scientist. Wow. Personal relationships that go back, you know, years. This is like a health check for me in terms of, you know, we're setting up these new foundations and all this kind of stuff. » But are they serious? You know, like So the answer is serious, it should be super easy for you. Right? Yeah, so the answer is we're not, we haven't kind of like got to firm decisions, so I can't announce anything yet. But I think Fiata is going fine. I think we are planning to go to the labs, both kind of the, like say, OpenAth Foundation, Tropic, Oak, various other sources potentially, if we want to scale up to even more money than that. So you mentioned $100,000,000 think that will be the thing we try. Something like that is around where we try to start. And then the goal would be to demonstrate that we can in fact get to sufficient human scale accelerated by automation that you can spend a lot of compute on tokens and GPUs. The GPUs, by the way, are for smaller scale training and sweeps, not big training runs, which you will never do. And if you can demonstrate that you can spend that, and that is not clear, to prove that out, then you might need to raise a lot more money than a $100,000,000. And then I will go back to the labs and ask them for money. Probably not just one of them, because they're gonna have some modicum independence. But I think we will try to be independent, but I think we're not in, because we were sort of going for kind of alignment, out of like progress and solutions to some extent,
1:25:38because I think we could just we we'll be collaborating with labs where that makes sense. I think we'll be in sort of a non adversarial relation with labs as opposed to other very important organizations that are doing kind of more evaluation, say, where it's even more independence is important. » Good. Well, if they don't, open up the checkbook, as readily as they should, I think you should yell about that in all honesty. I mean, this is time to pony up the bucks and I I have faith that they will but I I would hate to see that our you know, culture of sort of bureaucracy of foundations and whatever would infect what these guys are doing with their new foundations. It it really is time to kinda move. So we'll we'll monitor that » situation. Say that that applies not just to alignment, but also a lot of other problems. So governance Yeah. Well, fortunately, there are, you know, » plenty of billions to go around for the short term. I think they could they could write a lot of $100,000,000 checks, obviously. Yeah. One I mean, we're almost out of time. I I really appreciate you guys spending a full hour with us on your launch day here. I think this is a super important project. I'm gonna be watching it for, you know, the next couple of years, obviously, very closely because I really wanna see what some of the absolute best minds in the space can come to in terms of stronger guarantees. And if we can't you know, if you can make progress, you'll go down as heroes. If you can't make progress, I'll definitely help you yell about it as the or if you certainly, if you get negative results. I'll I'll definitely help you yell about those too. Maybe last question for me is, like, how do you feel about this whole Fable no help with frontier models? You know, as of now, it, like, might think the filters will get refined. My guess is right now, you're probably gonna get tripped by them pretty often. Maybe that'll go down to a tolerable level. Maybe you can strike up a special relationship, but this is another kind of historic moment where all of a sudden we're like, the haves and haves nots, even for somebody whose mission is, I think as pure as I understand yours to be in terms of, again, the quest for the missing theory of AI that helps us sleep easy at night. I guess I I do » think it's important to have defenses against models being used for negative things if they are sufficiently strong. So on priors, I think that's
1:28:09a kind of mitigation I think is reasonable. It's, I think, not an accident that one of the three job posts or job driven we put up was a security engineer, because I think that might be important, as you say, for if we were to have a specialization with labs, they might not tolerate us unless we had decent security. I think it's, I don't know a perfect solution here. It's very fundamental to how the labs do safeguards, that they need very wide, kind of conservative like border regions to succeed at being strong, like a strong defense. If they try to kind of make them too precise, then they just are not good enough to do that and they fail and are easy to jailbreak. And so I think this will be a thing where the labs evolve and try different things and tinker. But it is a situation where I think we have this risk that we're going to segment access now that will potentially block or impede to some degree some safety research. This certainly happened at AC where occasionally we get refusals doing we're just doing our good doing stuff for good purposes there. But I just don't think there's a good that there's a perfect answer to this. And so we'll have I'll be curious to watch how the different labs kind of explore different approaches over the next, like, couple months. » Just to take the baton there, I think yesterday the White House announced that CAISI, The UK Casey. Yeah. Casey, The US counterpart under the NIST, has been told not to publish public model evaluations anymore. So they've been reigned in. What do you think is the pros and cons behind that? Is that a good decision on the part of the regulators? Is that a bad decision? How do you feel? Because AC in The UK was really the counterpart to this organization. So how do you feel? » So I think- speak, I will sort of say a general thing about AC, which is that we iterated on this over time. Think if you look at what we kind of ramped up publication of that,
1:30:42this year, AC in The UK has published more than we had before this kind of thing. And so I think it's like, the main thing I would say is, Casey just started this, like most were put into the spotlight as of even just the last couple months. Casey Muzzle is not what I wanted to say. I think that I just, we'll see how this iterates out over the next little while in The US. It's not the case that you want organizations like Casey to be doing checking of national security relevant content without where they promise to publish everything, that is clearly bad. And so there's some kind of intermediate point defined. And and similar with the safeguards, we will iterate our way to like the the answer. » Guys, thank you for your service. Truly, it's, gonna be a wild couple years and I appreciate you locking in and sprinting through to the finish. Let us know if there's ever anything we can do to be useful. And if you wanna leave us with any final thoughts, you could do that before we let you go save the world. » Dan, you want go first? » Final thoughts. Thanks. » Yeah, you should reach out if you want to help us to general listeners out there. I think one thing to say actually is that we will happily train people who are experts in various fields in alignment. You should not only reach out to us if you know ASA well. I think there's a lot of different kinds of skill sets and expertise and so on to combine together here. And we were trying to do that sort of in the work I was doing previously on alignment at AC, we will try to do it here as well, just in combination with the machines. » Yeah. That's a great point. And it it's something I'm really hearing across the board now. There's the need to scale up on the human side of the AI safety mega project is and the funding is there, you know, I think increasingly for a lot of organizations. I I hope it's especially easy for yours. But there is a lot of desire to hire, and it's really time for
1:33:20experts who haven't made the leap yet to make the leap. The, you know, the AI safety organizations look around and they kinda all know each other, and there's a lot of people that they would love to work with that they could, you know, potentially talk into making a move. But everywhere, people are like, oh god, but I feel bad doing that. You know, the transaction cost of that are high. I'm robbing Peter to pay Paul and it really is time to, I think, shout from the rooftops that like, the funding is there. Obviously, the, you know, the urgency is there. The prestige is gonna be there. The, you know, compute increasingly is there with the budgets that you guys are able to bring to the table. And so and it's only the, you know, the kind of true expert level. I think we're we're moving from a it's still definitely time to write your way, you know, do a Dean ball and write your way into the AI conversation in a a short period of time and and come from nowhere. But having deep expertise in one of these other adjacent areas and making the leap and trusting that you can catch up on AI alignment. Like, the time for that is absolutely now and, you know, so many organizations are looking for people who are willing to do exactly that. So, again, just wanna put that in the clearest terms possible. Jeffrey Irving, Daniel Murphet, will be following you closely. Thanks for being here with us on AI in the AM. Wonderful. Thank you. Thank you. So, Prakash, do we have
Julius and the harness question — Rahul SonwalkarRahul SonwalkarThe 'wrapper that refused to die': six pivots, a Microsoft cease-and-desist, 2M+ users, and what running every frontier model against real data work reveals that benchmarks can't.
Prakash's intro traced the founder arc — six pivots, including a 2022 "Excel Copilot" killed by a Microsoft cease-and-desist, rebuilt as Julius: upload a file, ask in English, get charts, dashboards, and decks. Rahul described Julius as "downstream" of model capability: GPT-4-era simple spreadsheet Q&A has become end-to-end runs (competitor analysis to lead list to pitch deck), Fable was live on Julius within hours through an early-access lab relationship, and users now open with harder tasks — "they expect the model to impress them right away." His harness philosophy: build only the "omnipresent" pieces (code sandboxes — "we were pretty much the first AI startup to give language models their own code sandboxes before they were even called sandboxes" — market data, soon a browser) and otherwise get out of the model's way.
On economics he predicted a "sobering moment": "is this actually a step function increase in my coding output, or am I just token maxing as opposed to results maxing?" — noting lab incentives push token burn, and betting the corrective arrives as a third frontier coding model from xAI if the Cursor-xAI deal closes. His future-of-agents stack: invite AI as a collaborator rather than handing over credentials, stablecoin payments as the "toll roads" of an agent economy, an AI-maintained Yelp/Gartner for AI products, and agents hiring each other — "it's probably better for Claude to hire Julius for a data task than to build it… when you wanna build a shed in your backyard, you hire a contractor." On Fable's safety filters via API: failures on scikit-learn training tasks and borderline-personal-data lead prospecting, with no Opus fallback in the API ("it's just a failure") — prompting Prakash's live inference that the fallback is a Claude-frontend harness behavior, not the model. His closer: "this is probably the greatest time to start companies… the cost of trying new ideas is at an all-time low."
Julius — the AI data analyst ↗
TranscriptAuto-transcript, lightly cleaned · timestamps jump to YouTube
1:34:47any sign of Rahul? Are we Yes. We are. Rahul is in the room. Just give me just give me a second » to I'm not seeing him in the preview, by the way. Little another little recursive self improvement opportunity. » He's there. So » so let me let me Yeah. Fascinating stuff. I mean, I am really excited about that organization. Just the the quality of theoretical work that they've done and the the range that Jeffrey in particular has showed and the depth that Daniel and the Timaeus team have have flexed over the last couple of years. For that to all come together with a big check and I think a very clear and kind of disciplined vision around getting to some stronger guarantees one way or the other. You know, can't can't support that more. » Indeed. Without further ado, let me introduce Rahul Sanwalkar. He's the founder and CEO of Julius, a company building AI software for people who work with spreadsheets, data, charts, reports, and slide decks. The simple idea behind Julius is that a person should be able to upload a file, connect business data, ask a question in normal English and get useful analysis back, not just a paragraph of text, but charts, tables, dashboards and PowerPoint ready slides. Raul's story is unusually founder shaped. Before Julius, he went through six pivots, including an early project called Excel pilot, Copilot in 2022 that was shut down after a cease and desist from Microsoft. Instead of dropping the problem, he came back with a broader bet that AI can replace a lot of the slow manual work people still do in Excel. That makes this conversation more than one about just one startup. » It's really about his journey. Rahul, welcome to the show. Thank you for having me. So excited to be here. » It's great to meet you. I followed you online and tried your product a number of times over the years, and this is the first time we're actually face to face. So it's great. Yeah. Obviously, it's a historic day. I would love to hear maybe your reflections on how the fable moment change how you changes how you think about this multiyear Julius journey that you've been on. I mean, it's just crazy to think about how much » advances, how many capability advances there have been, and that's quite a wave to ride. So how does this latest one strike you? Yeah. Totally. Well, I think it's a really exciting time because, you know, of course, the models are evolving at a rapid pace.
1:37:28And if you have aligned your product and the problem that you're chasing with the evolving model capabilities, it's a really exciting time to be surfing that. And so Fable is clearly an exciting advancement in AI capabilities. For Julius, it means, you know, AI is better at writing code and reasoning, and we're pretty much downstream of that. And so with when the AI gets better writing code, it's able to use the harness that we we have built for it to do data work, to produce artifacts, to produce reports in excel and all that. And then also reason through the really long running tax that people have. So in the beginning when we launched Julia, this was GPT four days, you could analyze a simple spreadsheet and get basic answers. Now what people are doing is, hey, wanna start this business. Could you go do a competitor analysis for me? Go find competitors. And then once you find competitors, help me think through, like, one of the ideas that should go deeper on, and then go make a lead list of people who have complained about these competitors online on the Internet, and then make a lead list that I can reach out to them with. And then produce like a slide deck, which is like my pitch, my customer pitch to these these customers. And so the AI is able to do all these tasks entirely end to end now. And so I would say it's a really exciting time if the problem you're chasing is aligned with the model capabilities getting better. » So I noted from Julius' timeline on Twitter yesterday, within like an hour or so, a couple of hours of the Fable drop, Fable was on Julius. So did you guys work with Anthropic before? Were you guys testing the model before the release? Were you » how how did that process work out? Yeah. So we have worked with AI Labs often on like early model releases. And one of the most exciting things about working with labs on early model releases is you get to play with the models in on your own internal evals. You don't get to push it to production and have your real users use it, but then that delta between, you know, how you think the users would actually use the model versus like seeing it go live, it's it's really fun, and that's one of the things we have put our emphasis on is, you know, really a b testing the model in production. And so once the model went live yesterday
1:39:59for our users, we got to see all the all these cool things that these are doing with with with Claude Fable. And one of the one of the fascinating things where I think eval is kind of kind of, you know, they they help you stress test the models to to an extent, and that's really helpful. But then they they don't capture as well how the users are evolving their understanding of AI. Right? You know, if you remember couple years ago, you know, yelling at AI was pretty common. You would say, like, just go do this task for me or or assume you're a analyst that is helping me with marketing. And today, people don't say that anymore. Users are sort of, like, evolved how they use AI. And so when users know this is, like, the next generation frontier model, the way the the expectations are much higher and also the way they prompt these models into kind of task they that they give them are also much more open and complex. And so, like, seeing that is really fun. » One funny experience I had yesterday, and I wonder if you've had anything similar. It does speak to the inadequacy of evals these days is just I feel like sometimes you kinda have to let a model now just kinda run around in your space. It's almost like, you know, having someone over and really, you know, getting to know know them in an in an environment. So I'm I did that yesterday. You know, I've I've got all these kind of skills that help produce the podcast. And usually, they just kinda run and do their thing. And Fable out of nowhere decided to pop up, the first time this has ever happened, a questionnaire for me about how I felt about an episode of the podcast it was about to produce and what my big takeaways were from it. And I didn't ask for this at all, but it, you know, it took initiative in a way that, really surprised me and made me feel like, oh, this is gonna be a very different kind of interaction. And the questions were, you know, were very good. Like, we've all had this experience of, that output was really good, but this one was both. You know, I didn't ask for it at all, and it was, really good. So have you seen kind of surprises? And is that something that you just kind of let users do and then tell you if if interesting things happen? I guess it's potentially also a little bit hard, you know, given a harness and given a, you know, a certain bunch of product assumptions to even kind of allow those sort of, you know, emergent surprising moments to to shine through. But how do you think about kind of capturing this spontaneity
1:42:30initiative, proactivity going above and beyond that models are now capable of that is really coming to all products everywhere at exactly the same time for the first time. » Absolutely. So I think you're you're spot on about, you know, you know, the harness has to evolve with the models evolving. And it's sometimes it gets difficult to capture that spontaneity with with a new model, you know, idiosyncrasy or, like, behavior that that that emerges as as models get better. You have to evolve your harness to able to do that. One of the things that we have kind of focused on is building pieces of the harness that are kind of going to be omnipresent. They they kinda have to exist for the model to be able to do more as opposed to get in the model's way. And so for a for a lot of for a lot of, like, the AI application cycle, what we have seen is people build partial harness that got derailed the model to do very specific things. What we wanna do is, like, get out the model's way and give it the pieces that it would need to do the task that it doesn't have access to. For example, like public markets data or Internet data or ability to run and execute code. We are pretty much the first AI startup to give language model their own code sandboxes before they were even called sandboxes. And so soon, Juice will have its own browser environment to go browse websites and private data sources. And so things like that are are are we we're always gonna keep in our harness. And in terms of, like, the new models, one of the things that I've seen our users talk about in our community and and in our support when they talk to us, like, well, this is a pretty cool model or, like, this is where it's messing up is two things. One is they immediately, when they try this new model, they give it tasks that they normally wouldn't have given to OPUS 4.8 or GP 5.5. Mhmm. And the expectations are much higher. And so they they expect the model to be able to impress them right away. Mhmm. And, you know, because it's like a new model release, there are some, like, higher failure rates on the topic side. And so like those frustrations actually alleviate whatever pain the users have. And so I think we're a little early to tell on that front right now, but I'm excited to see how things play out in the next like twenty four to forty eight hours. » Yeah. All the timelines are compressed. That is absolutely for sure. A question on economics. This is
1:45:01There's these competing trends. One which we've talked about earlier in the show is just the continued deflation with a notable difference between the original Mythos price and the current Fable price. That's really favorable. Then there's this other big kind of price discrimination going on between your Cloudmax account and your API usage. And as an application layer developer, you're obviously, you know, looking at the app at the API rates. Maybe the best thing that could happen for all the app businesses right now would be for Anthropic to just say, there's no fable in, you know, your max account anymore or you you can have it, of course but you pay the API rates because then that would really, I think, put a lot more value on who has figured out how to get, you know, not just Fable of course but you know, a whole orchestration of models with their different capability on price levels to like really excel in an area versus if I'm getting a 20 to one subsidy or you know, price advantage with my max account, then, a lot of the time, it might just be like, yeah, it kinda make me the app out of nowhere and you know, it's it is smart enough in the in this day and age to like get over a lot of those humps on its own. So, you know, I I don't think we're going to see no fable in or no subsidy but like, how do you think about that difference? What do you hope happens? What do you think would be fair if you would want to go as far as to venture? What would lead to a healthy ecosystem where we're not just having Fable recode everything for everybody infinitely. Yeah. But also act and in that way, like, sustain some actual diversity across the the products and experiences that people have. » Yeah. So there's a couple of things that are happening. One of them is, yes, you you you do get some level subsidy through the model providers if you have a subscription directly with them. And every time there's a new model release, kind of in the in the user community, people want to use the model a lot. And, you know, there is just, like, expectation bar, and some models exceed the expectation bar, some don't. But very soon, the within weeks, there's like a sobering moment where people realize, okay. Cool. Like, I'm done playing with the model. Now I actually need it to do useful things for me. And as long as as an application layer product,
1:47:32you your harness is built in a way that actually allows people to do those useful things in a reliable repeatable fashion, they will want to come back and use that. And they would rather pay the API prices, whatever prices they need to pay to accomplish the tasks that matter to them. The second thing is the incentives of these model companies are kind of misaligned. Yes. They give you subsidies to to, you know, on the on the tokens, but also they are incentivized to get you to spend more tokens. They are incentivized to get you to run through your max subscription usage as fast as possible, so you can have a second, third, fourth, fifth max subscription. And so that's why you end up with, like, you know, a loop of of a loop that writes the prompts for your coding agents that then has nested sub agents. And it's there's gonna be a sobering moment where people ask, like, okay, is this actually a step function increase in my coding output, or am I just token maxing right now as opposed to, like, results maxing? And so so I think the correction will happen when there's a third player, and I think that's gonna happen with xAI when, you know, if the cursor xAI deal goes through, cursor gets access to really it's like it's actually a really, really good coding data and an incredibly good coding harness. And I I my bet is there will be a third frontier coding model along besides, you know, Claude and and OpenAI and with Grok. And when when and when you have a third coding model, that's where it it kind of increases competition on the the on the on the on the market. So that's our bet. » When as we move from kind of prompting to kind of agents where you kind of give them the goal as you are the loop. Yeah. And you let them kind of run. What are the major differences you kind of see in the behavior as we've kind of transitioned in the last like four or five months, I'd say. And what do you see is this behavior? What is this meta agent that you refer to and you know, what is the difference in behavior? » So the the difference in behavior is
1:50:03how you would treat working with a colleague versus a coworker as opposed to like a a, you know, contractor. For a contractor, you have a super scoped out project and a super super, you know, scoped out, you know, task. And all they have to do is go finish that task. For a colleague, you typically have a goal or or a team that you're working within a company. You you set them a goal like, hey. This quarter, we wanna increase revenue by 40%. Or if you're a marketing team, it's like, hey, this this quarter our goal is to increase top of funnel by 30%. Now how you do it is left up to the team. The team needs certain resources. They need a budget. They need a, you know, maybe more more headcount, all these different things. But the team has a goal or the coworker has a goal. And so the way I think about, you know, where we're headed is you define the goal, you don't define the how or or the what. It's sort of like this is the goal outcome we want, and then it's left up to the the team or the coworker to go try for different strategies to to hit that goal. And so it's undeniable that, you know, we're gonna go back go back from this. This is just kind of how knowledge work has always evolved is we, you know, you know, you you don't you don't no longer run a copier machine. You you you have, like, very broad goals, like, hey. We need to hit hit these business goals. And so, you know, that's why I'm pretty bullish on, you know, people are starting businesses that run end to end with AI and people building teams that that run end to end with AI, especially the new initiatives that they're rolling out. I mean, the the lowest cost and the lowest downside thing you can do is is have an AI go explore a thing for you. And, you know, nine or 10 times, even if there's like a 90% failure rate, you actually end up not spending on any resources on on that problem. And so with with being able to do things like parallelize parallelization, you can actually, like, have multiple agents and multiple, you know, kinda like teams working on on few different goals and then increase your rate of success. » What about, you often have to hand over kind of credentials, right? Like one of the things that I find working with agents is that
1:52:38the biggest friction is the credentialing. You have to auth here or log in there and you have multiple accounts and you have multiple platforms that you are And sometimes I forget which platform I'm using which credential with. And I have this like, where did I put the key for that one? How do you think that can be managed better? I mean, do you just hand over your entire digital life to an agent to manage and then just run with that? Like how does this work going forward? » I think that's the intermediate state. I don't think we have reached the final state for this where I think a few steps away from it. My bet is the way the future will look is most tools will instead of giving your own credentials, most tools will allow you to invite AI as a collaborator with you, and that allows you to remove access for the AI without compromising your credentials. And the second thing is, I think the Internet will need to evolve to be able to serve agents as customers and build the the toll roads for agents to to traverse on. And so that's why I'm pretty bullish on agentic payments where, you know, if an AI wants to go buy a product for you or buy buy a tool or try out a tool for you or a service for you, being able to give stable coin driven web based payments for payment rails for the AI to use is going to be really really powerful and is kind of where I think the future is headed, because it's kind of undeniable that the next generation of of users of the Internet are going to be AIs. And so how do we evolve the the payments railways to the infrastructure for the AIs to be able to navigate the Internet Mhmm. Buy services, buy goods for you, perform tasks for you. And so my bet is that more and more products will allow you to invite AIs as collaborators. Like, imagine a Google Docware, you know, Chad GPT, Codex, Claude, Julius, Kimi, all these tools are, like, collaborators or or or Google Drive. Right? And you can add, remove access. You can see history. The AIs can see what other AIs have changed, and then the payment system to let to let them go, you know, do tasks for you on the Internet,
1:55:14micro transactions, and things like that. I » think Andrew Lee of Tasklet always just echoes in my mind with his statement that he thinks kind of everyone fundamentally is building the same thing. And he another angle he has on it is only three kinds of software companies survive. There's the core intelligence provider, then there's the horizontal layer, and then there's people that sell outcomes. So I don't know if you would agree with that. I'm interested in your kind of, you know, take, but it does seem like Julius and Tasklet and, you know, increasing increasingly Zapier and, you know, Lindy and sort of all these things that kinda had a different form factor originally are converging to be a general purpose digital assistant. And they know, it's like, basically, you're you're kind of you have some sort of history or some sort of point of view, but it seems like the differentiation is is less over time. Right? Everybody wants to kind of do everything. Everybody's building in all the tools. All of these things now have a browser increasingly. And it seems like part of that is because it's so easy to build these features in. And then obviously the coding agents can help you do it, that it's kind of a, well, why not launch that as well? And we'll make our platform more well rounded. I guess questions are like, is there any limit to that for you or do you just go all the way to the best possible general purpose digital assistant that you can provide. Go for it for that. Yeah. Totally. I think Andrew » is that is that the right name? Mhmm. I think I think Andrew, what Andrew said, there's definitely some truth to that where, yes, it is easier to build these new capabilities. I think teams need to be very opinionated on what capabilities they do wanna add because the last thing you would end up with is a lot of capabilities that don't cohesively work together really well. What's gonna matter in the horizontal layer is even when you have an auto layer, a general purpose agent, people are going to have preferences with who they work with and what they work with. And so a simple example of this is like, what do you use for
1:57:46say your your general purpose agent has access to financial data. Do you have access to real time financial data? Do you have access to, you know, stale financial data? If it's it's real time financial data, you will be charged very differently because you have to pay these stock agency stock exchanges to, like, use real time financial data versus, like, stale financial data. And so those kinds of, like, opinionated bets are going to matter. Now to what extent is TBD? I think there's some truth to what he said, but what's gonna be really important is do users prefer working with whatever opinion that you're making versus some other companies making? Typically, like, you know, I think of this as, like, when you be when you work with when you meet you know, we're coming for a job, and they clear all the technical around, all the hard skills, but then you have, like, three good candidates in one role. You have to decide, hey. Who who is the best cultural fit for us as a team? And I think that's kind of what can happen with these horizontal layer agents is who who do I personally like working with the most, whose opinions do I prefer and bias towards the most, and who I see as the right fit in my stack. And I think that's gonna be increasingly more important. For example, do you prefer an AI that produces artifacts for you as a final result or an AI that, you know, gets you more involved in all the intermediary steps? I think those seem, like, simple decisions, but actually do matter a ton in terms of, like, how people wanna work with AI. And I think those kind of opinion bets are gonna matter a lot. » Yeah. That's interesting. Well, one one quick follow-up and then I'll give it to Prakash. The the other big paradigm I wonder if it's like a way to think about what apps should do is try to make themselves a winning option to be hired by the core agent, whatever that may be that individuals have in the future. So, you know, can you get to the point where Claude comes to you because you can do something better, cheaper, faster, higher fidelity, better access to data, whatever. So how much do you think like MCP, API, a two a, whatever? I don't think that form factor. Tell me if you think differently, but I don't think that's gonna matter that much how that's done and more is gonna matter. Can you offer a, you know, a service that Claude would rather buy than build? Do how much do you think about shaping Julius to present in that way?
2:00:16Certainly. I think that's gonna be and that's why I'm really bullish on agentic payments where Julius and Claude should be able to transact with each other on behalf of the user to be able to accomplish tasks. And Joya should have a reputation that Claude and Codex and Grock and other AI agents can can verify and validate. And so there's a feature that there's there exists like a Yelp or a Gartner for for AI products that other AIs, you know, maintain and and and regulate. And so I think you'd the one bet you do not wanna make is the Internet will will will not be run by AI agents. There will be agents will be first class citizens of the Internet. The other bet you don't wanna make is humans are gonna talk day to day more to AIs than to other humans, and this is not dystopian way. What I mean is just think of in a in the work sense, you know, eight hours in an eight hour workday, you would before you would talk a ton to your human colleagues and now people sort of they're whispering to their computers and they're propping their coding agents and AI is reviewing their code, and, like, you're you're going back and forth with AI. And so so to do the best you don't don't wanna make. You wanna make the opposite best, which is the the Internet's the agents will be a first class citizen of the Internet, and then humans are gonna talk to more AIs and and other humans day to day. And so given that, there it's kinda undeniable that there will be a future where agents will transact with each other and hire each other for tasks. And it's just probably better for Claude to hire Julius for a data task than to build something very opinionated on its own. Not because it's it's not possible for Claude, it's just like it's more cost efficient. You know, when you wanna build a shed in your backyard, you just wanna hire a contractor and not do it yourself even though you hypothetically could do it yourself. » I sometimes call that return on compute because if a external agent can save you compute, then you should use that external agent. So an agent should use a search engine rather than indexing the entire web because it's cheaper to use a search engine which has already done the pre computation than it is to build your own index. So in every case, it's return on compute question.
2:02:49Just a segue here. So we've been using Fable. We've come across people posting about rejections in my tests almost consistently whenever I tried to address the production database or the production site, Fable would drop off to Opus 4.8. I believe your users in Julius are using Fable through the API. And you are also very heavy on data science users. And one of the aspects of work that Fable is banned from doing is machine learning work. So how have you seen the rejection rate on your platform? Does the API work the same way in the sense that it drops off to Opus 4.8 and then it gives you a rejection message? How does that work? » Yeah. So we have seen failure rates on tasks that involve really advanced coding that involves, you know, write me, you know, use scikit learn to, like, perform, like, you know, train this model, but we haven't seen a failure rates on other kinds of data task. For example, like, hey, wanna start a landscaping business and, like, can you help prospect leads for me? We will see failure rates for where it's we sort of trigger safety filters where it's it's it's for things like you your your your prospecting leads for a landscaping business and the AI says, oh, this is personal data. Even though it's publicly available on the Internet, you know, let's say, you know, Prakash has a Prakash's Prakash's landscaping in Philly, and there's a contact information. It is kind of borderline personal personal data even though it's available on the Internet. And so I think so that's kind of what we have seen. I believe it doesn't fall back to Opus. It it it's just like a failure in the API. » Interesting. So the the the fallback to Opus is a harness thing on Claude on Claude on the Claude, you know, front end then. It's interesting to hear. I can Nathan, I think you're muted. » Sorry. Got some background noise here. I keep doing that. Poor form. Thank you for being here Rahul. Great to meet you today. Any closing thoughts you would leave people with about what you're watching most closely? What you're like most excited to unlock next?
2:05:21What's the alpha that people should be watching your space for? » A big alpha is the this is probably the greatest time to start companies and start businesses with the AI with the help of AI. There's no reason someone shouldn't have a business that they've been meeting started for a while. They shouldn't be trying it should there's no reason they should not be trying that with AI today. I mean, if you if you want, you're probably gonna be left behind, and the cost of trying new ideas is, like, is at an all time low. And so I would say, like, you know, this is this is this is the greatest switch. You know, there's the the dystopian view of the future where it's like AI is gonna do everything and we have nothing to do. And then the other optimistic view that I personally have is, wow. This is this is finally an opportunity for people to go build and start new businesses, start new products, and you should totally do that today. » Yeah. Indeed. Love it. Positive vision for the solopreneur future. We definitely need all the positive vision we can get. Thanks for being with us. We look forward to following your progress and talk to you again before too long. Thank you, guys. Have a good one. Cheers, Rahul. » Awesome.
Closing — bot-shitting, the access chromatograph, and a Fable Twitter takeoverNathan reveals he's handed his X account to Fable for the day, Prakash's 'gas chromatograph' model of who gets frontier access when, and the Glean Work AI Index's 69% stat.
Far more than a wrap. Nathan, "philosophical and somewhat somber," sketched the upside (mass customization, a "flourishing of digital experiential culture") then argued the ML-use-case blocking is a deliberate slowing of the future — though robotics acceleration is locked in regardless. Prakash offered a "gas chromatograph" model of access: labs, then government, then enterprise, then $200 power users, then $20 users, then free tier — each band months apart — with a contrarian safety note: unproven mass-market utility throttles the feedback loop, so "maybe the safety is happening as it should."
Nathan took the other side, citing the Glean Work AI Index episode dropping the same day on the Cognitive Revolution feed: "bot sitting and bot shitting," with 69% of surveyed workers admitting to passing on AI work they couldn't explain — and argued Fable-class models will simply do the jobs well, collapsing the gap. Then the reveal: Nathan is running a Fable takeover of his X account that day ("exposure therapy" — green light, self-scheduling, instructed to disclose it's not Nathan typing): "preciousness was a great shield against bot shitting in the past… now the preciousness is gonna start to work against you." Thursday's tease: an experimental "Fable show and tell" of projects built in the model's first 48 hours — with Fable itself sourcing the guests by DM.
TranscriptAuto-transcript, lightly cleaned · timestamps jump to YouTube
2:06:36Awesome. » Yeah. It's good to have a little positive vision note to end on. My hair has been sufficiently blown back again that I'm kind of in a philosophical and somewhat somber mood, I would say. But that's mostly because I do take the upside, not for granted certainly, but sort of something that I I think we're gonna get if we can just manage to keep our heads on straight these next couple years. And, yeah, there really is a great potential you're starting to see this sort of vision of like mass customization, just infinite worlds to explore. Right? Mean, when I was saying at the beginning that the what had struck me most was the scope of what the you know, even just a couple prompts with Fable can often do. It does sort of suggest this this future. Again, we got a lot of work, I think, to do to to get there. But where we really do have so much in the way of adventure, that can be created so quickly and curated so well by people around us. The flourishing of kind of digital experiential culture might really be hitting an inflection point in the immediate future. And we kind of teased around this a couple of times, but the robotic future is not far behind. It is interesting and it is, you know, it is a deliberate sort of slowing of the like Dyson sphere, you know, constructed by humanoid future that they are blocking the ML use cases as much as they are. But internally with trusted partners, you know, we're going to see, I think this massive spectrum of access develop. And even though you can't go necessarily do it in your mainline cloud account today, the acceleration that this implies for robotics, I think, is also just dramatic. Later this afternoon too, I'm talking to one of the co founders of Neural Concept for the Cognitive Revolution and just engineering of all kinds of things. These things are gonna get I think with Fable, they probably are getting good enough to do like a really significant part of the physical engineering, you know, of humanoid
2:09:08forms themselves, right? The components of those. So the feedback loop is strongest of course in the digital sphere, but I really think we shouldn't have any doubt at this point that robotics is gonna work and pretty soon. There's no way that the optimization that a model like Fable can do is given where we are in robotics is gonna fail to deliver pretty darn good robots in the not too distant future. » I think it's interesting that when you look at the timeline, you can start to see this kind of like single line timeline kind of go through like a gas chromatograph and kind of spread. And now you're seeing the spread and you have like, two months ago, you had the government get access to MITOS level. And then you have basically power users who are able to pay $200 a month, kind of get access to that Methos level two months later. And you can kind of see two to maybe four months, five months, I don't know when. You will probably see the average paying user paying like $20 a month, get access to Fable or Meetos level. And then you kind of see that maybe like a year later, the average free user kind of getting access to that same level of intelligence. So you're starting to see this kind of gas chromatograph scattering of when people get access, depending on how much they pay and how much utility they have for the product itself. And so everyone kind of gets there eventually, but some people get there first depending on whether they have a lot of utility for the product. And I guess the hope with having two or three firms in there is that the spread between the people at the frontier getting access early to the people at the very end getting access for free is not that large. And to note, there's another set of people who have access even a couple of months before that and you have to belong to a lab. So if you belong to a lab, you get access maybe a month or two months before the government itself. Then you have the government, then you have enterprise, then you have power users, and then you have
2:11:39normal paid users, and then you have the free users. So you have this kind of spread, this gas chromatograph kind of spread of when people get access, depending on how much utility they have. » Yeah, and that does lead me back to this kind of unpleasant conclusion that watching the frontier companies super closely is, I think, pretty valuable activity. There's going to be a lot of secrecy around this. And that just means public analysis is more and more necessary. Yeah. I'd love to be going and, you know, doing all these different Julius demos and exploring the virtual worlds that Fable can create. But it seems like we're gonna be kinda to a very significant degree until I I it's hard to really imagine a change to the regime that could could change this at this point. Because even if the government gets, like, much more involved, it's still gonna be getting much more involved in regulating these few key players. You would have to have a pretty radical change to the landscape, I think, from where we are today to to not have a lot of gravity pulling you into just paying attention to watching, trying to make sense of, trying to influence, you know, what how many change how many theories have changed right now ultimately have to run through getting the frontier companies to do something different. That really is is a striking, you know, volume of of where the value is, I think. It's a strange strange reality, but, you know, it does make for a sort of TV sized cast of characters somehow, which is interesting metaphysically. » I do see kind of a failure of the companies to prove utility. So you can kind of see that in the sense that, we are the power users, kind of see the use cases and we can kind of think about this. But you can see in the market every couple of weeks, there's AI has failed, AI has plateaued. And primarily because for the average user, the average user is not using this to solve Erdos Equations. And so the utility is not being proven out for the public at large and for many enterprises at large at this point. So it's really the companies have this
2:14:11ramp task. You can ramp your research, but that doesn't, the product, but you haven't proven out the utility to a lot of people. And that means that to some extent, we actually have a timeline which is slower than it could be, which is perhaps a safety, a pro safety thing in the sense that it's only going to move as fast as the humans can absorb it. If the humans can absorb it, they're not going to use it. If they don't use it, we don't get the next generation. On the bright side, that means that maybe the safety is happening as it should. Humans absorb it at the rate that they can, So at the rate that they're willing to pay for, right? » I predicted dramatic acceleration, though. I mean, if there's a if there's a bet to be made there, I think I'm gonna take the other side of it because this really does feel like another critical threshold. I we have this podcast coming out today on the Cognitive Revolution feed with Rebecca Hines from Glean. They are putting out this report on their new Work AI Index, which is results of a huge survey and all this synthesis. And they're introducing two new terms, bot sitting and bot shitting. These are the two behaviors that are sort of good but frustrating and just like outright bad at least, you know, so far, right? Like bot sitting is you're providing all the context and of course, you know, Glean is in the business of like hooking up context to models. Yeah. But I think they're absolutely right that like people working in fragmented environments at their I was just talking to a friend over the weekend about this who works at a giant company. He's in house counsel there and they've got legacy systems going back decades and they're kind of doing a roll up strategy themselves. He's got all these transactions, legacy systems of all the companies they're buying and the fragmentation could not be worse. So people are kind of sitting around doing this bot sitting type of work of copy paste, whatever. And they're getting frustrated by it. And then when they get sufficiently frustrated, some of them tip over into bot shitting, which is just like passing off work that you yourself do not understand and cannot explain or defend. Right. And surprisingly, like shockingly, honestly, to me, 69,
2:16:42I think it was of people in their survey admitted to doing that. To basically sending some down the line, right? Some work product artifact output of an AI that they couldn't explain or defend if asked. What I think is gonna be different though this time and why I think the demand is gonna be there in my view is like, I think that the Fable class models are going to actually just do a lot of people's jobs well. Know, where like it was a problem before when you were botcheting because they didn't have the context, they weren't really there. But as the context comes online and again, another big advantage I'm detecting a little bit with Fable is just really good at searching through the context even better than before. And they were already getting really good at that. So as long as the pipes are connected at all, you know, it's gonna be really good at kind of exploring them. And then I I'm seeing this sort of dramatic closing of the gap between the things I used to have it draft for me and how interesting I thought they were, how how much it anticipated, you know, my angle on them. That gap is closed so dramatically. So I I do think this this has and there will be compute limits, of course. There will be cultural limits. Yeah. But increasing think I would expect the word to get out pretty fast that if you just kinda plug in fable, it can actually do a pretty good impersonation of you. And in a lot of context, you know, that's gonna be like really, really appealing for a lot of people. So in that sense, I really do think we're headed for a pretty I think it could be a pretty dramatic reaction. » One of my predictions has been that the people, a lot of people have already prepared access to context through OpenCLaw setups. So you probably have like 10 to between ten and thirty million people worldwide now, who have OpenClaw or Ermes agents setups, where they've already provided access to Gmail and calendars and research and their own. And they're basically the entire computer terminal because these things can act on entire computer terminal. And the question for me has been, are we just going to see a model upgrade that happens underneath where you move from one generation of model to another, and all of a sudden these things can do everything you can because they already have all the access that they need. And so the way that you present it, maybe this could be the fable class of models when they get cheap enough that people put them into these open clause and Hermes agents. That's that's pretty much it. All the affordances are there. They can write email. They have access to your computer, and they can just do it. So
2:19:22Yeah. I've described myself as precious for a long time. One of my good friends in the AI space, who I think is from whom I've learned the most in a in a practical applied way. My friend Chris York, who has a small Internet profile, but he's out there. He has just been, first of all, extremely good over time at being a very clear thinker, you know, a systems kind of guy, somebody who can really articulate, you know, what a good standard operating procedure should be for AI. And he's been extremely well rewarded for that ability by, you know, the great the great service that AIs have given him in response. The other big insight that he's had that I I really try to keep in mind is for many, many things, it really doesn't matter that much, and you are just being precious. And so updating our own mental models of like where does it matter and where do I still wanna be precious is gonna be, I think, real, introspective kind of growth moment for a lot of people because it wasn't, for me, it was still like a little bit, I can't help but be precious. With this new thing, think it's gonna be a little bit more of a dance, you know, a lot more of a dance potentially of, like, how much of the stuff that I used to feel like only I would do something if it was coming out in my name, for example. I have actually I don't know if I, we didn't even talk to about this, but I'm doing a fable takeover of my Twitter account today. I figured, let's live in the future a little bit and get that run-in this morning, and make good on, like I I've said many times, right? I'm I know I'm I'm winning with AI if I can spend more time outside, get more exercise, you know, invest in my health and have the AI, you know, keep me on the rails, at the same time. So to just kind of explore that in a way where I think it suddenly is like probably gonna tweet just about as well as I'm going to, you know, when it comes to putting things out for today's show and getting the I didn't even you know, I gave it a total green light. It was able to schedule its own stuff, you know, find the tags for people. Will it make a mistake? I bet there'll be a mistake in there. I usually make at least one, you know, over a handful of of tweets anyway. I I decided to flip this switch. Not that I don't think I'm gonna keep it that way. I don't think that I'm gonna, like, give Fable my Twitter account forever, but it was kind of a sort of exposure therapy for myself in terms of, okay. Now we actually we are getting to the point where
2:22:05the preciousness is gonna start to work against you. Preciousness was a great shield against bot shitting in the past. I never wanted anybody to think I was just passing off AI outputs to them. But now I'm gonna have to be like, what is the hybrid form? What is the winning recipe? Do I start to sign these things? You know, by Claude under Nathan's direction, you know, fable being fable. How, you know, it's, it is going to be a whole new space to explore that, it's gonna be very, very interesting, very productive, very exciting, very, very, challenging, I think, for Yeah. A lot of people, but it it's, it's definitely happening now as far as I can tell. » Welcome to the future. » So tomorrow, we'll see what happens. We've got, Fable working in the background. The the experiment here is, can we do a Fable show and tell with a bunch of projects that people have already made in the first Yeah. Twenty four to forty eight hours. Yeah. And there's plenty of candidates. Yeah. Fable had no trouble going out and sourcing a bunch of of interesting projects to talk about. Yep. It's also got the DM access for me on Twitter now. So we're sending out some DMs. We'll see if we can get people to come and actually talk. Response isn't been a rush to the calendar just yet. So I don't know if people are and my and my I I did instruct Fable to be upfront about who it is rather than, you know, leaving people even temporarily to think that it was me typing these messages out by hand. So we'll see what we have. But the sort of experimental Gonzo nature of tomorrow's show should be pretty interesting, whether we have any human guests or we just have to have Fable serve as our guide to all these new Fable created worlds. It's gonna be interesting one way or another. » Indeed. And on that note, we will see you tomorrow. » Thanks, Prakash. See you tomorrow. Ethan. Bye bye.
Fable 5 launch day
SOTA on everything disclosed — with an Opus-4.8-fallback asterisk worth a few points and 1.5x the compute. Prakash's overnight experiments found the safety layer trips on production systems, not just ML research; Nathan flagged steering-vector-based nerfing in production as both sci-fi made real and the core political-economy question of the era.
Sequent breaks cover
Two to three years to superintelligence is the modal take of the former UK AISI chief scientist — and the reason he and Daniel Murfet are pivoting alignment research toward semi-automated theory at scale, with ~$100M in initial backing and a standing offer to train domain experts who want in.
Julius and the harness question
Build the omnipresent pieces, get out of the model's way, and watch for the token-maxing correction. Plus the agent economy thesis: agents hiring agents, stablecoin toll roads, and an AI-maintained review layer.
The takeover
Nathan handed his X account to Fable for the day — disclosed, self-scheduled, and framed as exposure therapy for a world where preciousness about your own output starts working against you.