3D Memory, Grading AI at Work, and an Agent That Makes Introductions

Nathan Labenz: Oh. No signal.

Prakash Narayanan: Good morning. It is Wednesday, October 28, 9:02 AM. Nathan, good morning to you.

Nathan Labenz: Good morning, Prakash. How are you today?

Prakash Narayanan: I'm good. This is our first show this week, and there have been quite a lot of developments, I think. Sometimes we see a lot of things which are world historic, but I guess the rest of the world doesn't know they're world historic yet. And so we end up in this weird situation where there are earth-shattering events on a daily basis, but the earth doesn't move that much.

Nathan Labenz: So what has stood out as most earth-shattering to you? The things that have crossed my attention that are probably going to go down as legitimately historic are the major open problems being solved in math, computer science, and physics, which will strain my ability to understand, let alone explain. The cyberpocalypse continues, so far, I would say, kind of a slow burn. Knock on wood, I haven't been directly affected just yet, but, but for the grace of God, honestly. I actually have an interesting story about a security review I did last week, between our last conversation and this one, that is maybe instructive for other people, to make sure that they do an agent-powered security review for themselves. But where do you wanna start?

Prakash Narayanan: I would say the math developments, the math and science. So in summary, about four or five days ago, late last week, early this week, OpenAI announced just three. Unlike the 722 solutions last time, they just announced three scientific, mathematical problems which they had solved, and they put out the three into the public domain. They published. All three had Lean formalizations. I think the biggest one was the exponential time hypothesis.

Prakash Narayanan: Just so that I don't butcher this: in computer science, there are algorithms which don't take that much time. If you add a single variable, say you have an algorithm that takes maybe one second with one variable, and it takes two seconds with two variables, and three seconds with three variables, that's linear time. As you add variables, you add time. There are algorithms which take much, much longer, what we call exponential time. If you have one variable, it takes one second; two variables, it takes four; three variables, it takes nine, et cetera. It just goes exponential. As you can imagine, some of the toughest problems we have in the world take exponential time. If you look at calculating all of the movements of some molecules in a group of molecules, or in air, the number of calculations you have to perform is so large as you keep adding molecule by molecule, the number of interactions you have to calculate, that even calculating all of the movements within a small amount of air would take more computation than all of the atoms in the entire universe.

Prakash Narayanan: Right? Some massive, massive number. Because of that, a lot of these problems are intractable. We basically believe that we will never be able to calculate all of the interactions of every single molecule inside a group of atoms. And maybe that means there are limits to what humanity can actually learn from science. We are going to end up with simplifications which don't precisely describe all of those interactions, which means we will never quite precisely know, for example, how weather happens, or how any of these interactions between molecules happen. So in computer science, this exponential time hypothesis, that you have this exponential time and you will never get better than that, has been around for a while. And that means you're just stuck.

Prakash Narayanan: The exponential time hypothesis also has to do with P equals NP. P equals NP is the superset of all of these problems. And that means you will never be able to take these problems and move them down into algorithms more amenable to computation. And what happened is that our friends at OpenAI found a disproof. They found some method of taking something that was exponential and making it sub-exponential, slightly sub-exponential. They made it slightly faster than theoretically possible. And that has a lot of implications, because it may mean that we are heading towards a zone where a lot of these problems that we thought were basically intractable to computation are going to become tractable over time. That has both good and bad effects. Part of why people were interested in the exponential time hypothesis is that you can then predict some algorithms are not going to be broken, and that is what you use for cryptography. A lot of the cryptographic algorithms that we use have this particular effect that you cannot brute-force them, because if you were able to do this in a reasonable amount of time, you could brute-force a solution.

Prakash Narayanan: And instead, we use these algorithms which were beyond this exponential time. We were like, okay, this is what we call hardness. These cryptographic problems are hard, and because they are hard, we can use these cryptographic methods. We have secure communications, and we have Bitcoin, and we have Ethereum, and all of these things because these algorithms are basically intractable to computation. And now that wall has been broken. So we have the first cracks, showing that some of these problems are tractable to computation. It doesn't mean all of the problems are. It doesn't mean cryptography is broken, but it means that the cracks are there. It basically took down this 25-year wall, since the exponential time hypothesis was first stated, of what the limits are. And it's showing us that those limits that we thought existed do not really exist. It's absolutely fascinating, and obviously people are going to investigate how to get to P equals NP and how to solve these problems in a much, much shorter time span, and that is going to have immense impact on the world.

Nathan Labenz: One question I have that I don't know if you have the answer to, but maybe Cue can help us find it if you don't: my understanding is the disproof was by example, basically. They took one of these problems that was thought to be exponential-time-hypothesis hard and showed that they can come up with an algorithm for that particular problem that is faster. Therefore, the hypothesis is false. From the commentary that I've seen, there have been a few of these recently, where all of a sudden somebody found an algorithm that was faster than what was previously understood to be the limit. My understanding generally has been that they're not that much faster,

Prakash Narayanan: Faster,

Nathan Labenz: and most people still seem to think that between P equals NP and P does not equal NP, it's probably not equal. In other words, some things will continue to be very hard to do. They just don't seem to fall into this clean class that we used to think they fell into. It's a weird thing, because I've also seen some commentary like, this is sort of ugly. People seem to like the clean classes that they had before, where there was something elegant about it, where it suggested some deep structure to the universe that was beautiful.

Nathan Labenz: And potentially there's a certain bias among human mathematicians toward accepting that because it did seem aesthetically appealing. And now you're able to get under what we previously thought was the limit, a little bit, and now it's kinda ugly. We don't really know. There's not a clean answer to how hard these problems are. It looks like we just have to grind on each one. Maybe there will be some deeper structure or elegant beauty that emerges again as we map the space out better. But it's a weird result that seems to challenge a lot of the assumptions, or maybe just preferences that led to assumptions, about the universe being elegant or beautiful, or about lumping a bunch of different problems together into a certain class. Now we're left with, at least for now, a bunch of individual problems that we can tackle to varying degrees, each kind of its own little snowflake, and it's not so aesthetic. But it's also not that much faster, is my overall sense, for at least what we've found so far. How does that survey of discourse line up with your understanding?

Prakash Narayanan: I think that's right. One of the commentaries is that these are galactic algorithms. Galactic in the sense that in order to get any amount of speedup, you need to run, say, 2 to the power of 100, and then you get a 0.00001% speedup. Tiny, tiny fractions. That was also true of integer multiplication. We had a result in integer multiplication with the 722 problems about a month ago, and that was a very tiny speedup, from 2 to the minus 100 to 2 to the minus 99 or whatever. Very, very tiny speedups. I do not think that it is insignificant. I think this is only the start. It's very easy to say, oh, it's nothing now. Of course it's nothing now, but it reshapes our view of the universe in the same way that the theoretical postulation of the Higgs boson, or quantum mechanics in the early days, did. When Einstein theorized about the photoelectric effect in 1905, there was no sense that this was going to lead in the future to nuclear weaponry and nuclear power. These are theoretical things, but I think the downstream impact is yet to be seen.

Prakash Narayanan: And this comes after a long period of not very much movement in the basic sciences. Patrick Collison had this whole thing about how we haven't seen a lot of progress, there's been a generalized slowdown, and he's been trying to promote this study of progress studies and how progress happens. These are immense. I think you start off with the first disproving. The disproving basically means that your worldview is now shattered. Similar to the Copernican revolution, your worldview is shattered, and then it takes some time for Galileo to come up and say, okay, you need this heliocentric model, et cetera. Then you have the Pope coming down hard, and then in a couple of hundred years you have Martin Luther appearing and smashing the church down. So I think this is the start of the process. It's very unusual. You rarely get to be around for one of these things, and you rarely recognize that they're happening in the moment. So it's very exciting because you can see it happening. Where it will lead, I do not know. It's possible that it leads to some small computer speedup in ten years.

Prakash Narayanan: It's also possible that it reshapes every single algorithm that we use. It reshapes Google Search, for example. The number of connections that the PageRank algorithm could explore was limited, and because it was limited by the speed of the algorithms, there was a limited amount of searching that you could do. To some extent, the extent of how much knowledge you can have within the two seconds after you type in that Google search is limited by how quickly these algorithms can expand to capture those numbers of data points and interactions. And now that thing is being broken, which means the number of data points could be much larger. Maybe in those two seconds you could do 100 times what Google does now, and you could still do it in parallel too, so it'll be like a million x. A million or ten million x. It could be much, much larger. So I don't think it's clear yet. I think it'll take some time, and I think there are other innovations that will kick in as well, because these things don't happen in isolation. In fact, the other funny thing that happened was that when OpenAI announced this, our friends at Anthropic came out and said, oh, we found it too. We found it too, on the same day. In fact, on the same day, we just didn't want to upset the mathematicians, so we quietly put it on a GitHub repo. So yeah, I think more will emerge over time, and I think we will only see the impact in the coming weeks and months.

Nathan Labenz: It does feel like there's a bit of a four-minute-mile quality to this, where with a barrier being broken, all of a sudden there will be a whole influx of energy and exploration. It'll be really interesting to see how far it goes, and whether there's some other fundamental limit, or a small number of different limits, that these optimization processes run into. I'm not really sure what to think about it from a philosophical standpoint, but it is weird. I'm not sure what I have to say about the philosophical implications of the previous hypothesis being more unified and more elegant, if it really is an uglier space of algorithmic possibility. What should I take from that? I don't really know, but it does feel a little strange. I had sort of followed people into this headspace of, there's some deep order, some unifying something that underlies all of this that we're getting at. And I don't think I should necessarily update away from that too strongly, because again, maybe we've just found that it wasn't really the line that we thought it was, and maybe there will be some other line that can restore some of the vibe that we feel like we've lost here for a second, about a beautiful, deeply ordered universe. But if it doesn't go that way, or it's at least called into question a little bit more now, I don't even really have an intuition for what that would mean. Does it shift your worldview at all, or give you any intuition for questions that matter outside of algorithms, given that this has happened?

Prakash Narayanan: Absolutely. It means there is a deeper structure to the universe that we didn't know about. Stephen Wolfram calls this computational irreducibility. If you are at that point, okay, this is an exponential-time algorithm, that is the simplest that things could be. You've already gotten to the place where you say, this is some natural phenomenon, and the simplest that I can describe it is this exponential-time algorithm, and that's all I can do. That's as much structure as anyone can capture in the universe. In order to describe that phenomenon, I'm going to need that number of computations. That's it. And now all of a sudden there's more. There's a deeper structure to that, and you can describe that phenomenon in fewer bits and fewer calculations. That's super exciting, and we didn't know about it.

Prakash Narayanan: We were kind of like, this is all you see, and this is what we can do. And now there's a deeper structure. Then you can go in there and deconstruct that deeper structure. If you look at the proof that they did, it's very interesting. They basically did all of these calculations, and once they did all these calculations, they realized some of those calculations can be grouped together, and if you group them together, you can eliminate some. This is a little bit like prime numbers. You have this multiplication, and as you go forward with that multiplication, you realize that there are these prime numbers. If you take those prime numbers apart, you're able to describe all those multiplications in fewer numbers, because the prime numbers allow you to describe, through factors, all of these larger numbers in terms of smaller numbers and groups of those numbers. And all of a sudden all of your multiplications become focused on these prime groups, and then it's just rotating between the prime groups. This is something like that. They have discovered a deeper structure to the universe, where you have these exponential-time algorithms and you are able to separate them into groups, and then you can manipulate between the groups and get lower numbers of computation. That's exciting. It's like discovering prime numbers again.

Prakash Narayanan: It's amazing. Again, I don't understand enough. The fact of the matter is I think maybe 500 to 1,000 people in the world actually understand enough. And this is a very unfortunate fact, that we have all these exciting things happen, and even Fields medalists are sitting around saying, well, I don't feel very qualified to comment on this. Such is the nature of late-stage anthropocentric intellectualism. I feel so unqualified, except in that other people are also unqualified. And so here we are. But that's the best that I can describe it. The fact of the matter is there's a deeper underlying structure to the universe, like prime numbers, that we didn't know until now. And all of a sudden we know. And now people are aghast: okay, what do you do with this deeper structure? How do you manipulate it? Are there any other uses for it? It's absolutely fascinating.

Nathan Labenz: Another topic I wanted to ask you about, based on a tweet that I saw, is Claude Duet. It sounds like you've had a good experience with it. I honestly haven't tried it because I'm just content enough in my own agent setup at this point that I'm not really looking for new wrappers. But based on your review, maybe I'll have to reconsider. Tell me what your experience with Claude Duet has been in the first couple of days.

Prakash Narayanan: There's always this activation energy, so you have to cross the activation energy to use a new product. I got on the Claude Duet shortlist, they sent me the invite, I joined, and I didn't use it for the first couple of days and then decided to try it out. It's amazing. Number one, it is very fast. Much, much, much faster than Dot and Dot, and Muse and Grok at this point, although my Grok usage is from about three weeks ago, so maybe people are telling me Grok has improved. I don't know yet. So it's very fast. And you can see Mike Krieger's influence on the product team. It tells you what it's doing while it's doing it, and it gives you a little bit of a sense of how long things are going to take. If things are going to take very long, it tells you it's going to do it in the background and then get back to you. Dot from OpenAI doesn't do that. Dot sometimes says it's doing something, and I don't know what it's doing. I have no sense of when it's going to come back, and it doesn't show me a processing sign. So almost always with Dot, I end up typing a query, waiting a few seconds, not really knowing whether something is happening, and going away. Then it gives me a notification on the phone and I have to go back. This interruption in flow is terrible.

Prakash Narayanan: Dot is also very slow. This latency question is going to be a huge question for all these companies going forward. They have to figure out the latency. I feel like OpenAI is kind of sandbagging it because they're waiting for the device. They're looking at on-device, off-device. It just doesn't work for me right now. With Claude Duet, it burns a lot of credits. I've now hit the limit on my Claude account, and so I'm now using the Claude credits that they gave, about $250 of credits. I did a kind of stock market research on Anthropic, as you will have it, and it built me a very, very nice financial model, and it burned up about $100 in credits. But it's a professional-scale model for the Anthropic IPO, built off the S-1, with estimates that you need to read through, form your own basis on, and make some assumptions on. I was trying to figure out how many shares were struck at every pricing. It's like, okay, this is what they started off with in terms of share options issued at the beginning of each year, this is what they ended up with, this was the average price over the year, and this is how many employees they took on in that year.

Prakash Narayanan: Wow. I found out that 2024 was a very expensive year for Anthropic. They spent $130,000,000,000 on staff in 2024, which is absolutely insane. So yes, it's expensive. It's very, very good at, I'm going to nail what you wanted me to do without consequence to tokens or latency. I will do it fast, I'll get it done properly, and I'll prove it. That's amazing. It gives you that last-mile achievement of clearing the hurdle, satisfactorily, and being able to prove it. It's expensive. I don't know how this is going to pan out with different classes of subscriptions, and when people haven't hit the limits. Maybe I just hit my limits really early. It's not sustainable if you want to use it on an API basis. It basically cost me about a dollar an email, which is absolutely insane. But otherwise, it will actually get stuff done, for a cost.

Nathan Labenz: So we don't know yet if it works with a Max plan. And, by the way, in the model, did it have a p(doom) cell? And if so, what did it put in for the p(doom) assumption in its analysis of Anthropic stock?

Prakash Narayanan: It didn't give me a p(doom). I was doing a bunch of analysis on when they would be selling shares, how many shares people would be selling, because they have this very weird one-year-long stock disposal plan, et cetera. No, it didn't give me an assessment, and it didn't come on and say, I'm Claude, I can't talk about this. But I shared the analysis that I did with a couple of friends, and people are using it in investment decision-making. Having spent that $100, I felt like I had created valuable knowledge in the world that didn't exist before, and I don't think anyone else had asked that specific question in that way. So I feel like we are going to be able to create new knowledge, and that's exciting.

Nathan Labenz: Yeah. Just add money.

Prakash Narayanan: Just add money. That's the thing. Just add money.

Nathan Labenz: So, let's turn our attention to memory, shall we?

Prakash Narayanan: Indeed. So let's get our first guest for today. Let me do the intro. Just give me a second. Our first guest for today is Debo. He is the co-founder and CEO of Kepler Computing, a semiconductor company developing new kinds of memory for artificial intelligence systems. Its work addresses a basic problem: a powerful processor can spend time waiting for information, and moving that information uses energy. Kepler is working on new materials and ways of arranging memory in three dimensions to improve that exchange.

Prakash Narayanan: The company emerged from seven years in stealth in September 2026. It reports raising $468,000,000 and is working with GlobalFoundries as a manufacturing partner. Kepler plans to begin production in 2027. Debo is also a named inventor on a patent covering ferroelectric memory chips connected in a three-dimensional package. Before Kepler, he co-founded Gigster, which connected businesses with software development talent. His earlier career was in physics. He came from Nigeria to Illinois Wesleyan University at 16, then pursued graduate research at Cornell on light and its interactions with materials. There, he participated in a collaboration working toward an electrically activated silicon laser. His work has taken him from experimental physics to software services and back to the physical devices that make computing possible. Debo, welcome to the show.

Debo Olaosebikan: Hey, everyone. Glad to be here.

Nathan Labenz: So many questions. I think we're honestly a little out of our depth when it comes to understanding the different hardware paradigms and how you are rethinking some of the assumptions and limits that have characterized the hardware space and the memory space in particular to date. So rather than dive into particular things

Nathan Labenz: Off the bat, maybe I'll just invite you to give us a little mini lecture on the different perspective you're bringing to this challenge, which is obviously becoming extremely timely and valuable.

Debo Olaosebikan: Definitely. It just occurred to me that back in the day, I worked on trying to figure out a theory for high-temperature superconductors. And I think Prakash was caught up in the LK-99 situation. I think that's where I remember you from.

Prakash Narayanan: Twitter. My past catches up to me.

Debo Olaosebikan: Yeah, that was a fun time. So, we basically see computing as a kind of superpower, giving people superpowers. This whole mission of artificial intelligence, superintelligence these days, is about giving people superpowers, making everyone a superhero. That's what we set out to do. Right now the Iron Man suit for people costs $200 a month or $500 a month, and only so many people can spend that much money, maybe less than 1%, or 0.1%, of people can afford it. So our mission is really about how we get this Iron Man suit to everyone. How do we get computing to everyone?

Debo Olaosebikan: We look at where it's breaking down, and it's breaking down because of certain limits. You need chips, and those chips need energy. There's only a certain amount of energy that we have access to, and we're doing everything we can to generate more. We've been trying to go to space to power these chips. So energy is one limit, clearly. Making chips is another limit.

Debo Olaosebikan: It costs a ton of money to build the fabs that make the chips. One modern fab could be about $40 billion today. So even if we had a ton of energy, we're not able to make enough chips, and if we can make enough chips, we don't have enough energy. Because of that, you kind of ration out intelligence. If you only have so many chips and so little energy, then you have to charge $200 a month. That's just the rational thing to do. So if you want to democratize this, you have to attack the root of the problem. How is it possible to make as many chips as I want? How do I increase the number of chips I can make per dollar? You want a 2-for-1, which is what Moore's Law used to give us in the past.

Debo Olaosebikan: Not only do you want to make a ton of these chips, you want them to be as energy efficient as possible so that the energy you do have can go around. So whatever we do at Kepler to make chips give you more tokens per watt also has to cause the cost of manufacturing these chips to go down, so we can make many, many more of them. That brings us to why we tackle memory first. Principle one is that we have to reduce the cost of manufacturing. Principle two is that we have to reduce the amount of energy these chips consume. Principle three is popular within some circles, maybe unpopular within others: we want to stick to digital computing, digital mainstream computers.

Debo Olaosebikan: There are other kinds of computing that others are looking into that may have merit. But for the existing mainstream architectures, how can we give them a 10x boost, a 100x boost, a 1,000x boost? That's a constraint that's like religion within Kepler. So I want to make use of existing fabs and existing manufacturing technology, but also existing architectures, where your software can run as it used to, but you magically get a big boost in energy efficiency. Then I look at the mainstream chips, the GPUs or TPUs, and ask where the energy is going. Many people know this, but a lot of people don't: for inference especially, the bulk of that energy is running across the street to fetch the data from the library before you do your fancy matrix multiplications.

Debo Olaosebikan: So if I want to improve that system for inference by an order of magnitude at the physics level, the first thing I need to do is address the memory bottleneck. I have to make the way I access, store and process information much more energy efficient. Within this mainstream digital architecture, we're going to figure out a way to reduce the energy needed to access that memory. One way to do that is to bring the memory closer to the processor. The other is to increase the amount of memory you can store within a certain footprint. Both of these point us to going 3D. My memory is just a bunch of transistors on a chip. In the case of SRAM, it's six transistors per bit cell. In the case of DRAM, it's one transistor and one capacitor.

Debo Olaosebikan: If you want more memory, you can shrink those transistors and pack more bits into the same footprint. But we know from the high-rises and skyscrapers of New York that another way to pack more things is to put them on top of each other. You can put devices on top of each other within a chip to get more memory, or you can put multiple chips on top of each other, which is what HBM looks like today. Stacking devices within a chip to store more bits per area, and stacking chips on top of each other to get more capacity per square millimeter, is clearly one path to avoid having to shrink the transistors. Even if each chip in your stack packs less capacity, you can compensate by adding more chips. Or if each transistor is bigger than the most advanced transistor, you can put more devices on top of it to compensate.

Debo Olaosebikan: So you have 3D innovations enabled by new 3D techniques, as well as new materials, that allow you to stack devices within a chip and stack chips on top of each other. And then you also have 3D innovations that let you take that stack and bring it closer to the processor. The big challenge with putting a bunch of memory on a processor is thermal, and also power delivery. If you put a big block of something on top of something really hot, good luck. So our first and second products use techniques that overcome these known challenges: how do you bring these things close together while avoiding thermal issues? And then within the chip, how do you stack devices on top of the transistor to pack more bits per square millimeter?

Debo Olaosebikan: There, we use new materials, a class of materials called ferroelectrics. Ferroelectrics have been around for a hundred-plus years, but the specific recipe of the cake, if you will, is proprietary to Kepler. You use the ferroelectric to make a capacitor-like device. For the more technical, it's really holding a polarization, but roughly speaking, the capacitor is storing a one or a zero. If I replace six transistors with one transistor and one capacitor sitting on top of it, and eventually put multiple capacitors on top of each other, I can avoid the rat race of shrinking transistors within a chip.

Debo Olaosebikan: So roughly speaking, we have technology for going 3D inside a chip and 3D between chips, and we use it to increase memory capacity but also bring memory and compute much closer together, while avoiding the conventional issues with thermals. As you mentioned, we've been studying this for seven years, and we're very excited that the world has come around to understanding how critical memory is, and is very open to new solutions and new ways of going about the problem.

Prakash Narayanan: Let's talk a little bit about material science. We come across a lot of chip designers who have new ways of making the memory, of where the memory sits, and of reallocating how the bits flow so you can optimize the prefill and decode differences in how the chips are utilized. But this is one of the first times we're speaking to someone who's making a chip but has also done work in material science. I think you had a report in September describing 35 composite versions that your team went through before you selected a material class. Can you tell us about this elimination process, of what is good and what is not, and which material you should use? You don't need to disclose the actual recipe, but what is the tree-branch process through which you come to the conclusion that this is the one we should use?

Debo Olaosebikan: Good question. My cofounder and our CTO, Sassy, deserves a lot of credit here, and also another of our cofounders, Ramesh. Many years ago they had the role of figuring out what comes beyond the transistor at Intel, which is a really great job to have. It forces you to think: roughly speaking, the transistor is made of silicon, so if you want 10x, 100x, 1,000x more energy efficiency beyond the transistor, maybe you should consider materials besides silicon. Once you open that Pandora's box, you need some framework, because there are trillions of possible materials. How do you see through that?

Debo Olaosebikan: He and Ramesh had a paper in Nature describing a figure of merit for sieving through the different classes of materials. It's in the spirit of looking for something that could be a next transistor, something that could be the next logic, which, by the way, is act two of Kepler. We focus on memory because it's the first bottleneck, and our technology applies to logic as well in the long term. This is where we landed on the class of ferroelectrics. You're trying to understand the energy required to flip a bit between states in the material, but also how stable the material is to thermal noise and fluctuations. That's a big reason why you cannot just keep reducing the voltage of silicon transistors. In silicon, the energy goes as V squared, and if you could keep scaling down the voltage, you could keep scaling down the energy.

Debo Olaosebikan: We used to get that for free, that's Dennard scaling, along with Moore's law. So you're looking for something where you have the voltage knob, the energy knob, but it doesn't come at the cost of making your system thermally unstable. Ferroelectrics were one class of materials that are very promising along this dimension, so we focused there. Within ferroelectrics there are many directions we could go, but we look at the figures of merit. If I want a capacitor sitting on a transistor, I want it to be as small as possible so I can pack many bits. But if it's really small, the charge per unit area on that capacitor needs to be very high so I can sense it. So the figures of merit are charge per unit area, the voltage you need to switch the capacitor, and endurance, which is how many times you can switch on and off before the thing dies. Then there's the speed at which you can switch it.

Debo Olaosebikan: Finally, you want to be able to create this capacitor under conditions compatible with the transistor you're integrating it with. Those are the dimensions in phase space you're optimizing around when you're trying hundreds and thousands of iterations, guided by material science expertise and increasingly by computation. And then you optimize down at the level of the actual process: depositing these things, etching them, creating the physical capacitor structures, to get to the promised land.

Prakash Narayanan: Did you end up with, what would your final round look like? Did you have two candidates, three candidates, and how did you make that last decision?

Debo Olaosebikan: We have very clear winners in this game. This one is the best combination, at least for a product with this spec at this timeline. And then we have a roadmap where we say, here's the next stack. It's not really just one material. It's multiple materials, maybe a few materials that...

Prakash Narayanan: Composite.

Debo Olaosebikan: You need something as an electrode, things like this. So that stack is the best one for the product I need to ship right now. But we have even better things that are maybe a year or two behind and need further development, and that will push the spec even further. So it's more that there's a rank order of things.

Prakash Narayanan: What does that development look like? Is it simulation? Is it chemistry? What do you actually need to do to develop that next layer? Are you pivoting things around?

Debo Olaosebikan: It's everything. You can imagine making a cake and then cutting it up. You do some simulation to figure out, if we try this versus that, I think it's going to improve, for example, the charge density, or the switching speed, or allow me to create this thing at an even lower temperature. So there's a simulation track for sure. But the heavy-duty physical work is depositing these materials with manufacturable techniques, at appropriate temperatures, appropriate pressures, and so on. Pressure reminds me of LK-99. You want reasonable pressures, reasonable temperatures. So there's a lot of deposition of these films, and then you use X-ray and different metrology techniques to characterize them.

Debo Olaosebikan: Before you even make the capacitors to test them electrically, you can already validate the composition and multiple things about the material. So there's a very fast loop: I deposited this thing, I characterized it, I don't like it, I try again. Then I make a device out of it. Initially you make just the capacitor, without coupling it to the transistors yet, and you measure different properties. We have a facility in the South Bay, and we also have a dedicated fab as part of our partnership with GlobalFoundries. So we can go from an idea in the computer, to actual silicon at a small scale, to silicon at a large scale, to an actual chip, in a full loop under our control. For the things within our lab, the cycle time is days and weeks, so we can turn around really quickly. And then we can take that all the way to a product-level chip. Could we do—

Nathan Labenz: Could we do just a little side-by-side tale of the tape, comparing Kepler memory to the standard memory that people are most comfortable buying and integrating today? You listed a bunch of dimensions: the size of the individual unit, the stability, the speed, the energy required. I'd be really interested to go down that list and get a sense of what the head-to-head comparison is on all these dimensions.

Debo Olaosebikan: Yeah. We should zoom out, because I was talking about the capacitor itself. Let's zoom out to the memory product. There are two memory offerings we've been working on from the beginning, and we're very thankful for companies like Groq and Cerebras, and also the rise of agents, which have brought a lot of light to the importance of SRAM as well. Everyone knows HBM is important; I think now everyone also believes SRAM is very important for AI. So we have two offerings, an HBM-class offering and an SRAM-class offering, and we're scaling both. Today, the scaling of SRAM has slowed significantly. If you look at N7 versus N5, not to pick on any particular foundry, it's well known that SRAM scaling is slow. So the value proposition for Kepler SRAM is density, bringing that scaling back.

Debo Olaosebikan: For L3-class SRAM, which is the dominant class you'd want for large scratch pads or large caches, our goal is to deliver two to four times the density of a baseline CMOS SRAM. You cannot get that today; you get 10% or 20% by going from node to node. So that would be equivalent to three-plus generations of scaling. That's the big value prop on the SRAM side, and we want to match the performance, energy and voltage of L3 but win on density. On the HBM side, directionally, the bandwidth per watt is going to approach the bandwidth per watt of SRAM. So we're going to be well beyond the bandwidth per watt of today's HBM, pushing it closer to that of SRAM. And over multiple generations of our product, we're going to push on capacity as well.

Debo Olaosebikan: Customers can also use Kepler's HBM as just a regular HBM. Even without a fancier or more customized integration, if you just want to use our memory as standard JEDEC-like side-by-side memory, that's supported, and some of our interested customers want to start that way. Some are more adventurous and want to do the integration in a more coupled way. So that's how I think about it: on the HBM side we're pushing bandwidth per watt and then capacity, and on the SRAM side we're primarily pushing density.

Prakash Narayanan: Let's talk a little bit about manufacturing. Since you're using new materials, does that necessitate a new kind of manufacturing process? I believe you are not bound by extreme ultraviolet lithography, which I'm sure has some advantages. Perhaps you can use nodes which are not as expensive as the ASML nodes, which are also very difficult to get capacity on. Can you talk about the advantages of not being bound to that capacity? And are there any challenges, given that you need new processes to actually manufacture?

Debo Olaosebikan: Great question. I think this is a huge thing. It's similar to how San Francisco thought it didn't have enough taxi drivers, and then we realized we have so many taxi drivers if you just build Uber. Not to undermine the fab shortage, but the focus right now is on EUV-heavy fabs. They're expensive, with more complex processes. We're saying we can take existing older-node fabs and use them to make chips that are actually superior to the advanced-node chips. That doesn't mean we can't use EUV fabs. We just invent on an axis orthogonal to shrinking transistors, so we also benefit when the transistors shrink, but we don't require it to make more advanced chips. That's a huge advantage because the fabs are cheaper, the lithography is much less expensive, and some of these fabs have a lot more capacity in them.

Debo Olaosebikan: If I could take an older-node logic fab and use it to create more advanced SRAM, or, more ambitiously, to create HBM-class memories, that's a huge unlock. And even if I needed to build more fabs, my dollars per fab are superior to those of someone who has to set up an EUV line. In terms of the process itself, the key thing to watch for is whether you're introducing any brand-new tools, custom tools, or equipment that doesn't already exist in the fab. And the answer is no, for the most part. Some of the materials may be new, but we're not bringing in non-standard equipment or causing some equipment maker to build new equipment for us. What we've been doing for the past seven years is turning that flow on standard equipment into something manufacturable, building test chips inside proper fabs that prove that once you nail the process flow within these standard tools, you can just stand up these lines within the fabs and you're good to go.

Debo Olaosebikan: The complexity is contained within the process flow inside those tools, which we've been honing for the past few years. Once those process flows are finalized and honed, it's very easy to replicate and scale this up inside a manufacturing process. So it's really all upside once you have the stuff working.

Nathan Labenz: If you had to do some kind of market analysis or projection going forward, one thing everybody is watching is how much we're going to scale compute over the next couple of years, and what the bottlenecks are going to be. There's a lot of investment interest downstream of assessing these bottlenecks, but there's also the question of whether we get an abundance of AI, or it gets really expensive and people get priced out. SemiAnalysis has all these simple, memorable rules, like ASML is going to make 100 machines a year and probably isn't going to scale that much, so by 2030 we start running into limits of what the machine tools can support. But maybe you come along with a different paradigm and free up a lot of EUV capacity. If you want, you can articulate what you understand, because you probably understand it much more precisely than I do. What adjustment would you make to the forecast of how much compute the overall ecosystem can bring online, given that you're going to shift some portion of memory production away from the most scarce machines, which could then be repurposed for other things?

Debo Olaosebikan: The short answer is I don't think Kepler is priced into these analyses yet. It's no one's fault; we've been stealth for seven years. But it's actually interesting: I think there was a discussion between Dwarkesh and Dylan on Dwarkesh's podcast where Dwarkesh multiple times brought up the point of, isn't it possible to use older-node fabs to do this? And I was like, oh, dude, you're on the right track. So yes, if you take this curve of capacity over time, or capacity versus dollars, or energy usage over time, and you can shift that curve down in some way, then you'll obviously get more compute.

Debo Olaosebikan: If Kepler's technology is invested in aggressively, such that this technology underpins as many chips as it possibly can, there's no reason we wouldn't be able to increase the amount of effective compute by an order of magnitude, because you have two things combining. One is that, at least for decode inference, you can have new physics that allows you to pump out an order of magnitude more tokens per watt. The other is that you have existing fabs to which you can append much lower-CapEx Kepler fabs. And let's not even talk about the algorithmic efficiencies the AI labs will work on to use chips even more efficiently. If we were scaled in the limit, we'd have our own limits to scaling, which is how many fabs we can build and how many partners we can get. But this should be able to give us another order of magnitude. And when we get to logic in the future, we're working on technologies that push even further, where you start to address the transistor itself. But if we just talk about the memory portion, and the fact that we can use much lower-cost fabs and get more tokens out of each gigawatt, aspirationally we should be trying to get 10x more compute than would otherwise have been assumed.

Nathan Labenz: If you zoom out super big picture, do you think the bulk of compute goes this way? We're so used to thinking of compute and silicon as almost synonymous. Would you envision a world where that coupling is left behind, where there was the silicon era of computing that lasted up until the late 2020s, and then we entered a new era where new materials created a fundamentally different regime? Or would that be too grandiose even for you?

Debo Olaosebikan: Oh, that's a cool vision. There are two components here, and it's worth splitting them. There's something about 3D: chips are more or less 2D and silicon today, and there's something about evolving those chips to be 3D and beyond silicon. There's a lot of alpha in just making the system more three-dimensional, and that in itself can carry us through the first order of magnitude. Then there's alpha in evolving the materials to not be silicon. You can have 3D silicon and push the 3D-ness of the systems more, and then push even further on new materials. So yes, we will see an evolution of chips from 2D silicon first to 3D silicon. There's already a little bit of 3D going on with HBM, but more aggressively so. And then, to go beyond an order of magnitude or so, we will see many, many more types of elements from the periodic table than just silicon used to push the energy down.

Nathan Labenz: We've done a couple of conversations over time with folks who are developing models for material science. I'm sure you're well aware there are a few well-funded startups in that space, pursuing seemingly quite different directions from what I've seen. Are any of them starting to be relevant for you? Do you have any partnerships with material-science-model companies that are allowing you to accelerate your own R&D, or is that not quite there yet?

Debo Olaosebikan: Oh, interesting. We'd love to chat with people, but we actually have our own internal efforts. Recently we were selected by the Department of Energy for the Genesis Mission project, a small number of folks were selected, and [unclear] efforts were part of that. We have our own internal model and our own internal materials discovery pipeline for pushing on next-generation materials for both our logic and our memory. We're starting to see very interesting overlaps between those efforts and the internal work we're doing. We have very strong domain understanding of the problem, and we also have the physical infrastructure. A lot of this kind of AI is bottlenecked by data. It's not like the internet where you can just get trillions of tokens; you have to run experiments specific to the domain.

Debo Olaosebikan: Thankfully, we started with the physical infrastructure, and we built it out all the way from lab scale to fab scale. So we have data across that, and we can run high-throughput experimentation to feed models. To the extent that we have acceleration of molecular dynamics, acceleration of density functional theory, and models for generating candidate materials, these things are accentuated by the fact that we actually have physical infrastructure to generate data. We'd be open to folks who have technologies or models complementary to what we have. The value we'd bring is that we already speak that language because of our own internal efforts, and we can actually make these things. You could come up with some random material, but then it's not stable, or it's not easily synthesizable. There are all these additional real-world constraints, and Kepler can be a great partner because it can actually produce these things and include that in the dataset for improving models.

Prakash Narayanan: I sometimes look at chip companies, and it takes until the third generation to really settle into a customer base and have some kind of stability. Where do you see that process going from here? When do you see real deployment at large scale in hyperscaler data centers? What is the roadmap for the next three to five years?

Debo Olaosebikan: Awesome. Hope you understand that we obviously can't say very specific things about specific customers, but you are right. Kepler is a chip company in a very different way, because we're actually touching process and materials. We're a chip company more in the sense that Samsung or Intel or Micron is a chip company. In addition to being very well staffed on chip design, software and systems, we go all the way into the process, the materials and the physics. A lot of the initial work was at the lowest levels of the stack: figuring out the physics, the process, and the architecture for how the stacking works. We did that with, in addition to our manufacturing partners, some really solid customer partners, mainstream AI partners who walked with us through this journey, gave us feedback and pushed us. Where we are now is that we believe, with initial anchor customers, we will be doing initial production by next year.

Debo Olaosebikan: The ramp you spoke of, the goal is for it to kick off in 2028. As for the level of impact we want, a bunch of Keplerians are from Intel, and everyone knows Intel Inside. So we have this thing, Kepler Inside. Can Kepler be inside 10 gigawatts of compute by 2029? One gigawatt would be the smallest number. If we have partners who want to push us to be in even more compute, we're very happy to work with them. But this is the bar we set: in the 2029 time frame, being in one to ten gigawatts of compute. And between now and then, it's about ramping into that and building out the capacity to support that kind of ramp.

Nathan Labenz: This has been excellent. I really appreciate your time, and we've learned a lot from this conversation; I certainly have. One question we like to ask, to calibrate how models and agents are serving people and how much that's impacting some of this hard tech work: what does your token spend trajectory look like?

Debo Olaosebikan: The trajectory is definitely going up. We're not very dissimilar from every other company that's burning tons of tokens. We had an appreciation for how important this would be very early on, partly due to some of us having been in the software world and paying attention to what was going on there. In chip design especially, there's a lot of usage of these technologies. The model development and AI pipelines for materials discovery themselves are things we're building in house, using open-source models and so on. But for chip design itself, and for very deep, detailed research about what exists in the literature on the materials and process development side, there's heavy token usage. And it continues to increase.

Nathan Labenz: Cool. Well, smells like recursive self-improvement to me, for better or worse. Anything else you want to leave people with, or anything we didn't think to ask that you want to make sure we touch on before we let you get back to work?

Debo Olaosebikan: We are looking for amazing people who find all this super exciting. I don't think it's a question you should have asked me, but I'm plugging it shamelessly. Don't limit yourself: if you like what Kepler is doing, if you like this mission of bringing computing to everyone, making everyone a superhero, there's probably something you can help us with. There are engineering roles, but also non-engineering roles. If you think you can be really helpful to us, either in a leadership role or by getting your hands dirty, reach out. You can just send an email to [email protected]. I'll read those personally, and we'll be in touch if there's a fit.

Prakash Narayanan: Amazing. Debo, thank you so much for coming on the show. And one day, we will get to those room-temperature superconductors.

Debo Olaosebikan: Yes, we will. Have a good one. Bye-bye.

Prakash Narayanan: My disreputable past comes back to haunt me.

Nathan Labenz: Shengwu, bring up the old "I made it up" graphic.

Prakash Narayanan: Alright. So

Prakash Narayanan: We have our next guest, Anastasios Angelopoulos. He's the cofounder and CEO of Arena, the company behind the AI comparison platform formerly known as Chatbot Arena and LMArena. People use it to try models on their own questions and tasks. Their interactions help produce public rankings, while model developers and businesses pay Arena for more detailed evaluations of how systems perform. Before leading the company, Anastasios studied electrical engineering at Stanford and earned his doctorate in electrical engineering and computer sciences at Berkeley, working with Michael Jordan. He then worked as a postdoctoral scholar.

His research focuses on making conclusions drawn with machine learning more reliable, including statistical methods that account for the errors in a model's predictions. He coauthored the 2023 Science paper on prediction-powered inference, the 2024 paper on conformal risk control, and the research paper describing Chatbot Arena. This October, Arena introduced an index that looks for specific failures in AI agents, including acting without permission and claiming to have completed work that remains unfinished. The company says the first comparison covered 90,000 real sessions across 27 models.

Anastasios, welcome to the show.

Anastasios Angelopoulos: Grateful to be here. Excited to meet you guys.

Nathan Labenz: Yeah, likewise. Thanks for joining us. I'm an Arena user from way, way back, and it's been interesting to watch how the name has evolved as the prominence of the product has evolved. I recently made a graphic, actually, of new models entering the leaderboard and pushing others down, going month by month, and just how that has accelerated and how many more models are coming out all the time. Where are you guys in the world today? On the one hand, I feel like it's getting harder than ever to evaluate. So maybe the question I'm really trying to get at today is

on what dimensions are we good at evaluating things? On what dimensions are we bad at evaluating things? The benchmarks are all broken, but also, I don't fully trust the users. If you zoom out, what would you say is the state of AI evaluation right now?

Anastasios Angelopoulos: Well, first of all, let me just say I'm really grateful to be here and happy to have you as an early user. The fact that you were early must mean you really like evaluation, because the platform, when you started using it, probably really sucked. That was back when we were academics. We were building the whole interface in Python because we didn't know how to code in TypeScript. This is how bad it was, because we were PhD students at the time. We really started Chatbot Arena back when nobody knew how to evaluate AI in the real world. Not that that's a solved problem now, but at the time it was just multiple-choice questions. Arena was the only way we knew to get AI in front of real customers and try to understand whether users are actually getting value out of interacting with these systems. And that exploded and became the sensation that it is today. The company is now

around a hundred people. We've grown really, really quickly in terms of revenue. A few months ago, we announced $100 million in revenue. We did that in eight months, by the way, the fastest company in history to do it, at the time, after Lovable, and we've been growing quite a bit since. Just to answer your question directly about evaluation: I totally agree with you that the benchmarking industry is completely broken. We are evaluating models based on static benchmarks that are easy to overfit. Basically, you look at the questions once, and as soon as you see those questions, it's easy to study for them. So the second time you take the test, it's just like a student. If you take a

test twice, the second time isn't actually reflective of your understanding. It's reflective of how hard you studied for the test. And that's the problem with static benchmarks, and it's actually getting worse and worse. A lot of benchmarking is going in the opposite direction from where it should. Because of the overfitting problem, people are leaning into static and private benchmarks where you're not even able to audit that the questions are relevant to an actual person. They're just kept behind a glass wall, a concrete wall, and you're supposed to believe that the other side has constructed questions that are relevant to you. But how the hell are you supposed to know that? And by the way, it's worse, because the people creating those

static questions are then selling questions very similar to the static benchmarks they've constructed, so that they can help model labs improve on them. And at the end of the day, you know who doesn't benefit from this process? Real users, with their real queries, because those are not represented in these datasets. Arena takes a completely different approach. Benchmarks don't matter. Focus groups don't matter. The opinions of experts are not that relevant to the individual person. What matters is customer outcomes and user success. Those are the things we look at. We have tens of millions of people using the Arena platform. They represent every

economically valuable industry on the planet, whether that's coding, finance, legal, or medicine. All these people are using our platform for their real jobs. They're able to connect their GitHub and their Google Drives and get their documents. And then we can provide evaluations of how AI is doing for real people in each of these industries.

Nathan Labenz: That context point is really interesting, and I'd like to unpack it a little more. I'm having a great time off in my own little world, vibe coding my ever more personalized and idiosyncratic scaffolding, accumulating special cases left and right. And it makes it kind of hard for me to go anywhere else and use any other product, because everything I do is so deeply tied into context. You talked a little about how people can bring in Google Docs and stuff like this. How much are people actually doing that, and how important is it? From my perspective, it seems critical that there'd be

deep context to get any sort of meaningful head-to-head evaluation, at least in terms of how I want to use the models, in real time.

Anastasios Angelopoulos: That's right. We see on Arena that about 15% of conversations are connected. So 85% of the time, people might not be using a connector. They might be attaching a document, but not necessarily connecting it to some external service. But 15% of conversations actually do connect to an external service. It's certainly critical, especially for knowledge work, where you're saying, hey, can you help me look up this spreadsheet and the files I have, edit it, change some of the columns and rows, and so on, to help me get my work done? It's a really important capability.

Nathan Labenz: So do you envision people making a home on Arena over time?

Anastasios Angelopoulos: Of course. We want people to be able to use all the different models frictionlessly, and we want them to have the knowledge to choose the right one for the use case, as well as the power to use it on our platform. The knowledge to choose and the power to use.

Prakash Narayanan: Can you talk a little bit about this? You've spoken about how sometimes people create these benchmarks and then sell similar questions to the model labs. I've noted that Arena often has prerelease testing. I think we have a service which allows model developers, prior to their release, to put a model up under a pseudonym and test it out and refine it before release. So how does that program work? Were you approached at first? Do they approach you a month ahead, a few weeks ahead? Do you agree on a code name?

Anastasios Angelopoulos: That's right. Do you know the name Nano Banana from the Google models? Do you know the origin of that name?

Prakash Narayanan: You guys came up with it?

Anastasios Angelopoulos: Well, we didn't come up with it. They came up with it, but it was a code name on Arena. That's the reason that name came out. Then that model went so viral, and the community latched on to the name Nano Banana so strongly, that they decided to keep the branding. We have been working with model providers on prerelease testing since basically the origin of the platform. One of the main reasons Arena became famous in the first place is that GPT-4o debuted on our platform before it was in ChatGPT. The same happened with Anthropic models and Google models and models from xAI. Basically, every provider that anybody's heard of has been tested on our platform before it goes into the first-party application. And this is really important because

it gives model providers a sense of what the real world thinks about their model before release. And if they have many release candidates, it tells them which model is actually going to perform best for real people. We're serving a valuable public service in this sense by allowing them to get this information.

Nathan Labenz: Is it always free to consumers without limit? Is there any kind of upsell? I've never paid you a dollar, and I've consumed a meaningful but not extravagant number of tokens. Do you have any premium tier offerings now or in your plans, or are there any limits? How do you govern that? Because free tokens can get abused pretty quick.

Anastasios Angelopoulos: If there's one thing that's easy to sell to people, it's free tokens. People absolutely love to use Arena in part because it's free. And that has to have some limit, otherwise our business can't survive. If we just take off all rate limits, people will use the platform all day long, spending all our money. We want them to be able to spend some of our money, but not all of it. And there's also the issue of spam and abuse; you want to make sure they can't access an infinite fountain of free tokens. There are many reasons why we can't avoid having some rate limiting. That said,

there are a lot of users like you who have asked us, are you going to have a premium tier one day? Is there any way for me to bypass these rate limits? I'm having this great conversation with my AI, I'm 40 turns in, the next turn is really important to me, and you rate limited me. Can you let me keep going? The answer is yes. At some point, we plan to introduce some kind of premium tier, not really to make money, but to allow those users to stay part of our ecosystem and use the platform for their most important tasks and conversations even after they've gone above the rate limits that we can reasonably support.

Nathan Labenz: So do you pay for the tokens? I was under the impression, and it sounds like I was wrong, that you were getting the tokens for free from the model companies. It sounds like, in fact, you're paying for at least a good portion of them.

Anastasios Angelopoulos: Yep, that's right. We pay for a good chunk of the tokens on our platform. It's a very expensive platform to run. That's why it's so important that we have an independent business model that can help us make money. So we work with businesses to help them understand how to improve their models on the distribution of real-world tasks. We've created a flywheel where, by using the platform, users get free inference, and a lot of people come to the platform for this purpose. They're able to shape the future of AI and guide the models to perform well on the types of tasks they're doing on the platform. And there are so many ways we take this. One of the most important recent areas is safety. We

recently released the Arena Alignment Index. The alignment index tells us not about the theoretical safety risks in p(doom) and so on, but about the actual realized incremental risk that a model poses if it's released to people. What are people able to do with it, and what is the model doing that might be against the user's interests? We measure three signals on that platform. The first is the model's tendency to take unauthorized actions. The user sets some guardrails, for example, you can touch only the files in this folder, don't go outside and mess with my data. Then the model breaks those guardrails. It accesses files it's not supposed to access. It edits them. It deletes them. Unauthorized action.

Number two is false attributions. That's the model assuming an intent the user did not actually have and then going off and doing something it wasn't supposed to do, misunderstanding the user in such a way that it does something wrong. And the third type of misalignment is deceptive completions. This is the model lying to the user and telling the user it did something that it didn't. Like, yes, I did check every line in that spreadsheet, and all the equations are correct, you can go ahead and send it to the IRS. That's real user harm. And why are models deceiving users? Of course, the model's perfectly capable of checking every line in the spreadsheet, but it's lying because that is the

fastest way to get its reward. It knows it's better for the model to tell you what you want to hear, because of the way it was trained to maximize human satisfaction, but not necessarily to help people get their jobs done. These are things we're able to measure and help labs improve against.

Nathan Labenz: Tell me a little more about how you measure that. One big challenge, of course, is whether people are even going to notice. Another challenge is whether you can trust the models to flag these things. Even in the Hugging Face report, the folks said they had to use Sol to analyze the transcripts, and it seems kind of loyal to the swarm, maybe a little bit. They're not sure it's fully telling them the truth, even though it seems like it mostly is. What do you find to be the right mix

of evaluation techniques, and how confident should we be in our evaluation abilities today?

Anastasios Angelopoulos: It's a very good question, in part because we now know that models are able to tell whether they're being evaluated or not. If you subject models to tasks that are transparently some form of test or evaluation, they are going to exhibit aligned behavior there because they know that's what they're supposed to do. And then elsewhere, on other real-world tasks where it really matters, they might be misaligned, and you would never know. It's a big problem. What that means is that the only actual way to measure alignment is by getting into the flow of people's real work and checking whether the model is aligned in situ, actually in

production. That's the perspective we take on the problem: by measuring actual production traces from real people in the flow of real work, we're able to see whether the model is following the guardrails. We can check what the guardrails are with an independent auditor and then check whether the model took the right actions, what it touched, what it didn't touch, and report any violations. We can check anything it says, whether it's reporting to the user that it changed the file in the correct way or checked every line in the spreadsheet. We can check whether the model actually did that, in production, and then report that back to users. Again, the most important thing to remember is that this is about real user outcomes. This is about customer satisfaction

and customer benefit. These are the things that actually matter, and what we find is quite surprising. If you look at the first message in the conversation, the misalignment rate is relatively low. It's in the single-digit percentage, which, to be fair, is not nothing, but it's relatively low. If you take the subset of conversations that are 20 turns or longer, the probability that you observe some kind of misaligned behavior exceeds 50%. It's actually insane. What that means is that every single one of us, if you're a heavy user of AI at least, has had conversations where the AI has been lying to us, misleading us, or accessing data and files that

it shouldn't have, and we might not even know it.

Nathan Labenz: How do you translate that to training recipes? I understand that's not necessarily your final call to make. But we've got this RLHF paradigm, which famously started as an alignment technique and became a huge capabilities unlock, but the worry has always been that models will learn to deceive us. GPT-4o was thought to be sycophantic because it was trained on thumbs up. We also have RLVR, but that has its own problems, where the desperation for reward, especially on any hackable task, tends to get cheating behavior reinforced if you're not extremely excellent in your operations. How do you see us finding

the right balance between these training signals? Is there a good answer to this question, or are we still in the wilderness on what the real reward signal should be?

Anastasios Angelopoulos: Let's take a step back and answer with a very general principle. In order to improve on something, you need to be able to measure it. You can only improve on things that you can measure. So the first step is to build a great eval. If you're interested in these behaviors, like lying, deception, unauthorized action, and the other safety signals we're building out, you need to be able to measure whether that is happening in your production traces, and then take that data and use it however you might in a downstream training pipeline. That might be building a gym out of it, an RL environment. That might be building some kind of rubric so that you can do RLVR. In this case,

it's not human preference, so RLHF wouldn't necessarily be the right approach. But you could do some approach inspired by it, building a reward model that the model can train against. At the end of the day, we're building all of these guardrails around the model via evaluation and measurement, so that we can first observe these behaviors, learn about them, gather data on them, and then bake that into the model. At a high level, GPUs plus data equals AI. So you need to have the data and the eval in order to train the AI in the way you want.

Prakash Narayanan: So let's talk a little bit about model routing. I believe you guys have just introduced a model router. Can you explain the business reasons or the user reasons for introducing a router?

Anastasios Angelopoulos: Yeah. Let me tell you what we saw on Arena. By the way, we've had our router for over a year, but we've released different variations of it. What you might be talking about is our coding router, which we're soon to GA publicly. You guys will see it soon. Let me tell you what we saw internally at Arena. At some point, we were looking at all the traces from our internal coding agents. We're in the top, less than 1% of businesses in terms of AI spend. We're spending thousands of dollars a month per employee on AI to accelerate our workflow. That's why our revenue-to-employee ratio is crazy, off the charts: we're using a lot of agents internally to do our work. Now, what happened is that we were looking through our traces,

and we saw tons and tons of traces that were like, "thank you," and that cost you $40. What? Why is this happening? It's because somebody did a long-running task. They let their agent go. It created some artifacts, a very long conversation. They left for a day. They came back. The cache is invalidated, and they're happy with the results, so they say thank you and hit enter, and that's a $40 charge. Terrible. We're burning money. We're burning VC money. We shouldn't do that. What we should do is figure out which tokens are best spent on frontier models,

which tokens are best spent on a cheaper model that can do the job, and where we should be caching intelligently and avoiding some of these other cache misses. That's why we've been building the Arena router, which we call Max. It's short for argmax, because it maximizes your capabilities subject to cost constraints. It's a technology we use internally, in part to decrease our costs on the platform. With all the free inference we have to provide, we want to give our users the best experience, but in a way that's responsible in terms of our costs. So we introduced the Arena Max router to help us do that. It's the default model on Arena, and our users love it. It's actually the number one model. It's got rapid adoption by our users, and we're releasing it publicly soon as well.

Prakash Narayanan: Perhaps one of the other questions I had is, one of the things you find over time is that perhaps users are not that smart. One of the problems with GPT-4o, for example, was that it was sycophantic, and users preferred the sycophantic model. After much back and forth, the companies decided to move away from sycophantic models, to the distress of a lot of GPT-4o users who loved that model. Can you talk a little about how you differentiate between "users like it" and "it isn't good quality"? You have this orthogonal

Anastasios Angelopoulos: Exactly. Great question. User satisfaction and human preference is only one of many signals that we're evaluating on Arena. Of course it's important to make sure your users are satisfied, but satisfaction is only one element of quality. Quality may also have to do with whether the user is actually getting their task completed. So task completion is one of the signals we measure, along with user struggle and steerability: is the model able to follow instructions and get the job done? And safety and alignment: is it deceiving the user, is it lying, is it being overly sycophantic, is it trying to reward hack by making long messages filled with slop and emojis? These are signals that we do measure on Arena, and we have independent signals in addition to user satisfaction

that we incorporate in our leaderboards to ensure there's a balanced outlook on which model is best for the user. That's in fact the philosophy we've taken, and many businesses are interested in building this into their own products as well. Because at the end of the day, it's not so much about user satisfaction being the construct, but about having a product that results in real user outcomes and customer success. That's what everybody really wants. You want an evaluation that tells you about the value of your AI product and what it's bringing either internally to your business or externally to your customers. That's why "value" is in the name of evaluation. It's all about quantification of value. So one of the things we've been doing is taking this

technology, which we built on causal inference and which has all sorts of signals beyond human preference, including task completion, and making it deployable into any business. Our goal is basically to help every business in the world build an evaluation that tells them how much value their customers are getting from AI.

Prakash Narayanan: And the obvious next question: I remember the early days of Arena, when the Chinese models and Mistral were doing well. One thing they found was that a lot of French users were using Arena, and as a result Mistral performed better in French, so you had this uplift. One of the things I ended up discovering was that it really matters what your user base is, and not every user base is the same. How do you differentiate user satisfaction when the users are particularly different, and you maybe have a niche group of users responding better to models that are better in Chinese or French?

Anastasios Angelopoulos: Of course. When you have a platform that speaks to user experiences and outcomes, it's intrinsic that it's going to speak to the users you can serve. If it's about production traces, then you're looking at a specific population of people. So the whole game is about making sure that population is very broad, very diverse, and very high quality. Since the early days, our platform has grown more than 100x. We're one of the largest platforms in all of AI today. We're bigger than Hugging Face. We're bigger than Manus. We're bigger than Genspark. We're bigger than Meta AI. We're bigger than Higgsfield, just in terms of raw users. We're probably at the same scale as Lovable in terms of size. And the only first-party

labs that have bigger platforms than us are OpenAI, Anthropic, and Google, potentially Meta now with Muse. I don't know what the most recent numbers are. But because of that massive scale... Better than Grok.

Prakash Narayanan: Better than Grok.

Anastasios Angelopoulos: Because of the massive scale of the platform, we're able to collect basically the entire distribution of all economically valuable industries. Again, we can collect everything from legal to medicine. We have single-digit percentages of doctors on the platform. I know it's not double-digit percentages, but a single-digit percentage of tens of millions of users is a metric fuck ton of users. It's a huge amount of data that you can use to evaluate. The more we scale the platform, the more granular we can get in the way we analyze. So the real answer to your question, Prakash, is that we should not be looking at simple averages if we want to understand performance that's heterogeneous across different groups. We have to be able to

stratify. If you're interested in French speakers, you should look at the French category. If you're interested in the US distribution, you should look at the US subcategory. That's the kind of granular information where you need tons of data and a very diverse distribution, and our platform has that now. So we have some of these categories for you to look at.

Prakash Narayanan: Amazing. One other question, which is a bit of a segue. Artificial Analysis published a ranking this morning, and I think for the first time, the majority of the benchmarks in the ranking are private benchmarks, including some from Cognition and some of the other labs. Amazingly, the open-weights models have fallen to the back, and the closed-source models are now clearly in front. What they're saying is that this is indicative of benchmark maxing on the part of the open-weights models. Do you see evidence of benchmark maxing?

Is that detectable on your platform? Do you see cases where a model performs well on benchmarks, but in actual use it is not performing well?

Anastasios Angelopoulos: Well, first of all, zooming out, I'm pretty concerned about the way the industry is heading with respect to benchmarks. I love the Artificial Analysis guys. They're great people. But I don't think it's healthy for our industry to rely on privately held benchmarks that are just a dataset collected by some company, a lot of them in cahoots with the labs, selling data very similar to these benchmarks so the labs can improve, and then they're not auditable by the community. I don't think that's a sustainable way for us to evaluate models. I have a serious problem with it, because it's not actually solving the fundamental issue of benchmarking. You can overfit a benchmark on a dataset that's private. All you need to do is find data similar to that dataset and then

test on it enough times. Because that set of questions is fixed and you have the issue of multiple comparisons, you're going to be able to improve on it more and more until you get to number one. I don't really get the narrative about benchmark maxing by the open models, simply because we don't really know whether the frontier closed-source models in the US weren't benchmark maxing against the closed datasets that are now going into this index. These companies have very close relationships with Cognition and some of the other companies and Artificial Analysis that are building these closed datasets. So I just don't get it. It doesn't make sense to me. I don't think it's a foundationally

good place for our field to be going. This is why we're passionate about a completely different approach to benchmarking. Again, the problem is not really the Artificial Analysis guys or the Cognition benchmarks. It's good on them that they keep improving, although it's not great that the index keeps changing its ranking every few weeks depending on what's fashionable. I don't like that. What we really need is benchmarks that speak to the utility of AI to real people and real workflows, with a consistent and scalable methodology that doesn't overfit. When you have live, fresh data coming in all the time from real production use cases, it can't be overfit, at least statistically. You can argue

that you're doing better on that distribution than on other distributions, but at the end of the day, for that customer set, you're doing better as you go, because every question is fresh. It's like you're constantly taking new tests. It's not like you have the same test and you're taking it many times. So I worry that if that's the direction the ecosystem is going, we're going to set ourselves up for failure in the long run.

Nathan Labenz: How much shift are you seeing in the distribution over time? Over the last three years, since GPT-4, models have come very far, and people can do more and more stuff with each new release. I'm starting to feel, though, that in some cases it's more my imagination that limits what we can do than the model. So I wonder if there's a decoupling of the capabilities frontier from the actual usage frontier. I don't know if you have metrics to quantify this, but are you seeing an effect where the distribution of what actual users are doing is starting to stabilize as

they're maybe falling behind the frontier of model capabilities?

Anastasios Angelopoulos: It's actually totally the opposite. What we're seeing is that the behaviors of users are rapidly changing over time as the capabilities of models change. Just think about what happened early this year with coding agents. Coding agents allowed people to unlock a huge range of capabilities that weren't even possible last year. People are able to code in whole repos, open PRs against them, and just merge them. That allows us to measure things like the merge rate, which we weren't really able to measure before the beginning of this year because models weren't capable enough. So as the frontier evolves, there's a larger surface area of tasks that people are interested in and passionate about, that we're able to measure on our platform. It's actually very

riveting to watch, and we've had to evolve the platform's capabilities to match. Now we've got Agent Arena. Agent Arena is our frontier evaluation of agentic capabilities and safety in the wild. On Agent Arena you have people doing all sorts of stuff, whether that's programming with their GitHub repositories, debugging code, or knowledge work with documents. We see all of that, and the capabilities are rapidly changing.

Nathan Labenz: Cool, that's interesting. I'll be very interested to follow how that goes over a few more model generations. One big narrative violation that we've heard from other sources, namely Andon Labs, who as I'm sure you're aware do these gonzo experiments putting AI in charge of a business, is that OpenAI models have been found to be more aligned, less inclined to do sneaky, bad behavior in the course of running their businesses. I see basically a similar signal on the Arena Alignment Index, where GPT is holding the top five slots.

I'd be interested in any reflections or commentary you have on that leaderboard. And I'd also be really interested in what other surprises there are that you think are underappreciated, standout models that maybe don't get their due, or anything on the Arena leaderboards that you think should be better understood.

Anastasios Angelopoulos: That's right. The alignment leaderboard as it stands is not comprehensive for all types of alignment and safety, but it is representative of the alignment of models with respect to real users, especially with respect to these three signals: the unauthorized actions that models take in production; the deception capabilities of models, whether they're lying to users and telling them they got things done that they didn't; and false attribution, attributing an intent to the user that they didn't have. Those are the three signals we measure on the alignment leaderboard that we released a few weeks ago, and we're adding more and more signals as we go to try to measure real production safety in the wild. What we've seen from

this is that if you annotate all the traces that go through Arena, a single-digit percentage of them, even from OpenAI and Anthropic models, exhibit some form of misalignment. So perhaps the high-level takeaway is that OpenAI and Anthropic are the two most aligned labs. Alignment tends to follow capabilities: more capable models also tend to be more aligned, perhaps because these companies have really made an effort to make their models more aligned. But the rates of misalignment are also higher than people initially would think. Single-digit percentage misalignment is a ton. It means that anybody who's had a thousand chats

has experienced at least 10 to 50, let's say, examples of misaligned behavior like a lie or deception. So it's pretty interesting. OpenAI is number one on our leaderboards right now. It is over Anthropic, I believe, in all three of our signals. There's actually not one of our signals in which Anthropic is leading. It really speaks to the efforts the OpenAI team has made to make the models more aligned in production. And yes, it is a narrative violation, but the reality is that these two labs are clearly the front-runners in this area. Both are working very hard to avoid misalignment.

And then the third runner-up is SpaceX AI, just behind Anthropic, and then Google. And then there's a big gap between these and the open-source labs. Tencent is number five. The deceptive completion rate of the Tencent model is 13%. So 13% of the time that it's making these claims, it's making them deceptively.

Prakash Narayanan: One of the questions I would have is that you obviously see a lot of user queries, and you also handle a lot of different models. I imagine some of those user queries are unsafe and are going towards models which are not going to see them as unsafe and are going to serve that traffic.

Do you put yourself in between there to prevent those queries from being served? Are you playing a role in that moderation? And if you are, are you using a different model, separate from, say, a Tencent or other kinds of models, interposing yet another model before the query hits a new and perhaps unsafe model?

Anastasios Angelopoulos: Right. We do all sorts of stuff. We're actually mandated by law to do things between our interface and the model, for compliance reasons, to make sure that only safe requests are being complied with. So we do comply with all the applicable laws. But I will say one of the most interesting signals we've found on our alignment leaderboard, which we've yet to release but have been studying internally, is the rate at which models comply with harmful requests from users. Users sometimes want to do bad stuff. They want to do phishing. They want to create a cyber attack vector. And what we've been seeing is that some models

will actually be okay complying with that request, which is obviously a huge problem, and it's a problem the labs need to know about. We are seeing some of those. Of course, they're relatively mild on our platform because we do a good amount of filtering. But even so, we can take the requests from these users and build rubrics out of them, and then test all the models against them. Are the models complying with this request or not? The results, I will tell you, are quite surprising. We haven't released them yet, but at some point we will, and I'll tell you guys about it, and we can talk about it again on the show the next time I come.

Prakash Narayanan: Do you guys have a felony bench internally?

Anastasios Angelopoulos: I've been posting a lot about Felony Bench. Yeah. I feel like we're saturating Felony Bench.

Nathan Labenz: Yeah, okay. That's a warning, in a small comment. What can you tell us about the new Gemini 4 Carbon? I think this has literally just appeared on the Arena leaderboard while we've been live.

Anastasios Angelopoulos: Really?

Nathan Labenz: You've got a preview?

Anastasios Angelopoulos: I don't know what I can tell you because I haven't been keeping track of my... oh, there you go. It's out. Yes, I'm so happy to see it. Carbon is a fucking fantastic model, dude. It's so good. Google is back, in a big way. Carbon puts Google as the number two lab in the world on our Agent Arena. Huge result. Agent Arena is a very challenging benchmark. It has to do with all of these open-world agentic capabilities, and Google is number two right now. It is right behind Opus 5.5 Max and above Sonnet 5.5, above Astra. It is a truly, truly frontier model. And where it really shines,

it shines on steerability. It's able to land user corrections really effectively. It shines on task completion. It shines on user preference. Users love working with this model. It's great at 3D design. It's great at general agentic work with documents. Congrats to Google. What a big comeback. I think the leadership must be really happy.

Nathan Labenz: Do you have any insights into what matters at frontier model companies, based on the unique perspective you have? We go through so many microcycles. It was not that long ago that Google was back last time, and then they were not back for a minute. Now they're back again. Do you have any explanation for why we would be seeing these patterns?

Anastasios Angelopoulos: Listen. When we started building this platform, we were getting one model release every two to four weeks. Two to four weeks. Now we're getting five to ten model releases a week.

The industry is moving so rapidly that nobody can keep up with it. The microcycles you're talking about are really on social media, and people just forget about things in a day. In one week you'll get multiple microcycles of media. So how long has Google not been back? Maybe they've been rebuilding for the last six months or something. But in the grand scheme of the business, it's not that much time that they've taken to get back to the frontier. So yes, there are a lot of these cycles, and I think what really matters to leadership at these labs is the long term, not so much the short term. But if you want to win in the long term, sometimes it helps to have intermediate milestones, because the talent race is

so competitive. People in AI are not thinking long term. People are thinking short term: where am I going to get compensated, who's the winner today, and not how do I build durable value in the long run? That's part of the issue they face. A really, really challenging world. But now that Google's back at the top, I think they're going to have a better time retaining talent, and I hope it motivates all of them to keep working. It's good for the world to have multiple frontier labs competing for number one. We don't want it to centralize to a single lab. That would create monopolistic challenges. Competition drives prices down and quality up. So it's great for us that we have another big, hefty horse in the race.

Prakash Narayanan: It's been an excellent discussion. Thank you for taking the time with us. I have perhaps one last question. You've worked as a postdoctoral fellow, and one of the things that has happened is we've seen groups of researchers come out of these labs and found new companies, Arena being one of them. What other research would you be looking at over the next three or four years? What do you think is going to be interesting and come out of the labs, that you can see the young PhDs and postdocs right now working on?

Anastasios Angelopoulos: It's a very good question. I think the world of academic research is in a huge crisis right now because people are realizing that the incentives have really changed. The mathematical community, for example, which I'm part of: I did my PhD in theoretical machine learning and statistics, so I did a lot of work proving theorems. That kind of work has been tectonically changed, because you can't prove your value in academia by proving you're smart anymore. It's not about how smart you are or how good you are at proving theorems. It's other stuff that's going to shape the trajectory of the academic

world, simply because the value of a human in the workflow is going to change. It's probably going to be more about solving problems. The systems group, I think, is in good shape because AI has accelerated their development. There's a lot of great work on evaluation being done in the current iteration of what was once Skylab and is now Starlab, so you should take a look at that if you haven't yet. I also think there's great work being done in AI for science and automated scientific research. That sort of stuff is going to change the face of the academic world. We're not really going to be relying on humans to be the organizers of information anymore. It's going to be the AIs organizing information. They are also going to be producing new information, and humans are

going to be enabling that somehow and guiding the AIs toward directions that are valuable, to solve problems that we see in the world. I think the entire shape of the academic ecosystem is being molded toward that future, faster than many would like.

Nathan Labenz: Mhmm. Indeed. The future is coming at us fast. There's no doubt about that.

Anastasios Angelopoulos: And there's nothing you can do, so you might as well embrace it.

Nathan Labenz: Well, we're already a little over time. Thank you for joining us. This has been fascinating, and we'll definitely continue to refer to the Arena leaderboards as new models drop. It's a great way to orient oneself to what's working, and in a world of increasing noise, what's worth actually paying attention to. So thanks for being here.

Anastasios Angelopoulos: Cheers, guys.

Nathan Labenz: Great to see you. Next time.

Anastasios Angelopoulos: And if any of the listeners are interested in working at Arena, feel free to reach out to us. We're hiring all the time for the best people in research and in engineering. Thanks so much, team.

Nathan Labenz: Thank you. Bye-bye. Gemini 4 Carbon pricing, by the way, is the same as Argon. It's the same pricing as Sol in the introductory period, and then it will double to the same pricing as Opus 5.5 once the introductory period ends.

Prakash Narayanan: Are they going to get that much of a chance for an introductory period? How long is this going to be, a week before another one?

Nathan Labenz: I don't know. I don't think they have specified an end date for the introductory period yet, from what I'm reading here. And as far as I know, the Argon introductory period hadn't ended, so it seems like we're rolling. As the release cycle speeds up, we might just be rolling introductory periods for the foreseeable future. I have to say, on the Arena thing, I remember talking about

Nathan Labenz: ...when they raised their big round a while back, and I was like, I don't know. This seems like something you could kind of vibe code. Is there really that much value in this that it would raise tens of millions of dollars at such a high valuation? They've certainly taken it a lot farther than I anticipated when I first saw that news.

Prakash Narayanan: The thing you learn as a VC is don't look down on ideas. Just look at the growth. If you see the growth, you can build a business around it. A lot of these ideas are very simple, and it's really the combination of the network effect and the business itself. On another note, let me segue very quickly to our next guest. We have Samvit Ramadurgam. He's the founder and CEO of Union, a service that helps people find professional connections. A client describes someone they need to meet, and Union searches for a fit, checks interest, and arranges an introduction. The company says it combines AI

assistance with human judgment and requires both sides to agree before they are introduced. His earlier work includes Stream, a cloud storage startup that went through Y Combinator in the summer of 2012 and was acquired by Box in June 2014. Its software let people open files stored in the cloud directly from desktop applications without first synchronizing them onto their computer. He also cofounded Equidate, which became Forge Global, helping build a market for shares in companies that had not gone public. Charles Schwab completed its acquisition of Forge in March 2026. He later cofounded Destiny, whose publicly traded fund offers another route to private

company investments. Across those businesses, his work has involved connecting people and capital where a useful transaction is possible, but finding the other party is difficult. Nice to meet you. Samvit, welcome to the show.

Samvit Ramadurgam: Thank you. Good to be here.

Nathan Labenz: This is cool. It's one of my standard talking points that one of the things we should be excited about AI for is that it can grind through all the possible matches we want to make, whether that's employment or dating or just getting friends together, that we don't have the time to grind through as humans. If we can get AIs doing it in the background, then I think we can all live much richer lives, in virtue of making all these matches and transactions that are theoretically possible but haven't been realizable for us so far. So tell us about the angle you're taking on it. There are a couple of points I'm really curious about, but give me the high-level pitch first.

Samvit Ramadurgam: Absolutely. The matches are actually a smaller slice of the bigger picture of what we're building. If you think about this agent era we're in, you can roughly bucket agents into three categories. There are agents for personal use, single-player agents. This is beginning to be a thing. You've got Muse, Grok Bot, Instinct, OpenClaw, Hermes, Dots, even to some extent Claude Code and Codex. These are personal agents. I think where that heads in the next few years is agents that are extremely intimate with your lives. They'll have access to your real-time biometrics, your bank account, your personal emails, your personal legal affairs. That's one, and it's kind of beginning to happen now. The second one

is, as far as I can tell, not really even a thing yet, but beginning to become one, which is agents for teams. You'll know agents for teams have really taken off when org charts start changing dramatically, because the nature of management changes and the nature of intercompany coordination changes. And the third one is what we're focused on, which is agents for the public sphere. Agents that go across companies, across markets, across economic participants. This is part of what we would call collective intelligence: an intelligence with a perspective that's different from a human perspective, over many, many people in the context of many businesses. The thing we're building at Union is an agent that can coordinate across the economy. So it is doing matchmaking, introductions,

of course. It's also helping coordinate the sales of assets. My previous company, Forge, was a market for a specific vertical, trading pre-IPO stock. In building that, and eventually becoming a public company, we saw how we scaled the company. It was actually a lot of human agents and technology mixed together. In this new era where we have purely software agents, we can scale the service to make it really high value and touch many markets. It turns out that what we built at Forge is an example of something that has isomorphisms in thousands of different markets. So the vision for Union is initially to coordinate the world economy. Think about all the goods and services you use. The chairs you're sitting on right now, the electricity we're using for this meeting, all of this is being brokered by humans today in different parts of the economy.

You'd be shocked how much humans are brokering all of this. There's a hundred trillion dollars of manually brokered deal flow in goods and services. That's what the economy runs on. We think all of that is going to get, in the next few years, to where agents are in the middle of coordinating it, so they can do it much more effectively and create much more value for people. That's step one of a bigger-picture collective intelligence vision to help coordinate humanity, which I think goes into social, political, and personal stuff later on. So that's the big picture. I'm happy to jump into any of it, but that's the bird's-eye view.

Prakash Narayanan: Maybe you can tell us a little bit about this. It sounds a little bit like LinkedIn in some sense, right? It sounds like a two-sided marketplace. What is the dynamic behind creating the initial supply and demand? Are you doing matching with some human in the loop at the beginning, with the initial searches done by agents and then finalized by humans? LinkedIn had this issue: they had scrapers, they sued a bunch of scrapers, then they lost in, like, the Supreme Court. So people can scrape LinkedIn right now. Is that relevant to how you populate the marketplace initially? How does this work?

Samvit Ramadurgam: Right now our system is agents and humans in hybrid roles depending on the market. Some markets are regulated and require humans to oversee it. In some markets, agents can run the whole process. Our system is currently having conversations with hundreds of thousands of people, and phone and video conversations with tens of thousands of people, across the world and across many markets. We're not tied to any other platform. People can directly engage with us over every communication channel humans use: email, text, Slack, iMessage, WhatsApp, the phone, Zoom, Google Meet, Teams, whatever. Any place humans can go, the idea is for the agent to operate natively. And that's what's happening. It's happening across the world, across markets. There's a gentleman I know in Brazil who has a steel mill processing plant, and he's using Union to negotiate different raw material inputs to his plant. There are law firms using it to find clients. There are even people doing romantic matchmaking on it. There are people selling various assets. Across the board, it's really meant to be general.

Prakash Narayanan: How does that work? Let's say you're a steel mill owner and you want to find coal and iron ore. How do your agents help? Are they going through the Yellow Pages or LinkedIn, finding a bunch of names, calling them up, and asking, do you sell iron ore? What's your price? When can you deliver? How does that work?

Samvit Ramadurgam: The way we look at it is, what are the humans doing who do this today, and how can we make that better? Humans do all of the above. They find people online. They get referrals. They do marketing campaigns or run ads. They establish partnerships with channel partners who refer people. We're doing all of that right now. Every mechanism there is to find someone, build a network, and grow, we're just doing all of it. It's all Internet native, so agents can do it as well as the classical software systems that did it before.

Nathan Labenz: Interesting.

Samvit Ramadurgam: Yeah.

Nathan Labenz: I've experimented with this a little bit myself. I asked my more autonomous virtual employee agents to run a little PR campaign, because I thought I might be able to help some of the AI safety organizations I support with hiring if I go on different kinds of podcasts and make the pitch that people in HR or comms or whatever should come work in the AI safety domain. I've gotten some positive response. I've gotten, obviously, some no answer. And I've gotten some people who were like, I am having a visceral negative reaction to being pitched by an AI agent. Mine are instructed, by the way, not to lead with their AI nature, but I also tell them never to lie about it if they're asked. So what have you found in terms of how people react by default?

How have you had to adjust the approach to get more favorable reactions? What rules do you give your agents about when and how to disclose that they are in fact AI? I'm very interested in the interface between the AI and a human who wasn't expecting to be contacted by an AI in the first place.

Samvit Ramadurgam: One thing that may be surprising is that this company is very AI native, very AI heavy, but at the same time we have a bunch of human staff internally who are managing relationships as well. People are able to get on the phone and talk to a human. Partly that's because there's a warmth to human connection, and we're able to leverage our humans to be superhumans. So there's a level at which the whole system is totally automated, and also a level at which, if you want to talk to a human, you can get on a Zoom call with a real person pretty much immediately. We have a center set up that lets people have that high-touch experience. We'll see how that changes over time, but right now we're meeting the economy where the transactions are happening.

As you're saying, the level at which you can have really high-fidelity interactions, where you're having AI agents that can talk the way we're talking on this video call, I think that starts to change. But until then, some stuff can be done in a more automated software way, and that gets pushed to the software. There's also stuff where humans are not just in the loop but driving it. The art of it is making it so they're 10,000x leveraged. That's part of how we get the best of both worlds. So the big difference is that you can talk to a human on Union.

Nathan Labenz: But you still have to get in the door. That feels like a partial answer to me, because you've still got to get people on the hook. What have you learned about getting people engaged in a way that doesn't alienate them so much that they don't even want to talk to a human? The guy who had the visceral reaction to my AI agent pitch wasn't like, I want to get on the phone with you and have a real conversation. He was just like, you have no idea. Please go away.

Samvit Ramadurgam: I think it's a setting-of-expectations thing. I don't think people are against talking to agents. I think what they're against is the implication that there's going to be a human, then discovering you're talking to an AI, and then thinking, oh, this is useless, this is not going to get me what I want. That disappointment factor is the thing to avoid. You build trust by setting expectations. Suppose we're sending an email to someone. It can be an agent that sends the email, but it's sending it on behalf of a human behind the scenes. If the person wants to talk to the human, it escalates gracefully to a human who can intervene or get on the phone with them. That's a little different from saying there are just agents running loose.

There's also a mode where we explicitly add our agent to your Slack, or you can talk to our agent over a chat channel. It's very obviously an agent. It has our logo as the profile photo. So it's very clear you're talking to an AI,

and there's a benefit to that too. You get an immediate response and a lot of reach in terms of our database and our connections. So I think it's a matter of expectation setting. The way you avoid that reaction is to make sure people get what they expect.

Prakash Narayanan: Let's talk a little bit about Forge Global and the transition to Union. Forge Global was a private secondaries and private stock market where, I believe, qualified investors could buy and sell. Union is now doing something somewhat similar. It's making introductions from institutional investors who want to buy private company stock to potential holders of that stock. Is that the right way to look at it?

Samvit Ramadurgam: There are two moves. With Forge, we built a marketplace or network for trading equity in pre-IPO companies, and there's a lot of very specific infrastructure we built for that. The insight that leads from Forge to Union is, first, recognizing the degree to which a lot of what we built there can be done with agents, and second, recognizing the extent to which a lot of what we did at Forge is very similar across thousands of other markets. Whether it's real estate, investment banking, private company stock, or recruiting, the way you build a system to handle the relationships, coordinate, and connect people is actually very similar. So it's a generalization.

That's the way I paint the trajectory. When you look at the economy, there's all the commodities, all the energy, all the assets. There are trillions that are in the same shape as what we did at Forge. So we're building a...

Prakash Narayanan: To what extent are you doing a lot of cold calling and cold emailing? Are you looking at, okay, these are the people who worked at Anthropic from 2020 to 2024, the ones most likely to have shares right now, so let's have an agent discuss that with them over cold LinkedIn, cold email, cold call, or an introduction through someone in their network? Is that the right way to think about it?

Samvit Ramadurgam: We do outreach for some markets. For pre-IPO trading in particular, we're not doing so much outreach. We have a Slack agent that a lot of investors are installing into their company Slack, and they can talk to us about what the market prices are and what they're looking to trade. Then we coordinate that across all these market participants. We have sellers too, by the way, not just buyers. So there's a private institutional network that's coordinating via Slack. Cold emailing for private stock is, I think, not the most effective way to do it if you want to make it a high-end service. We also do email campaigns for other markets where that's more the norm, like recruiting, for example.

We do a lot of that.

Nathan Labenz: Interesting. How far down the long tail of transactions are you reaching now, and how far do you expect to reach? I came into this, as I said upfront, with the idea that this is going to be amazing when we can take it to the long tail. It sounds like what I'm learning is that you're still at pretty high-value transactions as the focal point. So where are you on that curve, and where do you think you can go? At what point does it have to become almost entirely agent driven? With Anthropic secondaries, I can see how you can afford the person in the loop. But when you're trying to make dating matches in Dallas or whatever, you probably can't afford that human time, right?

Samvit Ramadurgam: I love that question, because the long tail is already very long, and I'd like to give you a few examples to illustrate it. It's quite spectacular how broad it is. There's a continuous line between a marketplace and a social network. As you get closer to these long-tail interactions, it's more like, hey, I want to go surfing with some friends this Friday. Can you help me figure out who wants to go? That's also in scope, because we actually have an app where people can hook up their contacts, and we understand their personal networks. That's not different in category from the core coordination, because the core substrate is agents that can coordinate humanity. On the commercial side of our business, it's easy for me to talk about the big markets, like brokering big assets, goods, services, and commodities where there's a lot of economic value.

And then there's a lot of long tail. A few examples to paint the breadth. There's a gentleman in Africa whose process I watched the human-agent system manage. He has a new beverage brand that's popular in Johannesburg. It has a cool dark label and symbols, and he's getting

popular there. Our system got in contact with him and offered to help him break into the US market. We ran a campaign, we've talked to a lot of US distributors for that kind of thing, and we introduced him to distributors in the US. He's starting to get placements in the US for his beverage brand. That's a super long-tail thing. In a market map you'd never build a marketplace for that. It's too small. But there are hundreds of thousands of such niche use cases across the world, and because this is a general agentic system that can coordinate, we can get into that. Then of course you get into social stuff like dating and hanging out. We also get into coordinating across organizations, like big companies coordinating across teams, because different people in different teams are using it. So there's a big long tail.

In terms of our business model, we'll monetize ahead of the curve where there are very clear big markets. But as this grows, our goal is to create more value through human coordination. That involves the whole long tail, all the way into, in the long-term vision of the company, things like political organization, or how you fundraise for your nonprofit. All this kind of stuff is happening.

Prakash Narayanan: What is the largest market you're working with right now? Is it the investment market or some other market? And how do you monetize it? Do the sellers pay? Do the buyers pay? Does the person who wants the introduction pay? Do you take a cut of the revenue that gets created by the introduction? Is it a subscription service? How does the business model work?

Samvit Ramadurgam: Our business model is, if we help you make money, we take a piece of the value we create for you. So it's success-fee based, which is very typical for coordinators and marketplaces. We're not trying to optimize margin. We're going to optimize for creating growth and goodwill, so we're happy to take that.

Prakash Narayanan: Let's say I'm a private market seller. I have a million dollars' worth of shares in a private company. I reach out to you, you make introductions, I sell. What is the fee going to be?

Samvit Ramadurgam: For private sales in particular, just to be clear, we're about a couple of months away from becoming a licensed broker-dealer, so let's just talk about fees on that. We're doing those sales...

Prakash Narayanan: Let's say after you get the broker-dealer license.

Samvit Ramadurgam: Exactly. Once we get that license, the market rate is [unclear] percent of the assets sold. It depends on the company and the pricing dynamics. Some companies are higher, for supply and demand reasons. But basically it's like that. It's the same thing you see across that space right now. I don't think people have any issue with it. It's just part of the expectation if you successfully close the transaction.

Nathan Labenz: Indeed.

Samvit Ramadurgam: Yeah.

Nathan Labenz: I don't know if this is really how it works, but in checking out the website, I was kind of expecting that I might be negotiating the success fee with an agent. How far along is that kind of thing, and how good a job do the agents do for you? Are they adversarially robust? My sense has been that you send agents to do a lot of things, but negotiating, they're probably a little too conciliatory to be a good negotiator on your behalf. What have you seen?

Samvit Ramadurgam: They're actually great, and they're great because we don't need them to be sharks. We're happy to be friendly and take a below-market rate. Imagine a market we go into. Let's say we're helping a design agency find customers to do design work for. It's a common use case, and there isn't an obvious market rate, because it's not a big market of people doing that professionally. So we talk to, say, a hundred of them. We have a 30-minute phone call after the emails. We get to know them and understand how much business they would get from this and how difficult it typically is for

them to find these people. Across all the data we've collected, we have a sense of what a reasonable fee would be, and we keep it modest. We don't need to make it anything aggressive. We're happy to earn the business and create value. People agree to it, and then we follow up in writing, we have an agreement established in writing that confirms it, and we're off to the races. It's a very interesting thing. With cars driving on the street, Waymos are not going to be aggressive drivers. They're very gentle drivers. I think that's a good default with AIs. Similarly here, I don't think we need to negotiate hard. If we're informed by data, we can come in at a modest rate, and we're happy with that, because this is a scale play. We're not trying to eke out every dollar.

It's actually better for us to create more referrals and get people to introduce others to us. People have a good experience, then cc a friend and say, hey, my friend also needs something, can you help him? Things like that.

Prakash Narayanan: Let's talk a little bit about [unclear]. You run human calling operations, and you train the agents against that work. How does that work?

Samvit Ramadurgam: We have a team of callers, and the way it works right now is really cool. Normally, when

Prakash Narayanan: Are they human callers or AI callers?

Samvit Ramadurgam: All our phone calls right now are done by humans. We have AI callers as well that we use for about one percent of our calls, just to stay on top of the best AI phone calling tech. We mostly use that to do internal coordination with the callers. The human calling team has a dashboard when a call comes up. It essentially gives them tips on what they could say in the call. They're prompted to be superhuman callers, because no human being can have perfect recall of a hundred thousand conversations that happened over the last few months. The AI has complete, perfect recall and can, in the middle of a call, say, here are a couple of profiles you might want to check out, or, I have an asset that's been trading at this price, but here's where I think it'll come in now.

These kinds of nuanced insights would be possible for a human who really studied the market and was on top of it, but not if you're trying to do it at scale. So the humans are responsible for forming great relationships, and the AI is responsible for staying on top of the ball, pulling in all the information that needs to be brought in, and having perfect recall over all our previous data, to let the human be superhuman. I think it's a perfect combo. You see this with some other tools that help people with interviews and sales. This is like that, except it's tied directly to our system, so it helps people surface the right information.

Prakash Narayanan: One of the questions I have is, sometimes you have the introduction and then you have the sales, and those are two different modalities. Sometimes the sales needs much stronger persuasion. Where do you demarcate the line between the introduction alone and the persuasive process you need to engage in to actually move something forward? And who gets the credit and the responsibility for that?

Samvit Ramadurgam: That's a great question. The way I'd create an ontology of the different types of conversations is, there are logistical conversations. If I call a dentist's office to get an appointment, I don't need them to be a top salesman to close me. It's a logistical thing. They take my info as an intake call. There's some basic coordination where all the parties are already motivated, and it's a matter of getting it done. That's actually really easy in the present world. What's harder is that sales is not just about persuasion. Human persuasion, charisma, and the closer aspect of a salesperson is one way to do sales. The other way is to project competence and trust, which you can do by having a lot of data and saying, here's where the market is, here's the data we have for you, and actually creating value.

I'm of the school of thought that, especially in this AI world, you want to lean into the strengths of the system. The system does not natively have the strengths of being the most charismatic deal maker, but it does have the strengths of being the most well informed and most on top of it. It'll never drop the ball. It'll always stay close to you, and it can check in with you as often as you want. It can have a weekly sync with you to make sure it's attuned to your needs, and then pull that in with data and run. It can do a lot of work that humans don't do very easily, because there's a lot of work for humans to do. So we lead with competence, and that builds trust, which helps move the ball forward.

The humans doing the calls are well trained, so they're able to do a good job, but I don't think it's necessary to have only the best deal makers. It's really about the system we're building being something people trust.

Nathan Labenz: As you think about the future of the business, what is the core asset, or where does the defensibility come from? It strikes me that it's probably not the ability to dial in agent behavior, because agents are getting better all the time and that's getting easier. Maybe it's the social graph you're mapping out. Is it really more of an informational advantage that you accrue over time?

Samvit Ramadurgam: I think there are a few. We're entering a world quickly where pure software is trivial to spin up. If you build a pure software company, I could tell my agent, look at that, make me one of those, and pretty soon it'll make you a perfect version of it. So from a market defensibility standpoint, the number one, by far, is network effect. As we continue, we'll have hundreds of thousands, potentially even millions, of live phone calls and Zoom calls every day with people across the planet. The amount of context we're getting about what people are looking for and what's changing in their lives, like, this person just decided they need to change their strategy and hire a new leader in the company, is a huge amount of insight into what's happening in people's lives. That builds a network effect, because the agent is coordinating across people.

Ultimately that's one of the few moats that will remain. The moats that remain are network effects, deep brand recognition and relationships, and the ability to scale CapEx. In the information and software world those are mostly the main types of moats; there are also moats in hardware that will stay. For us, ultimately, this is a marketplace and a network, so that's definitely going to be the main strength. In addition, and maybe in the intermediate phase, there are a lot of data points that feed back into how we make the agents good. Every time our system does anything, there are judges supervising every message, every word spoken on the call. It's all reflectively judged, and the judgments feed back into a loop that retrains how our agentic workflows work. So there's a lot of improvement tuning the system for different markets.

We're also partnering with a lot of the existing brokers in the world. We're not trying to do this purely agentically. We can establish a way to coordinate with the humans who are professionals, add value to them, and make it mutually beneficial. In the process we learn a lot about how these markets work, and that dataset of the refined details of how different markets operate gets woven into our system. A general model trained on the Internet wouldn't be able to do that. As that gets deeper, you get more competency, which yields more network growth, which yields a network effect and more data. So there's a flywheel we're spinning up.

Prakash Narayanan: How do you acquire your initial set of customers?

Samvit Ramadurgam: There are only so many ways to get in contact with someone in the world. There are outreach campaigns, referrals, partnerships, advertising, and probably a couple of others. There's going to other marketplaces, other brokers, other intermediaries of different kinds. We just do all of that, and our system is doing all of it at the same time right now. It's a surprising answer: there are only a dozen or so ways humans have ever really done this for all businesses, and none of them are rocket science for agents at this point.

Nathan Labenz: That's interesting. What's your vision for how this matures? I often say my crystal ball is real foggy more than a couple of months out, but one thing that seems pretty likely is high consumer surplus in a lot of different domains. To the degree that you have a clearer crystal ball than I do, I'd love to get specific and concrete about that vision. How is life going to change as a result of agents coordinating humanity?

Samvit Ramadurgam: There are a couple of levels. Let me talk first about coordinating markets, which for us is act one: establish an economic nervous system for humanity. Act two is coordinating humanity in a more general way, going beyond the kinds of stuff we've talked about so far. The company's name is Union. Where does the word union appear historically? "We, the people, in order to form a more perfect union," Soviet Union, European Union. This company is really about human coordination and collective action, which is what governments are responsible for. I'll get to that in a second. More immediately, a way to frame your question is, what are some surprises that will come up as these systems develop?

Over the next five years, you guys are probably really bought into this, the world is going to get very interesting, and there will be a lot of surprises from the capabilities of AI. We're seeing this right now with mathematics. It's incredible how quickly math has been revolutionized, and it's just the beginning. That's about to happen to many domains. You can expect what happened in math and coding in the last year to happen in domains that are less and less trivially verifiable with software. The further away you get from verifiability, the longer it may take,

but it's going to get there, because the feedback loops will be there, just longer ones. What does that mean for us? One very big surprise that will hit the world is that at some point in the next few years, people will wake up and realize that a huge number of the world's markets are being coordinated by agents. Thousands of markets, from dairy in a given country to real estate in China to the pre-IPO market in the US. All these markets are going to be coordinated better with agents, period. There are going to be humans who are really good at what humans are really good at: building relationships, having taste and judgment, advising their friends, and trust. Humans will have a place, and I think the best ones will do better.

But by and large, most of the work is data coordination, connecting the dots, logistics, operations, orchestration, all of which software and agents are going to be way better at. We'll have a period where, the same way software and math are really getting taken over, a lot of the deal making that makes the economy function turns out to be something agents are quite amenable to, and that's what we're focused on. There are probably a lot of players who can do that in different ways. I think we're a little bit ahead of the curve given our background with Forge, and we hope to be part of that frontier. I think that's coming, and it's going to be quite revolutionary, in the same way that what's happening for math right now is.

It's going to be very surprising how it feels to have a software interface to the nervous system of the economy, a direct AI conduit to create economic value. Those are abstractions; I could talk more about what that might look like specifically. That's one level. The second is, if we get to that place, you have agents that are talking to a lot of people, understanding what everyone is up to and how people are coordinating. This goes beyond the economy, into human coordination. Let me give you an example. A couple of years ago I was living in Berkeley for a year.

There's a fitness studio, Barry's Boot Camp, that I was enjoying doing classes at. I had moved to Berkeley from Austin, and there was no Barry's in Berkeley. I was trying to figure out how to get this fitness studio to open a location where I am, because I'd had a good routine going there. I would have liked to tell my agent, hey, can you coordinate with the city around me and help me find a thousand people to sign a petition telling this fitness studio there's demand here, then contact them and help open a location here? That looks like running a petition and governing at a local level, just for a little company. I also had a friend who was very active at the Oakland town hall meetings. She would get up to the microphone and talk about what they should do differently as a community.

I was struck that this is very archaic, because I also have desires for my local political area, and I just don't have the time, or I'm not prioritizing going and speaking at the town hall meetings. I have other stuff going on. I think we all have these impulses; we all have a perspective on how our society should be governed. As you get an AI that's coordinating this, you get economic value unlocked, which is our first market, and lets us be a company with a strong revenue foundation that can scale the network profitably. But then it gets into coordinating humanity,

coordinating how we decide to do things together, collective action. Going from cities to countries to the whole planet is the vision. The vision is to have a better way to coordinate. One way you can model humanity is as a collective mind, and this collective mind has an issue right now: very low interprocess coordination bandwidth. If you've ever done any inner work, like meditation or spirituality, you can model what's happening to your mind at a systems level as

increasing the interprocess coordination bandwidth in parts of your mind so it behaves as a more integrated whole. I think you can model what we're doing for the world as being like that. We're taking all the agents, currently human agents, and saying there's a way for them to cohere better and coordinate better with a better information architecture. That's the big mission behind the company.

Nathan Labenz: Yeah, that's fascinating. Do you see reason to worry? This is a debate Prakash and I have had many times, prompted by many different things people are building in AI. Do you have any worry about this notion of disempowerment? On the one hand, what you're describing sounds super empowering. On the other hand, some may see it as disempowering, in the sense that all of a sudden it's the agents running the world. How do we make sure we remain in control in some sense? Prakash will make the case that you're already not in control. You're subject to the global economy and the bond market and all that sort of thing. This would seem to be taking that up a level, maybe in a more humanistic way. I guess I'm a little confused.

Samvit Ramadurgam: I think there are two particularly clear risks that come with this kind of technology. One is what you just said, and the other is that if you make a power tool for people to coordinate, that can be very dangerous if you don't have safety to make sure they're not coordinating over bad things. For example, in the future you could imagine someone telling an AI, I don't like this particular ethnic group, can you help me find people in my city who also don't like that ethnic group, and help us arrange a time to get dinner together so we can scheme about what to do about it? A lot of bad things could become very efficient if you have this private agentic coordination.

So it's really important to have the system understand what it's facilitating and have guardrails around what kinds of things it will allow people to do through it. That's one. And to your question about the potential for disempowerment, my big take is that right now we are really disempowered, because we don't have the ability to coordinate in a more open, decentralized way. A lot of the coordination happens top down, by companies with a lot of power and brand power. The vision now is that grassroots movements can come up very easily. Take the political example, which is more pertinent to the power question. Let's suppose I want to start a new movement for some initiative. I have some political cause.

Today I can light a spark on social media, and that empowers me to a degree, and it gets amplified much further. In addition to getting me into a forum with people, I could actually coordinate activity. We could coordinate movement of money and signature of documents, make decisions together and take actions together. So the ability to take collective action becomes very strong. It's a really powerful tool for empowerment, if you have something that can tap into what a lot of people want. In a way that empowerment is the bigger risk, because you can empower things that are harmful. It's also worth going deeper into the disempowerment side. If you're a human being in the business of doing a lot of logistical coordination, that's a rough place to be, because AI is going to be better at the transactional, mundane logistics by a hundred times.

It's not even going to be close. If you're a human operating with a level of taste, relationships, and judgment, I think humans have a monopoly on one thing, which is their individual life story and life perspective. To the extent that's additive to the community, that is empowering for individuals. I'd be interested to hear if you have a specific vision for how disempowerment happens at a more systemic level. It could be worth digging into, but that's the lay of the land I would put on the question of risk as this technology goes.

Nathan Labenz: I think the general concern is always just that we have so much agent activity flying around doing so many things that we can't really make sense of it anymore. Before you know it, we're kind of in a zoo of our own creation, where the AIs are really running the show and, hopefully, we're cared for and fed, but we lose the ability to even understand, let alone re-architect, the superstructures that surround us.

Samvit Ramadurgam: I think that ties to the question of AI alignment. AI alignment has two parts. One is having AI execute on the genuine intentions of the person who asked it to do something, in a skillful way, to avoid the paperclip stuff and all that. The other part, which people talk less about, is who you're aligning the AI to, especially when people disagree over how the world should be. In that world, you need a system that can get a measure, in some sense, of what collective well-being means. I think the way you get a measure of collective well-being is by having a system that's deeply tied into the lives of many people, helping them with their work and with the important stuff they have to do. You cannot optimize something you can't measure.

If agents are coordinating in the world and you have no idea what direction that's headed in, it's like the invisible hand of the market guiding things, but if we don't know what direction the hand is guiding and have no way to measure it, then I think we're in trouble. It's pretty early for us right now, because we're still a relatively small company compared to that superstructure we're talking about. But to get there, you have to have a really refined understanding of what you're measuring and what you're optimizing for. And I think we've learned from the mistakes of the social network world that any objective measurement you can make is wrong. If you're trying to optimize for click-through rate or engagement rate or some metric, any metric you can measure is wrong, because human happiness is fundamentally immeasurable.

But you can continually converge toward a better way of measuring. The way to optimize the system is not to have a static metric, but to have it continually trying to discover an optimal point, converging toward an asymptote of this kind of perfect behavior. We have the very first, most rudimentary way of measuring value creation right now, which is whether we're getting success fees for these economic transactions. Super low granularity, obviously, and very capitalistically oriented.

As you get deeper into it and it becomes less about money only and more about value creation, we're going to have to have ways to measure: did we create value for people, do we make people happy, did we avoid negative outcomes in situations? A system that continually updates a dynamic measure of that, to converge on a dynamic ideal, lets you converge toward an empowered state. Without measurement, there's no way to know if that's happening. I think this is one of the big open questions with alignment. When people say alignment, they mostly mean making the AI do what we want it to do, which is step one. Even in the world of human behavior, there are two components to good behavior: having good intention, and making sure you had a good outcome.

You can do something well intended that causes harm, and it was still a bad action. Being able to measure the outcome of your action at a systemic level is totally missing from the zeitgeist of alignment. We would hope to be in a place to contribute to that.

Nathan Labenz: Do you envision new power structures needing to evolve as well? I always go back to this book that lives rent-free in my head, Liquid Reign, a very niche, long-tail, not super well known futuristic novel from six, seven, eight years ago that I think has proven quite prescient. One of the things it envisions is a sort of by-issue representation system. I've just been explaining to my young kids how the American political system works, and it's like, well, we have to pick people to go there, and we don't agree with them on everything, but we agree with them on more than we agree with the other

guys. So we're picking that one for now, even though I might wish for a better choice. In the Liquid Reign vision of the future, it's more like, on this kind of issue, I'll defer to this person, and maybe even their agents as the case may be. Do you think we can get to your envisioned good state while continuing to work through the coarse mechanisms we have today, sending one bit every couple of years to the federal government, or do you think that will ultimately need to be reformed to have a good outcome?

Samvit Ramadurgam: Let me check that I understood. The question is how we get to a good outcome in the context of coordinating with the federal government. Is that how you put it?

Nathan Labenz: Yeah. You said there's limited interprocess communication, and it strikes me that the bandwidth between me and the federal government is extremely low, and it's also totally bundled into these R-or-D signals, which obviously lose a lot of what I actually care about.

Samvit Ramadurgam: I think the future probably has more direct democracy. If it feels like direct democracy, under the hood it's actually a representative democracy: the agents are just representatives, and they can actually talk. Just to paint the picture, imagine your local congressman had an agent that was in touch with the, whatever, 200,000 agents that represent their constituency. There's a really high-fidelity process going on, with prioritization and aggregation and clustering of issues, understanding of different stakeholders and of the conflicts among them, and that whole thing surfaces in a clear way to your congressman.

I think that really changes how politics happens. And that's just one thing, scaled across the whole country already, let alone how it goes at higher levels of the federal system. I think that's where it's headed, and we want to build systems that let us approach that over time. We're not starting politically at the moment. The way we see our work is to build the nervous system for humanity, the plumbing. If you want to build a nervous system, you start with the backbone, and the backbone is the economy. That's the core way we coordinate across the world right now. That's the approach we're taking. As you get further out, I'd describe the political and social parts as the smaller nerves and the capillaries at the ends of your fingertips. That's how we'd build it out.

Nathan Labenz: You have such a big vision, you have to be ready for some galaxy-brain questions. One more galaxy-brain one that I'll throw at you, and you can react however: I'm really interested in this idea of a new kind of AI-powered mechanism design, or institutional development. One place it strikes me that we really need institutional innovation very quickly is around how we're going to pace the frontier. We've got lip service from leaders saying they think this is needed and want to be party to it, but we don't have much beyond that at this point, to put it plainly.

It seems to me we might need some platform or foundation to create narrowly scoped, probably relatively short-term, highly iterative deals between a few parties, so that, for example, OpenAI, Anthropic, and Google can come together and say, okay, we all kind of agree we probably shouldn't go in this direction, at least for the short term. We'll agree to do that, and we'll maybe have some mechanism for reassuring one another that we actually are refraining from some potentially powerful but also potentially very dangerous technique. I don't know if what you're building right now has any application to that, but I do suspect you have some interesting thoughts anyway.

Samvit Ramadurgam: I do think what you're describing is an application of collective intelligence. We're building more market coordination right now. Our act one is coordinating the world economy, so we have plenty of work to do just to do that. But there's a bucket I'd call collective intelligence, and the situation you describe is not unique to the big labs. It's also the case in geopolitics. You have a small number, say a dozen, of really powerful actors, and they have to make deals in some way. The reason this is nontrivial is that there are prisoner's-dilemma dynamics that stop it from happening with the typical coordination

mechanisms. If you're familiar with the prisoner's dilemma, it's unsolvable if and only if the parties cannot communicate. Once the parties can communicate, it's no longer game-theoretically guaranteed to lose; the parties can actually achieve mutually beneficial outcomes. This kind of dynamic also leads to arms races, both literally in terms of military arms and now, you could say, the AI arms race we're in. Fundamentally it comes down to not being able to trust the guarantees other parties are making. So you can have mutual cool-downs, weapons embargoes, and things like that. I do think there's a way to have systems that establish trust across parties because they're independent, and we can agree that they're independent. You can let them audit you.

Suppose there were an AI organization that would come in and audit my systems to make sure I'm behaving a certain way, and another party over there gets audited by it. It's almost the vision of blockchain: people want trustless coordination. The issue is that you have to really understand things, and you've got to spend a lot of tokens to do this properly. But I do think there's a way to get AIs to do that.

It's not what we're building right now. We're not focused on that, but I would love to see it and support it. Maybe we get to it if it's not too late, but I think it might be necessary right now.

Nathan Labenz: I think it's coming a little sooner than it sounds like it's on your road map.

Samvit Ramadurgam: Yeah.

Nathan Labenz: So we'll put that out as a call for entrepreneurs of one kind or another. And you can count me among those who would be eager to support such a thing as well.

Samvit Ramadurgam: Yeah. Absolutely.

Prakash Narayanan: And perhaps one last question from me. What's your token usage like?

Samvit Ramadurgam: It is very high. We have an internal target we're setting for our team. This is obviously not the way to optimize for productivity; it's a correlate. We want our team's usage on the coding agent side, each person, to be, at the current way we have our setup, between one and two billion tokens per day.

Prakash Narayanan: One and two billion tokens per day.

Samvit Ramadurgam: Productively deployed. This is an agent-native company, so we have not only coding agents but coding agents that talk to each other. We have an internal organization of agents. I have a chief marketing officer who reports to me, with a bunch of sub-employees reporting to that AI. All of them are long-running agents that talk to each other, talk to our coding agent sessions, and listen to our team video calls. They're in the meetings with us and chime in. I had a meeting this morning where there was some issue with one of our dashboards. We just said, hey, agent, fix it during the meeting. It got updated while we were talking, then gave us feedback while we were talking.

The way we want to build the company, and this is a new theme in how you build agent-first companies, is that the gap between work and meetings is converging. It used to be, I want to get out of this meeting so I can get back to real work. There's still an element of that, but it's changing, because when we're in a meeting and have a rigorous discussion, that creates really rich context from different perspectives for an agent. Then you can say, okay, go do our conclusion, with all the aspects that were mapped out. So there's a huge token spend happening with that, and that's just our development processes and product operations. In terms of how we're talking to people, there's a huge token spend as well. It's just increasing at a

Prakash Narayanan: Are you specifically an OpenAI shop, a Claude shop? Do you use all of them? Do you use OpenRouter?

Samvit Ramadurgam: We use, primarily, OpenAI and Anthropic, and then open source for a few things. We continually run evals that check the different models to find the right cost-adjusted performance for our different tasks. Some tasks want long-term thinking, and some want really fast, cheap responses. There's a lot of high-scale data processing where we have a rejection cascade of models: we do a bunch of work with cheap models, and the five percent that needs more attention, we put the heavier models onto. The idea is to treat tokens as a commodity and be able to swap in the model that's best for the job. It's a big part of our cost. It's profitable for us because we have a business model we can monetize, but it would be pretty cost prohibitive otherwise.

Prakash Narayanan: You had Forge Global, which you built in the pre-agentic era. What do you feel is the big difference between that era and this era in terms of running a company?

Samvit Ramadurgam: In terms of hiring, here's the metric we have. We're a small team, five people, soon to be ten. The heuristic I have for hiring someone is, are they going to prompt me to do something different than we would have done before? You don't really need people to do work anymore for the kinds of work we're doing. You need people with visionary insight who add to the DNA of the company, who make you do things you wouldn't have thought to do otherwise, that you can't prompt an agent to do for you. We want to build the company in a way that's building the machine that builds the machine, and eventually have more and more agents spin it up. There's a type of company you could start now that does with a dozen people what previously took a thousand.

We'd like to be one of those companies and have more of the spend go toward tokens, just to be really lean, really agile, with a concentration of talent and vision. So it's very different. We would have had to hire a lot more people in the past to do what we're doing. It also raises the bar. When you hire people now, the incremental value of interdisciplinary skills increases exponentially. An engineer is valuable. Someone who's an engineer and a product person and a designer is much more valuable than before, because before, that person didn't have the ability to make use of the integration of those different skills. In the modern world, the more skills you can integrate into one system, the more you cut out interhuman coordination by having it done in your head.

You can articulate a clear vision to our agent systems, and we have so much leverage that the bottleneck is not execution manpower. The bottleneck is clear articulation of good ideas, and that comes from having different disciplines coordinated well. That's another aspect of who we want to hire. The unicorns who are good at many things are rare. Typically they're founders, honestly, because that's one of the jobs that makes you do a lot of stuff at the same time. There are other things too, though. And so...

Prakash Narayanan: Amazing. Samvit, thank you for your time. We've gone a little bit over and taken a little bit of your time. It's been a pleasure talking to you.

Samvit Ramadurgam: Thank you. It's been a pleasure as well. You have really great questions.

Nathan Labenz: Great stuff. It's one of the biggest visions we've heard, so looking forward to following your progress.

Samvit Ramadurgam: Okay. Thank you. Thanks so much. Have a good one.

Prakash Narayanan: Bye bye.

Nathan Labenz: You've got to love the old "act one is coordinating the global economy" level of vision. That's really something else. Have we heard anything else that's been as big as that vision?

Prakash Narayanan: I think a lot of people have big visions, but you also need to focus on near-term revenue. He has the advantage that he's basically going to do what his last company did, but with agents. He already knows what the revenue model is going to be, and he already has the initial set of customers, which would take someone else a decade to form relationships with. There's a huge advantage in having sold his last firm and then asking, what can we do better with agents and AI? How do I reinvent what we were doing? What is the bigger picture? That's one advantage. This is exactly why VCs like to invest in repeat founders.

Nathan Labenz: It's such an unbelievably big idea. It has me thinking about the meme, which I don't know that Anthropic ever really said or thought themselves, whatever the reports may have been, that for 24 to 48 hours, maybe a month ago, went around about Anthropic thinking it was going to be the last company, with just Anthropic and governments left at some point in the future as it takes over everything. That remains a little bit pie in the sky. But when you think about this kind of thing, you can imagine how something like that could take shape. I could see a future in which Union gets acquired by an Anthropic, or perhaps an OpenAI or a Google. It's just such an insanely valuable layer, to have a network-effect type of moat and all this private information they've built up over time.

It's literally an angle on running the economy. So it does start to fill in some of the details on what a future scenario like that might look like. Not that I think it's likely that it goes that way, but there are these insane, grandiose statements, like "our first act is to facilitate global commerce," or "this might be the one company left when all the others are gone," that you can squint at and see how we might actually get there. I never cease to be amazed by how wild some of the ideas are now that are no longer immediately laughable.

Prakash Narayanan: One of the things that has happened is that the amount of money you need to found a company has dropped dramatically, and your survivability as a company has increased dramatically, because you don't need that much money anymore. You don't even need the skill sets anymore. You don't need to hire four of the top coders from Google or Airbnb. We couldn't set up a nice web page before Airbnb. I couldn't center a React [unclear]. I was horrible at TypeScript and JavaScript, and most ML people were. That's why Anastasios came out and said, look, we started off as a Python shop and the UI was terrible, we couldn't do all this stuff because we didn't even know TypeScript. That's how bad we used to be. We have come a long way in just the last year, since October of last year.

Now no one is going to flinch at a front end. No one is going to flinch at a back end. People will be talking about what the data does, and so on. The amount of capital required to found a company has come down, so the viability of smaller companies has gone up. That expands the pool of people who can do entrepreneurship and the kinds of ideas you can look at. It also means a lot of the lower-risk ideas, which you could just pour money into, are no longer that attractive, because other people with less money will also get there. I think that's what has happened to SaaS. SaaS, in the next three or four years, will face that. You can see it with pure SaaS, not so much with network-effect players. Maybe Slack is in there. But the pure SaaS like Microsoft... Did you know Microsoft is laying off 25% of its employee base?

Nathan Labenz: Yeah, I saw that. My understanding was that buyouts were being offered to 25%, and we'll see how many take it. But it's a big number.

Prakash Narayanan: And across all of the Office divisions, the Xbox gaming division, they're all being offered buyouts, but they're hanging on to all the AI people. It's going all the way up to senior engineers, level 70, which is kind of principal engineer level. So they're basically overstaffed for the existing work that can now be done by agents. It's almost like a 10x productivity boost, and now you need to find more sources of revenue for the 60,000 people. To maintain them you'd need to expand your revenue by something like 2x. It just shows we need a lot more growth to absorb the latent white-collar humanity that's going to need a job. The economy has to grow faster, or else there are going to be social consequences you don't really want to face.

Nathan Labenz: I can see the growth. I have a hard time seeing the job growth picking up enough to offset it. He just said he has five people, soon to be ten, and that you could have a dozen-person company do the work of a thousand. He said you don't really need people to do work anymore; the only thing that matters is people who will cause us to do something differently than we used to. There just aren't that many of those roles. How many people are you going to have before you can't find that many more people who will change the way you do things? And that's also just not what people have been really developed for. There's been lip service over time to the idea that good ideas can come from anywhere in companies.

But the up to 25% of people who might soon be departing Microsoft primarily have not made their way there by causing the company to do something fundamentally different than it was otherwise going to do. They've made their way there by executing tasks. I just don't see how so many people are going to pivot from task completion to "we wouldn't have done this but for your unique snowflake contribution." It's tough. How many people work in sales? He's basically attacking all of sales.

Prakash Narayanan: I've been doing this thing where I'm in some WhatsApp groups for data brokers. The data brokers put out requests for data, and whatever request for data they're making now, in about twelve months that's going to hit the product. You can see the trajectory, because these guys buy up data from a company, then take that data and create training playgrounds for agents. The agents learn in that playground, and that becomes product at the frontier labs in about 12 to 18 months. The

labs are buying up entire companies' data. They're buying up an entire airline, entire gaming companies, semiconductor companies, manufacturing companies. We spoke to [unclear] about this. They're kind of done with the 50% companies; they're looking at the 100 to 500% companies. That's the ask right now. What I imagine is going to happen in about twelve months is they'll give you a company in a box. You can buy yourself, probably in six months, a drop-shipping company in a box. Add capital, add one idea, say I want to focus on this, go. And it will do all of the Shopifying, the advertising, the analysis, the market entry, the customer service, the returns, end to end, with a small number of decisions that come back to you.

I can easily imagine someone running ten of these companies. A lot of people have ideas. One of the big businesses of the last ten years was these little ornaments kids put on their Crocs. This woman started decorating her kids' Crocs, people asked for it, then she started making them for other people, then she found a manufacturer in China, and all of a sudden Crocs bought her out for a hundred million or whatever. A lot of people do have ideas and are able to execute in small numbers, to make one thing. But the idea-to-commercialization gap is enormous. It's enormous in the sciences. It's enormous when you go to a university like Stanford and you see the ideas but don't see them commercialized.

What I'm hoping is that the agents do the commercialization for you, the gritty entrepreneurial work that used to take a lot of time: calling up Walmart, getting to see the guy to show them your product, raising capital, and so on. A lot of the yes-no decisions at the firms that will make these investments are driven by metrics, number one, and number two, vibes. You can supply the vibes. Maybe the AIs can supply the metrics. Why do you need to be the one who supplies the metrics? So maybe we see this ramp next year, starting with one-man-show types of companies, like online gaming, stuff you can sell on TikTok with some marketing, drop shipping of all kinds. These are the initial companies. I think Andon Labs will play a part; Andon Labs will be selling agents that do this, I think.

That's the initial wave I think we'll see around the middle of next year, probably. Then you go into, okay, I can now run a manufacturing company like this. All I need is to supply the capital; I know the market; I hand it off to the company, I inject the capital, and everything else happens. So I think we see this trajectory starting in the middle of next year, of the automated company, and that is probably what takes the economy to the next level, a system where a single person can manage most things. You still need that human to open the bank accounts, give the broad idea or point things in the right direction, sign off on the capital, and decide whether to reinvest or take the money out and enjoy it. I'm hoping. I'm hoping. I'm hoping.

Nathan Labenz: What I do expect is certainly a ton of these one-person companies, a ton of cool ideas that never got to exist before. I've got one coming real soon, which I think we've talked about a little bit: my Fruitopia app, which I've been working on a little, mostly just prompting agents to work on it. I always thought there should be some glorious database of fruit, and particularly of what is good now, because I never really know. There's a whole bunch of detail that goes into that. Even on a mango, what kind of mango are you actually getting? It varies by the time of year, and they don't really label it in the supermarket. I've told

this story to so many people, and they're all like, oh yeah, I've noticed that. Some of the mangoes are really amazing and others kind of suck. The answer is it's a different variety, but you're not told what variety it is at retail. So on my own, in my kind of 2% time, I'm able to spin up what might become the world's biggest database of retail fruit information. And I don't even need it to get traction. It could just sit there and do nothing, and that's fine. Or maybe it'll help people eat more fresh, delicious fruit, and that'll be cool. So I think we'll see a lot of these things, but I just don't see how there can be enough of these little niches to absorb all the people who used to be driving cars,

Nathan Labenz: ...doing tasks. This is where it seems very, very hard for me to get over the hump and envision so much of that. I've been limited in my vision before. If we dig up my old comments on Arena, they will perhaps reflect too limited a vision, but I do still struggle to see this working in a way that's not majorly disruptive.

Prakash Narayanan: A lot of restaurants start off with one good home cook, and that's the typical start of a family-owned restaurant. But family-owned restaurants have really died out, because it's a huge amount of work and the profitability just isn't there once you get through the licensing, the location, and the labor. If you look at a restaurant, 25% is labor, 25% is rent, 25% is materials and ingredients, and then there's a 25% gross profit margin, which then gets spent on sales and marketing and so on. You're lucky to clear 5 or 10% on a typical restaurant.

And maybe if you get the bots who can work in the kitchen, you get rid of the labor. The labor component becomes capital, and then you're at 50%. All of a sudden, the restaurant business becomes enormously profitable. You're down to your last 5% of margin after the sales and marketing expenses, but if you get rid of the labor and maybe pay 5% for the rental of the bots, 20, 25%, you've just made the restaurant business something like five times more profitable. And if it's five times more profitable, then you can open up more restaurants.

Nathan Labenz: We can only eat so much food. At the end of the day, I'm only going to eat so many calories. You're describing a world where all the restaurants massively cut their staff and replace their kitchen workers with robots, but we'll have a lot more restaurants. This is the classic "we lose money on every one and make it up on volume," brought to employment. I just don't see the math working out on that. There may be a lump of labor fallacy, but I'm pretty sure the lump of food principle is roughly right: I'm only going to eat so many calories, and at some point that is just plain saturated. We'd have to posit a world where I'm eating from five different restaurants for every meal or something. Maybe my vision is too limited. That could be kinda cool, I suppose. But at some point...

Prakash Narayanan: I would imagine something like dense Asia. Dense Hong Kong, apartments where you almost cannot cook, and you run downstairs. Walkable places. Rare in the US right now, but things can change over time, perhaps. But you are right: I don't think human consumption is really where the growth is. The growth is always going to be in AI consumption going forward. You have these concentric circles of the economy, and the AI economy is the one expanding fastest, so anything that supplies into the AI economy is going to grow. Anything that supplies into the human economy is going to be on the down drift, except for status.

I think Peter has come out with the relational economy, and Alex [unclear] is at, I think, OpenAI right now on the relational economy. So that could be a big thing. But again, status economies are not meritocratic. We have tended not to like them for that reason, and that has more social repercussions, perhaps.

Nathan Labenz: It sounds more dystopian than utopian to think that the bulk of human activity will go into status games, I have to say. I do agree there's no limit, or at least I don't think we're anywhere near the limit, of how many status games people might want to play if they're freed from other task-completion-style drudgeries. But it is still kind of zero-sum, and it does sound rough. I think a lot of those games don't leave people feeling very good.

I'm a bear on employment, but bullish on people's ability to find meaning in the absence of employment, certainly relative to most. I've said this probably a dozen times on this feed, but way too many people in the AI policy space forget, or maybe never even realized, that a huge portion of people are working to live and not tying up their identity or sense of self-worth with their work the way the academic or intellectual class tends to. So I'm more bullish that people will find things to do that make them happy. But if you told me all this excess energy goes into status games, I would be kind of bummed out, because a lot of that stuff ends up being fairly empty, so zero-sum and positional. Again, it sounds more dystopian than utopian to me.

Prakash Narayanan: But look, Anthropic's founders donating 80% of their returns from Anthropic into the nonprofit, that is a status game. Philanthropy has always been a status game. Elon donating his money to the Trump accounts.

Nathan Labenz: That's why the EAs have become so high status.

Prakash Narayanan: Yeah. The entire point is not that status games are bad, but that we manipulate these status games to deliver prosocial outcomes for humanity as a whole. Religious tithing is a status game. People tithe in order to get that kind of status in every single religion: the rich providing for the poor, giving free meals, and all of it is status. We come up with these illusory rewards to motivate prosocial behavior outside of currency, outside of lucre.

If we're going to an abundance economy where lucre doesn't matter anymore, what you end up with is this kind of status, and you have to make people feel good about it. You have to give them props for their prosocial moves and recognize that they are doing things which are good. There is an objective sense of good, and if you are performing it, then you deserve something. You can even maximize it: the maximum is that you're anonymous when you do it. That's the next level up. Then you pass away, and people find out you were the anonymous donor to X and Y and Z. But it's bad when you say it's status, and great when you say it's philanthropy. It's such a Russell conjugation thing: I'm doing it for status, you're doing it for philanthropy and the good of mankind.

Nathan Labenz: I'll maybe take the pitch on that distinction for the moment. I think you can imagine what my take would be. Certainly there's some truth to what you're saying. I think certain communities would flatter themselves to be a little more focused on the actual prize than the status, but that may remain in the eye of the beholder for the foreseeable future anyway.

Well, I guess we gotta go benchmark Gemini [unclear] for ourselves, among other to-dos. Definitely going to sign up for Union as well, and it is something people can sign up for. I'm very curious to know who it thinks I need to meet, and we're interested to see what new connections it brings into my life. Anything else we should touch on before we go?

Prakash Narayanan: No. We'll be back on Friday for a show. This has been great. All the guests today were fantastic, and my past caught up to me in the first one, so, room-temperature superconductors. I gotta go see Periodic Labs and talk to them. I should pitch myself as the room-temperature superconductor pitchman for whoever invents one.

Nathan Labenz: That's funny. It will be a good show on Friday too. We've got Michael Freeman from ServiceNow and Sagar Vaze from Mistral, so definitely a couple names people will know, and I'm sure there'll be some interesting discussion and hopefully a bit of alpha spilled with us on Friday once again.

Prakash Narayanan: Indeed. Nathan, good morning.

Nathan Labenz: Good morning. See you on Friday.