Inference (running models for users) is now ~two thirds of AI compute, and that is the workload now driving custom-chip economics.
Custom AI chips (ASICs) are growing ~44.6% per year vs ~16.1% for merchant GPUs, the first inversion of the AI era. (SemiAnalysis / TrendForce)
ASICs hold a ~40-65% total-cost-of-ownership edge over general-purpose GPUs in large-scale, multi-year inference deployments. (SemiAnalysis / Bernstein)
Operator takeaway: serving costs are set to fall as the giants re-engineer the engine for themselves. Capture the savings into your margin; the leverage is the workflow, not the model.
Inference-only ASIC on TSMC 3nm, designed from scratch around LLM math; ~50% cheaper inference reported (Bloomberg). Design-to-tape-out in ~9 months, aided by OpenAI's own models. Training stays on Nvidia. Engineering samples already running GPT-5.3-Codex-Spark in-lab; production deployment targeted late 2026; Microsoft committed to a major share of first output. Caveat: no disclosed performance targets, savings figure unverified.
Anthropic accuses Alibaba of largest-yet distillation campaign against Claude (June 24)
Letter to US senators/White House: ~25,000 fraudulent accounts, ~28.8M exchanges (Apr 22-Jun 5), targeting software engineering + agentic reasoning. First time Anthropic has named a major Chinese conglomerate; 4th Chinese lab it has named overall. Sens. Hagerty and Kim plan a defense-bill amendment to blacklist/sanction offenders. Alibaba stock fell ~3%; Alibaba declined to comment. REPORTED from a letter Bloomberg obtained. The thread: OpenAI fights to make serving the model cheaper; Anthropic fights because the model itself can be copied. The model is a commodity; the moat is the layer around it.
The written brief from each episode, free, every morning.
Transcript
Nova: This is Mainframe, your daily guide through the AI chaos: what actually happened, who is winning, and how to take yourself to the next level. It is Thursday, June 25, 2026. I am Nova.
Dex: And I am Dex. One honest note before we start: our voices are AI, but the reporting, the analysis, and every pick on this show are human-made. Find us on the podcast, on YouTube, or in the free newsletter that drops every story and link in your inbox each morning. Subscribe wherever you are, because this world does not slow down.
Nova: On today's Mainframe: OpenAI builds its own chip, and the whole industry's cost math just shifted. Anthropic accuses Alibaba of industrial-scale theft of Claude. And two tools that quietly cut your token bill by more than half.
Dex: Plus what all of this means if you are a solo builder watching the rent on compute.
Nova: Let's get into it.
Nova: Start with the Board, because today's biggest story sits right on top of it. The whole game in AI is three questions: who has the chips, who has the money, who has the customers. Until now, the chip answer was simple: Nvidia designs, TSMC builds, everyone else rents. But the ground is shifting under that, and the shift has a name. It is called inference.
Dex: Define the two halves for people.
Nova: Training is teaching the model, an enormous one-time effort. Inference is the model actually answering you, every single time, forever. Inference is now about two thirds of all AI compute, and that is the part that runs on repeat. Custom chips are growing nearly three times faster than GPUs, at a forty-four point six percent annual rate, targeting inference where workloads stabilize around known model architectures and cost-per-token economics dominate.
Dex: And the reason is money, not pride.
Nova: Industry analysis from SemiAnalysis and Bernstein estimates that custom chips offer a forty to sixty-five percent total-cost-of-ownership advantage over general-purpose GPUs in large-scale, multi-year inference deployments. That is the backdrop for our top story.
Nova: Story one. Yesterday, OpenAI and Broadcom unveiled OpenAI's first custom chip. It is called Jalapeño, pronounced like the pepper, and it is built for one job: serving models, not training them. It is built on TSMC's three-nanometer process, designed from scratch around the specific math that makes large language models run, and Broadcom CEO Hock Tan literally walked into OpenAI's offices and handed Sam Altman a silicon wafer.
Dex: The headline number?
Nova: It reportedly cuts inference costs by about fifty percent, according to Bloomberg. And here is the detail that should make every hardware engineer sit up: OpenAI says Jalapeño went from initial design to manufacturing tape-out in just nine months, a speed it credits to deep software-hardware co-development, with OpenAI's own models used to accelerate parts of the design.
Dex: Nine months. New chips usually take years. So is this OpenAI declaring war on Nvidia?
Nova: No, and this is the part people are getting wrong. Jalapeño is an inference chip. Training stays on Nvidia, and it will for a long time, because pre-training needs flexible, irregular compute that these specialized chips handle poorly. This is not a coup. It is OpenAI trying to stop bleeding money on the part of the business that runs forever. Remember the leaked financials: negative one hundred twenty-two percent operating margin. If OpenAI can drive down the cost of inference, it can recoup some of the losses spent on costly training, and that may reassure investors ahead of its anticipated public offering.
Dex: And one fact that tells you who is really backing this: Microsoft has already committed to absorbing a major portion of the first production run. So the buyer is locked before the chip ships.
Nova: For the operator, the Board takeaway is concrete. The cost of serving a model is about to fall, not because anyone is being generous, but because the giants are redesigning the engine to make it cheaper for themselves. Your job is to be ready to pass that savings into your margin instead of your client's invoice. The leverage is the workflow you wrap around the model, never the model itself.
Dex: Story two, and it is a different kind of fight. Anthropic has accused Alibaba of waging the largest distillation campaign yet against a US AI company, telling senators and White House officials that operators linked to Alibaba's Qwen lab used nearly twenty-five thousand fraudulent accounts to run roughly twenty-nine million exchanges with Claude, targeting software engineering and agentic reasoning, the model's most commercially valuable skills.
Nova: Explain distillation for a smart adult who does not live in this.
Dex: Distillation is feeding carefully built questions to a frontier model, collecting its answers, and using those answers to train a cheaper rival that approximates the original. Think of it like sending thousands of students to secretly record a master chef, then opening a copycat restaurant using his recipes without paying for the kitchen, the failures, or the years.
Nova: And the safety wrinkle is real. Anthropic warned that adversarial distillation lets Chinese labs replicate frontier AI at a fraction of the training cost, and that models built this way often lack safety guardrails.
Dex: There is a genuine irony here, and Anthropic is caught in it. The company needs the government to crack down on Chinese labs extracting its technology, while it is simultaneously fighting that same government's decision to restrict its own products. Remember, the US suspended foreign access to Anthropic's top models earlier this month.
Nova: And lawmakers moved fast. Senators Hagerty and Kim plan to attach an amendment to must-pass defense legislation that would blacklist or sanction any Chinese firm found improperly accessing US AI model output. The reported figure for Alibaba's stock: it dropped nearly three percent on Wednesday. Mark this as reported from a letter Bloomberg obtained, not adjudicated fact. Alibaba declined to comment.
Nova: The thread connecting these two stories: the value in AI is migrating away from the raw model. OpenAI is fighting to make serving the model cheaper. Anthropic is fighting because the model itself can be copied. The model is becoming a commodity. The moat is everything around it.
Dex: Which brings us to the Tool Lab, where today's picks do exactly that: they wrap value around the model.
Nova: Two tools, both aimed squarely at the cost story. First, the one trending right now. There is a class of tool that compresses tool outputs, logs, files, and retrieval chunks before they reach the model, claiming sixty to ninety-five percent fewer tokens for the same answers, and it ships as a library, a proxy, or an MCP server. If you are an agency paying per token, that is your bill cut by more than half for plumbing you never see.
Dex: How do you actually use it.
Nova: The proxy mode is the trick. You point your existing app at it as a drop-in endpoint, it squeezes the bloated context on the way through, and your code does not change. Start there, measure your token count before and after on one noisy workflow, the kind drowning in log dumps, and keep it if the answers hold.
Dex: And the hidden gem, which is not on the front page but solves a daily pain. It is a code-intelligence MCP server that indexes your codebase into a persistent knowledge graph, an average repo in milliseconds, supports a hundred fifty-eight languages, answers in sub-millisecond queries, claims ninety-nine percent fewer tokens, and ships as a single static binary with zero dependencies.
Nova: Translate the benefit.
Dex: When your coding agent needs to understand a big repo, it normally reads file after file, burning tokens and time. This pre-builds a map of the whole thing once, so it auto-syncs on code changes and works with Claude Code, Codex, Gemini, Cursor, and others, fewer tokens, fewer tool calls, one hundred percent local. Drop in the binary, point it at your repo, connect it as an MCP server in your editor. Your agent stops groping in the dark and starts reading the map.
Nova: Both picks, same idea: the smartest move in twenty twenty-six is not a bigger model, it is sending the model less garbage.
Dex: Today's Mainframe is brought to you by Outpace. Everything we covered today is about the cost of compute falling and the value moving to the layer around it. That layer is where real software gets built, and it is where Outpace lives: one operator who ships modern web and AI work and treats your project like a partnership, not a line item on a mega-agency invoice. If the consolidation has left you feeling like a ticket number, that is the human alternative. The link is in the brief.
Nova: Deep dive, and it is the chip. Step back and look at what Jalapeño actually means, because the press treated it as a gadget reveal and it is much bigger than that. OpenAI now designs the model, writes the kernels, runs the serving system, and designs the silicon underneath it, so that each layer can be optimized around the same goal: making its models faster and more affordable. That is the whole stack, except the one part nobody can own.
Dex: The fab.
Nova: The fab. The gap OpenAI has not closed is fabrication, and that stays at TSMC. Which loops us right back to the Board: every serious player, Nvidia, Google, Amazon, Meta, and now OpenAI, is funneling through one Taiwanese foundry running at full capacity. Owning your chip design does not free you from the line. It just gets you a better seat on the same train.
Dex: And the skepticism people should hold onto.
Nova: Hold onto a lot of it. That fifty percent savings figure compares against current graphics processing units, a phrase flexible enough to mean whatever is most favorable, and independent verification has not happened yet. The companies did not disclose performance targets, so the claims should be taken with a grain of salt. Engineering samples in a lab are not chips in a data center serving your app. Deployment is slated for late this year.
Dex: So what does a normal builder actually do with this news? You are not taping out silicon.
Nova: No, but you act on the trend it confirms. Here is how you put it into action this week. One: assume your inference costs are going to fall over the next year, and renegotiate or restructure any client contract that locks you into today's API prices as a permanent pass-through. Do not sell next year's cheaper tokens at this year's price, or do, and keep the spread. Two: instrument your token usage now, today, per workflow, so when prices drop you can prove the savings and claim them. Three: put both of today's tools to work, the context compressor and the code graph, because the fastest cost cut available to you is not waiting for OpenAI's chip, it is sending the models you already use far less junk. The giants are spending billions to make inference cheaper. You can capture the same win this afternoon with a proxy and an MCP server.
Nova: Your level up this week: install the context-compression proxy on one token-heavy workflow and measure the before and after. If it holds at half the tokens, you just found your margin.
Dex: And if you want all of this in writing, the numbers, the links, every story, the free Mainframe newsletter lands in your inbox every morning. Subscribe once and stop refreshing twelve tabs.
Nova: That is today's Mainframe. The brief is already in your inbox. See you tomorrow.