Hey, everybody. Welcome to The Six Five Summit: AI Unleashed 2026. We're continuing the conversation with another AI Infrastructure Spotlight. I'm Ryan Shrout, joined by my colleague, Matt Kimball. Today, we're going to talk about a challenge nearly every enterprise is wrestling with today. How do you build AI infrastructure? How do you build it fast enough? How do you build it flexible enough, cost-effective, to move beyond pilots and into real production, which seems to be a topic we touch on quite a bit these days. Joining us for this conversation is a good buddy of mine, Anil Nanduri, Vice President of AI Products and Go-To-Market for Intel's Data Center. Welcome back to The Six Five, Anil. Hey, Ryan. How have you been? It's been a while. It's been good. I'm looking forward to this conversation. I know you guys have some exciting stuff happening over there. No, thank you, and great to have me on your show, and really looking forward to this conversation. Hey, Anil, let's jump right in, okay? Here's the setup. We talk to enterprise IT leaders all the time. The excitement, you can't get around it, and it's really the topic of every conversation. But as they get excited about the pilots that they're driving, what we hear is rising token costs. You hear unit costs coming down, but really in volume, it's increasing. New agentic workloads, how do you manage these? And this growing pressure to manage ROI. You talk to more customers than we do, obviously. How do you think they should be thinking about these challenges as we move into enterprise production? Great question, Matt, and I think the fundamental thing is a lot of our customers, they start thinking about hardware first. I think we really have to change the conversation to ROI. Why is that? Because it's all going to be around how AI can solve a problem. And from that perspective, you've got to really deconstruct how you're going to apply AI to what frame of the business and where you're going to drive the value. Based on that, you're really going to get a framing of what kind of budgets you're dealing with, what kind of cost economics you're dealing with, and what kind of a productivity improvement you want to be aspiring for. It's basic first principles of how you want to run through an operational aspect of it. Now, interestingly enough, because AI is coming at such a rapid pace, you want to have that innovation engine going, which means that you want to do a lot of pilots, and that's fine. You sandbox them, create your pilots. But as you think about getting into production, you want to be a lot more clear about how you want to deploy it. There's a lot of things, like you said, is happening. It's no longer about token maxing, which is it's not like the highest number of tokens will give you the best ROI. It's actually the most efficient use of tokens that get you the ROI. And then I think I do want the audience to have a framing that not all tokens are the same. And then elaborate over the next half an hour what that really means. But not all the tokens are the same, which means that you're going to have ultra-premium tokens, which is super low latency, super fast, very good for interactive. If you're trying to do some credit card fraud protection, you want to have almost real-time AI. You want that fast. On the other hand, you're doing some auditory work and you need to present an audit report in 20 days. You don't need that super fast AI. You could handle it very differently in a batch mode. So those ROI conversations start to play into how you think about putting AI into production. One of the things that we hear about, we are talking about all through The Six Five Summit, is agentic AI. We went through this whole early LLM to reasoning LLMs to now agentic AI, and it is the biggest conversation that we have, and it dives into your point about not all tokens being equal. I am curious from an infrastructure perspective, what fundamentally changes when these AI systems begin planning and coordinating and taking actions instead of simply generating a response? Ryan, another great question. What do agents do? They automate the work that we were trying to do. As more agents come in, you really need to have the agent loop to control how your agents operate. If you step back and look at how these AI systems work, we spend a lot of time trying to figure out where to run the AI. That AI is generating tokens, which is how you consume it. Token generation, as I said earlier, not one size fit all. You can generate tokens with small models and run it slow, but can very efficiently, even on a CPU, they are capable of running some level of tokens. As you know, GPUs start to go up the stack. GPUs are great for the flexibility they offer, some density of tokens you can run. But if you are really wanting to get a large throughput and you want to really scale it, even the GPUs hit a Pareto limit. That is why you have solutions like Dataflow architectures, like an RDU or a Groq-like solution with LPUs. They are starting to kick and take the low latency, high interactivity, large volume of tokens that you need to generate. Token generation itself is a lot of optimization you can do, and then we can talk about that and to how you bring that into the business economics. The second part of it is what do I do with these tokens? These tokens are actually, especially in the agentic world, you are creating outcomes. Say I am creating an application, or I am trying to improve and refactor an application. As you are thinking about that, agents are helping you create code. You are going to test that code. You need to verify, test, execute that code. That is where CPUs start to play a lot of role because CPUs are their execution engine, verification engine, and your sandboxing engine. If you see recently why are CPUs in a shortage is because a lot of these back and forth ping pong is happening in the system. You went a step ahead, as when I think about as agents come in, now you need to figure out who is orchestrating it, who is the quarterback? The quarterback function is becoming more and more important. It is about what we use the term harness, which is basically saying: how am I going to run what workload? Where am I going to run it, and who is going to run it? Here you are looking at saying: what kind of agents do I need? What kind of agent loops do I need to orchestrate? What kind of security do I need? Do I need to run the model that is sitting on-prem and where my data is, or do I want it to be running in the cloud and move my data over? These are all going to be very important CIO decisions, and the enterprise have to really look at am I taking my data to the AI or am I bringing AI to the data? That's one way to think about it. But then as you execute this, you start to see different problems in the data center system. Your stress points are changing. Your stress point moved from being about generating tokens, which by the way, is a key function and you're spending a lot of cost on it, to the running of the full end-to-end workflow where the wall clock time matters. These tokens, based on the systems you're running, you're stressing based on the context lengths, you're stressing memory, you're trusting storage, you're hitting databases, and now you're waiting on functions where if you're A good example is you're an enterprise and you're trying to look at your supply chain procurement, to a customer demand, and your financial tools to make sure when you can, say, invest. Say you have a huge amount of demand and you want AI to help you manage this. Your financial system is in one place, your customer relations and customer data is in one place, and your supply chain is in one place. You will now have to have APIs, and the agents are really good at getting data between all these systems. You're not going to move all that data over, but you're going to have these agents interact and get the information from those systems. The wall clock time has shifted from just the AI running to actually waiting on answers. You really have to look at a end-to-end system throughput. I like to use the word wall clock time, measure the wall clock time, not just how one independent piece of the system performs. We are at a very interesting point in the market now. Memory costs have gone up so significantly that in the past you didn't worry too much about it. You would say, "Hey, throw more compute, throw more storage, throw more memory at it." We were just running fast. Now you have to step back and say, "Okay, am I getting the best efficiency out of what I'm investing in? Do I have all the right set of pieces so that I'm taking the most expensive part, that AI engine and the AI intelligence, and I've got everything else working, humming with it in the right efficiency so I could get the best ROI?" The agentic infrastructure actually stresses this at a much more complex levels than before. That's interesting you say that, Anil, because enterprises are spending lots of money on AI infrastructure, right? Tens of millions of dollars. It seems like there are a couple thesis. The industry is saying, "Here's a benchmark score for you that shows you how great we are, and we're going to throw some gaudy numbers around tokens per second to really wow you as enterprise CIO and convince you that we have the best solution out there," right? We all know in enterprise, there is this intrinsic understanding that AI is going to be good for business, right? But the board doesn't care about that. I'm giving you $10 million to go stand something up. I want to see some measurement that shows or some way of showing that that infrastructure is creating business value. Have you thought this through and have you had this conversation with IT leaders and how they do that? Very much so. I think one of the more public examples lately, I think the Uber CTO had stated how they ran past and blew past their token budget in a few months, their annual budget that they had looked at, right? This week, in fact, yesterday or day before yesterday, they had their earnings call, and their CFO and the CTO shared on how they've changed in a matter of months. Then what they did was they said, "Hey, we've started to bring down our cost of AI implementation while increasing the amount of AI that they're implementing." How do they do that? First, what they said is that they optimized and made the AI system of operation more efficient. What does that mean? What they did was that if you look at it saying, how do tokens get generated? If you have a very deep agent loop, and if you have a very inefficient agent loop, or if you have a very inefficient prompting and context caching mechanisms, you are going to create more tokens, okay? As we move from a person in control to machine to machine, you really need to have the tools in place to make that more efficient, right? The algorithms at the agentic architecture level itself, you can actually optimize, which means that by using the right prompt and the right context and the least amount of tokens, can I get the same answer? That's more often thinking of the problem of not throwing more tokens at the problem, but think of it as an engineering efficiency solution problem, right? They did that first. The second one they did was actually more interesting. They gave visibility to the engineers themselves, and the employees the use of the AI. They give visibility in how much tokens are you consuming, how much budget have you consumed. Observability into the system is actually very important as an ROI measurement. Once you know where the problem is, you can go attack the problem, right? The first step is to understanding where in that agent loop, where in the AI loop are you actually being most expensive, and then you can go look at it and tackle it. The third thing, which is actually going to get more and more interesting, is the change in the landscape around open weights, open models, and the closed models. Frontier models are great, but they are expensive. There is a lot of innovation going on where open weight models are catching up. Going back to my earlier statement, not all tokens are same, you can actually now start to control what part of your workflow needs what type of a model. In fact, what type of a token and what type of an agent. That, again, is applying some human creativity into solving that workflow, right? Because you can come back and say, "80% of my work can be done with the basic open weight tokens, and I need, say, 20% of it to be that premium intelligence token." You are getting into, this is no different from what enterprises have solved for when the on-prem and cloud debate went on, right? Right. You would think that, hey, cloud is going to take over everything, and then you are going to say, oh, on-prem is more cheaper way to do it. What really transpired is a very hybrid infrastructure. I see that same thing happening. Why do I believe in it? Because if you think of the Hermes or OpenClaw moment that happened for consumers, where you are really creating a personal agent, and you can run them on a Mac mini or a workstation desktop and run a bunch of local models. You are going to basically enterprise that at scale and get into a hybrid mode of operation. Right? In that hybrid mode, that AI that you are running, not all of it need to be on-prem. You could say, you know what? The open weight models are still running on a cheaper cloud. I have a more expensive cloud. It is going to manifest itself into that way, and I think it is just a question of time. We are already seeing the ones who are early adopters in this benefiting a lot by trying to get to that kind of a hybrid deployment. I know it will not come as a surprise to you, Anil, but I have a lot of thoughts on how we measure performance, how we benchmark these things, and getting away from things like tokens per second as a metric of value and more on completed correct work per token or per million tokens or something like that. We have already been doing a lot of thinking and some developments on how do you measure success by business outcome, or by some kind of correct result at the end of the day. It is a combination of what model do you pick, what infrastructure do you pick, what are the tool sets that those agents may or may not be using. But I am curious from your view, if the goal is to measure business outcomes rather than individual component performance, which I think we can all agree is where you get your measured ROI. How do you talk to enterprises that are thinking about building out that infrastructure so that they can balance what CPU do I need, what GPU, what accelerators, what storage, what different deployment environments to support those workloads? I think that is a great question, and I think, Ryan, you would agree a lot more work has to be done here into kind of providing the benchmarks that kind of reflect this. But at a first-principle level, you could actually analytically come to that conclusion. The most important thing is that if you break down the hierarchy of your infrastructure into a few simple building blocks, I would call them first just a control plane. Okay, there is the application plane, which is where your software runs. That is fine. But there is a control plane. Control plane for me is like the quarterback. This is an orchestrator. Think of it as your agent router, your model router. You can figure out where your security harness is. This is where is really putting that the kind of the backbone of your operational controls in place. These typically run on CPUs, and so you want to make sure you have a very solid control plane thought process into saying that, "Hey, I got to figure out that I got the right CPUs in there and most general purpose and most flexible," because that is what you need. Then you really focus on things like having security features and capabilities that get you that. Then there is the agent plane or the data plane, where the tokens reside, your task execution, sandboxing. You, again, have CPUs for the sandboxing and execution and verification, but your token generations. Now, in the token generation, like we said, you are going to have to figure out what type of tokens do you need and where do you source it from, on-prem to CPUs to GPUs to RDUs. There, the measurement has to be around your token throughput and your dollars per token and what type of tokens, like whether it's an open source, open weight model, or you're using a foundational model. Costs vary, right? You need that configurator to know how much of your budget is going there. Then you need to think about the network plane. Network plane is that what's happening in the ecosystem, and I think people don't see this as often, is that the models are getting more and more complex and bigger, even in the open weight space. They are crossing chip boundaries, which means that one chip cannot run a full model. You have to run this across few chips together, stitched together, then you got to have to have data and systems communicate across racks and across servers, I mean, across data centers. Your network element, both scale up and your scale out, starts to play a big role on this outcome measurement, which is your wall clock time. Because that's going to be a bottleneck in it. Then the storage side of it, which is like, hey, where are your databases? Where's your KV cache? Where's your context memory pooling going on? Where's all your storage data, and how do you get access to it? Again, they all run, by the way, hosted with a CPU, but very appliance-like solutions out there. When you think in these layers, you got to make sure that your investments are kind of balanced evenly or to the right mix. Because if you don't have a great orchestrator, but you have a great token efficiency approach, you still are missing out. If you don't have the right storage to go with it, you're not going to get the right efficiencies. If you don't have the right network, you're not going to be able to then leverage all of these as they interact. That I think is the aha moment right now. Then you can go figure out from the bigger attributes of your budgets, is it better for on-prem, better for hybrid, or better to go to the cloud? That I think is the first principle view of what is the AI agent workload look like when I deploy it, then how is it computationally going between these phases of the workflow. So you have to think of this as a workload, and we've done these before. We've had HPC workloads, we had media workloads, we had health and life sciences workloads, and we've been able to break that up and understand the computational dependencies and which part of the system you stress. So I think it's just going back to that, and I think it's just that AI moves so fast that people always thought of it, but now they're like looking at- Yeah Is there a cheat sheet? Can you give me a configurator? Can you give me something easy for me to understand and deploy it? It's interesting you say that, Anil, because if you think over time, the last 30 years or so in the data center, there's this constant evolution. You see one inflection point after another, virtualization to cloud, even before that. Digital transformation. AI is different, as you said, it moves a lot faster and it's a lot different. I'm an enterprise CIO, I'm trying to look five years out on the horizon, and there's a point there where my data center, my workloads, my business is AI-ified, if you will, right? Yeah. There's a lot of different infrastructure that I deploy than what I've been deploying in the past. There are different strategies. You talk about first principles, and I think that's a good thing. If I'm an enterprise CIO, what's the biggest shift I need to make? What's that biggest kind of flip I need to make in my brain from a mindset perspective before I begin that mapping out that strategy and what that endpoint looks like? Actually, that's a very insightful question, and I would have a view on this in a very simple way. How does AI get smarter? It's a first, again, back to first principles. AI gets smarter by learning from more data. You kind of look at the ecosystem and say, why are all these AI models now really a lot of press and noise and about like, hey, these open source models are catching up to frontier models. There's a reason for it. They've pretty much scrapped all the data that they can learn from. You are getting to a point where now you're trying to create synthetic data, you're trying to go now create more simulations, and you're trying to get models better. But in terms of crawling for data that exists for you to train these, pretty much you could take and debate 90% + of the data is already being accessed, right? Now, where is the rest of the data? The enterprises have that data. They are at a very critical point on how are you going to now bring AI to your data? I believe that there is going to be ones who decide, saying, "You know what? I am going to take the easy button and just going to go access and provide access of all my datasets into these frontier models and other models." What is really happening is those models are going to get smarter and learn from your data and understand how your business works. Then there is going to be the other set of CIOs who are thinking about it and saying, "How do I protect my data? How do I put the right harness and the right hybrid structure where I want the benefits of AI working towards what I need to do?" Not, on the other hand, let that reverse osmosis happen where my knowledge is going back into the AI system. Can I do that? Yes, you can. How do you do that? This is where the open weights model, as well as how you create your harnesses start to matter. How do you create your architecture saying, I am going to sandbox certain capabilities, but I do want the latest AI tools so you can fine tune models, you can make them run smaller, because that is the great part. What they have shown us is that I do not need a 3 trillion model to be on a very domain specific expert. I can actually distill it, I can fine tune it, and make a much smaller model that is much more capable on my dataset than trying to solve everyone else's problems. That inflection point in the next few years where the ones who are going to win in the enterprises are the ones who very, very strongly protect the data and ensure that their data is again, and you bring the value of AI into where your data is, not the other way around. I think that for me is going to be that critical inflection point of figuring out how this AI-fication, as you said, into enterprise becomes a business success or they get disrupted. Anil, that's a lot for us to lean into, but I'm curious, what's Intel's view? How does Intel plan to address all of these problems and segments, right? You've got a robust portfolio and one that's growing, but how do you view Intel's place in this? Let's look at it from a perspective of the aspect that not all tokens are the same, and inferencing can be done in very different compute domains. Whether you are in a brownfield infrastructure or a greenfield infrastructure, your choices and decision trees change, right? How we've taken an approach is we do a lot of compute. We do this notion of heterogeneous computing because it is a combination of CPU, GPU, different kinds of compute elements that are going to work together. The best example of heterogeneous computing has been the PC. It's evolved, and you can see how things get better power efficient, more energy efficient, and better performance over time to run all kinds of tasks you want, right? Now we are taking that first principles from the PC, and we are scaling it to data center. We kind of bringing up from our Arc series of GPUs to Crescent Island. We are doing things differently, knowing the problem is going to be on ROI at cost, right? With Crescent Island, we intentionally chose low-power DDR instead of GDDR or HBM memory. We brought in the solution of bringing it into a much more energy efficient, cost efficient. It's not going to be the more ultra premium tokens. It's not low latency, but it has a very good sweet spot, where a lot of enterprise customers can plug in a PCIe card and scale their business as they need it. It's very easy to go install it and deploy it. That's on one side. On the other side, we've partnered with SambaNova. SambaNova, on the other hand, is like this huge pipe of tokens you can generate, right? At low latency, very cost efficient, but it works for a lot of high interactivity, right? It has its place. When somebody wants to get a huge pipe of tokens of a specific model to run, something like a SambaNova really works well. We are partnering very closely with them. As I said earlier, a lot of the wall clock time is going to be driven by tasks that sit outside these models, which is going to be access to your verification, execution, your orchestration, and your storage plane, and your network plane. We are doing a lot of work on the CPUs that actually can help provide that efficiency and performance so you can start to look at a wall clock time problem, right? And so if you think about our Xeon portfolio plus, looking forward to Crescent Island and our partnership with SambaNova Systems, we are looking to provide that capability purely from a cost efficiency, TCO, ROI kind of approach. Now the deployment mechanism could go from on-prem to a Neocloud to somewhere else, right? And so that's how we are thinking about it. We are super excited to put the building pieces, or the building blocks for these, and look at it from this notion of outcomes, than just look at it from a specific individual piece of compute. So there's clearly no shortage of problems and solutions to create from this, and I think Intel has a very interesting portfolio here and one that is growing and expanding. So I'm looking forward to having more of these conversations with you as that kind of progresses, Anil. Thanks for joining us for this AI Infrastructure Spotlight today. It was great to have