Ladies and gentlemen, the program is about to begin. At this time, it is my pleasure to turn the program over to your host, Wamsi Mohan. Please go ahead. Yes, thank you so much. Good morning, everyone. I'm delighted to welcome you to day two of BofA's Virtual AI Conference. Today we have with us IBM. We're joined by Matt Hicks, who's the President and Chief Executive Officer of Red Hat. As a reminder, Red Hat was acquired by IBM back in July of 2019. It's a leader in enterprise open-source software solutions and hybrid cloud for building and managing IT solutions. Matt, thank you so much for taking the time to be here. We really appreciate it. No, thanks for having me, Wamsi. Great to see you again. Let's start maybe big picture. Can you just at the highest level, talk about IBM's AI strategy and how Red Hat fits in? Yeah, absolutely. I'll start with Red Hat. I think that's the easiest piece there. We have two offerings and first is RHEL AI, and our goal with RHEL AI is to bring together a model, the ability to put your data into a model, then the ability to run it. It's sort of that the smallest unit of making a GPU work for you in that space. If you're able to be successful, if I've trained a model and I can work it, OpenShift AI can expand that to a cluster. OpenShift AI, we sort of say it brings the scale. I can not be successful with one model. I can be successful with 10 or 100 or a lot of volume. From that point, there are two really exciting paths for me. One is sort of depth to the infrastructure, where we take this work, we drive this all the way down to Z and the efficiencies in Z. It's at the heart of a lot of businesses on it. The Telum or Spyre processors, we want to make sure Z can participate in that really efficient running of the models that you build. At the same point, if you've built 10 or 100 models, we know the next journey for enterprises is I need to bring these closer and closer to my business processes. I need to bring more data to train fit-for-purpose models. That expansion brings you into the watsonx portfolio, like watsonx.data to help you bring your enterprise data in or, say, watsonx Orchestrate to actually take these models and integrate them in your business processes. That's sort of the software at infrastructure stack. The last component of this, which is really powerful, is IBM Consulting, because if you say, "I like this strategy, I like this software, but I don't have the skills right now to move as fast as I want," IBM Consulting, they know this stack. They have expertise. They have demonstrated they can do this with a US Open use case. They're the accelerant in this, if you like this path and technology, to help you get there faster. That sort of is the AI. Our goals with it is being open, being flexible but being able to apply it to your business problems in enterprise as well. Okay, great. No, that's super helpful. Maybe just following up on that, can you just talk about IBM's decision to open source its code base and the Granite models that you have? Being in Red Hat, I'm a bit biased to this because our development model, we use open source for this. The reason we use open source is for innovation, and that extends to IBM's decision to open source their work around AI, both their large language models, Granite, as well as the techniques to be able to train them, which we call InstructLab. When we talk about innovation, for those that aren't familiar with open source, you can think of it as if you're able to bring expertise across the entire planet to contribute into something. We have seen over the past in operating systems and technologies like Kubernetes, if you get the open source model right, the innovation potential outstrips any other model. If you're a company, no matter how big you are, you're limited by the talent you can hire. When you get open source right, you have no limits on it. You can integrate with university talent, with other competitors. The way you get open source to work, you also have to be comfortable with the flexibility, which is why we picked the Apache License for that. Not everyone is going to want to build on your stack and give the software away. Allowing them to build proprietary solutions as well as contribute back things they want maintained is usually a really key component. That's what we looked at in both Granite and InstructLab was open sourcing this to drive the best innovation model we could, and then allowing flexibility so that you don't have to be open source yourself to participate with it. You have the flexibility in commercial use as well. Right. Yeah. No, it's obviously worked extremely well for Red Hat over a long period of time. How does IBM plan to monetize this AI strategy then? Yeah. We talked about a few different layers. The monetization strategy is slightly different in each. If we talk about the unit, the smallest thing, RHEL AI, RHEL AI is going to attach to GPUs that are shipped. If you look at RHEL in the early days, RHEL's job as an operating system was to give you the most bang for your buck out of a CPU that you bought, in the early days, it was Intel and AMD, to run any applications and run them efficiently. When you look at RHEL AI, it's very simple. It's the same strategy, but on GPU processors. It brings that independence of it runs on Nvidia, it runs on AMD, it runs on Gaudi with it. You have that unit, and our goal is install that layer, and it'll be the most efficient way to get your models running and training on GPUs. You move up one layer to OpenShift AI, that's not GPU tied, that is server tied, because now you're taking a bunch of those GPUs, and you're making them act as one thing. I'm saying model, which is if you are clustering, we want OpenShift AI to be the best bang for your buck in terms of that server-attached model to giving you efficiency. When you go up past that, you are now into different models because it's business integration with it. You have watsonx on other models in software. They're not directly GPU tied, but they can tie to business outcomes, user models on that. You obviously have infrastructure on Z. That is sort of that stack there from GPU attach to cluster attach to business outcome attached across the software stack. When you think about the contribution from that, how do you see that kind of maybe stack rank over time? Because obviously the infrastructure needs to get built, but then I can see how applications kind of supersede that over time. Just curious how you think about it. It's an interesting question. I don't think we know the answer at this point, but where we focus those when we look at RHEL AI, it is the how ubiquitous can we make this layer? This is why we've driven an OEM model. We've announced a partnership with Dell and with Lenovo to provide that reach. You sort of have the common basis similar to RHEL in our experience with server attach. When you get to OpenShift as a platform company, there'll always be other options, but we like the affinity of OpenShift AI and the apps that will consume those models in OpenShift. That sort of moves up the tier. Now to your point there, once you have that, the infrastructure is built. At some point, in the olden days, I would say this was like the middleware spend versus the infrastructure spend. I think we are going to see the same trajectory play out where the business integration and application will be incredibly valued. This is sort of where the watsonx plays on it. That will sort of be the next generation middleware in this space. We will continue to see infrastructure grow. We will watch that attach to it. All of these phases are pretty exciting. It is getting hybrid infrastructure out there, making customers effective with it, and then driving this into their business processes and outcomes with a new form of middleware. Yeah. No, that is a great analogy. Maybe switching gears a little bit, what is on everyone's minds is really trying to understand what are people really using GenAI for, and so it would be helpful to see what are you hearing from customers as it comes to adoption of GenAI? Yeah. Disclaimers being we focus on the enterprise space. The trends I will say will not reflect what maybe we see consumers do, but what we see in enterprise. The first trend, I think I am a technologist, but I think most people understand that this technology works because they see it on the consumer side. They see what frontier models can do. We also hear this trend of there is not a lot in production. We bucket those into sort of three challenges. There is a cost, a complexity, and then flexibility challenge. The cost is I made this work on a frontier model, but I have tremendous volume to bring to this use case, and I cannot pay a tremendous amount per query on it. It is internet optimization, business process optimization. Cost is a barrier. Complexity in the frontier models, they are trained on internet data. When you get into enterprise business processes, almost by definition, it is not in the public domain. It is private to you as a company. Your ability to teach these models your business, your knowledge, is another blocker. It is very hard to train frontier really large models. The third is flexibility, where not all use cases can be tethered to the cloud. If you can make it work in the cloud, but now you have to make it work in a car or a factory or retail outlet, it is a limiter to that. We are pretty passionate about we think small models, especially when optimization can solve the cost challenge. The InstructLab work we do is, I describe it as we are trying to bring put your knowledge in a model to mere mortals with it. We use fancy terms like synthetic data generation, but it's your content. Can you train a small model which we know works at this point? Flexibility is GPU flexibility, other hardware to allow you to have hybrid options. That's where we're excited. Where I see enterprises stuck is they can make the technology work, but they usually hit one or more of these barriers. That's where we try to intercept them to see if a smaller model, unique training in this area in a hybrid approach can get them past that. Is there a trough of disillusionment from the technology side? Would you say 2025 becomes the year of enterprise AI? I really think at this point, we have good infrastructure options, both public cloud and private. We know the technology works, to be honest, we know the technology works on smaller models. I think 2025 is that first year of real production wins for companies. I think that's critical in any technology adoption journey is the first wins in prod are usually the hardest, then you see growth build off that. I'm excited about next year because I think all the landscape is laid to be able to show customers this is possible, even at that first model, first train, first win use case. Yeah, I think 2025 will be an exciting year there. Yeah, no, looking forward to that too. What would you say are some of the industries or companies that have been early adopters with GenAI? Do you see any particular trends over there? Yeah, this is an interesting one because if you look at some industries, I think biotech as an example, they've been doing what we tend to call predictive AI or machine learning for a long time. They're coming into this space with a depth of data structuring already done. They're going to be looking at convolutional neural networks versus foundation models with precision on it. There are a few industries that have already been doing this on the last rev of technology and just approach it differently. It's positive, but they have a depth beyond what most enterprises do. I would say if you put biotech and industries like that to the side, most enterprises, the trend is, can I take my somewhat boring business processes that are written in standard operating procedures, or they're in people's heads, and can I automate an outcome with an AI model to help my teams get through more volume, get through it more accurately, reduce toil? That is a very hard challenge on it. The intranet optimization, they don't have the necessary value. When you get to business processes, they have a neat mix of large language models can read long documents well, but you need that training and precision to be able to automate it. That's where I think most companies are, is picking the business process, understanding if they have the documentation, understanding if they can tell it's right or wrong on the answer without a human looking at every single one of them and implementing that. That's where the bulk of customers that I work with, they're still in that early stage journey with it. Right. In terms of what would you say then are the most common use cases that are being explored and how does that change as companies get a little more mature down this path around implementing AI? I think there's a common set of use cases around support optimization. It varies by industry. We do this ourselves as well in Red Hat. I have to interact with my customers and I want to get them the solution to their problem as fast as possible. AI models are a great way to augment traditional support tiers in this. From the typical chatbot, but also in what we call the needle in a haystack searches, where these things can process a tremendous amount of information and find good matches for it. Support's a common use case. When we get into the GNA world, HR processes are another area where there's tremendous benefit from being standardized. Rules are complex around the globe. Using these to help point to better processes, better associate experience and allows HR to be more efficient with it. The third bucket I would say sort of falls in operations for it, where from procurement to looking at discounting rules, you have processes that are really specific to your company. We have been on an automation journey for a decade plus on this. In business automation and RPA, this is just the next evolution of can we do these things better from pretty unstructured documentation. Those three I see come up all the time with it. There are others, but those are probably some of the most common. What would you say in each of these cases, Matt, is kind of the ROI, right? Part of what, at least from an investment community perspective, folks worry about is there's a ton of investment being made, but what is exactly the ROI of this and how sustainable is it? I'm sure you know pretty deep in the weeds how that is, but what can you share with us about maybe each of these different use cases that you just spoke about? What kind of ROI people should expect and does that actually improve over time? I think one, it will differ whether companies are growing or whether they're just optimizing in a flat market. In general, the trend that we look at is the work we do, the cost of that query has to be lower than the outcome that I get from it. If you say in support, "I'm going to answer a lot of customers' questions better, but I'm going to pay millions upon millions of dollars in doing it," it might be more effective you stay in your traditional support structure. It serves customers well today, it's more effective cost. This is why we're such big believers in the first premise of ours, which is small models, and then also being able to run them in the most applicable thing to us because I think the best way to get a return on investment is to keep the unit costs extremely small in these areas. If you can be efficient at the questions you ask and push those down from dollars to pennies on it, your return is going to be in areas like I track in Red Hat, which is basically call optimization. How many customers asked a question on our website and didn't put in a ticket and talk to a human? That's a very known cost for us, and I'm able to model the GPU cost of giving them better matches versus the cost of a person to go through that themselves. I think at the core, in any of these use cases, it is pushing the unit cost as small as you can get it. In our world, that is a combination of starting with the smallest model possible. When you're working with small models, you have to train them on your knowledge or they won't be precise enough. Basically being able to run those on the infrastructure that gives you the best advantage. Might be cloud, it might be private, it might be at the edge. That gives you the optionality on it. I think that's where a lot of customers have been stuck, too, is they know it can work, but they don't know that the ROI is there. We're trying to bridge that to say, we can take the same premise where you know it works, and we can get the unit cost small enough that your return will be close to guaranteed using this. That's our goal. Yeah. Okay. That makes a lot of sense. You recently announced the intention to acquire Neural Magic. Can you just talk about the strategic rationale there and how it fits within your broader AI strategy? Yeah, absolutely. There are a couple key tenets of Neural Magic. First, they're very comfortable. They work in an open-source development model as well. It's an easy strategic fit for us. We know the team, we know where they work on it. They have tremendous influence over impactful communities like vLLM. If you think about this of where it fits in, I'll use the RHEL AI bucket for this. The model, we focus on Granite right now in our work at IBM Research. InstructLab was invented in IBM Research. We sort of refine and carry that out. Then we talk about, you have to run these models with an efficiency that is your best unit cost possibility. That's where Neural Magic fits for us. We're able to do this today, Neural Magic brings tremendous depth from being able to use technologies like sparsification to allow models to run on CPU architectures instead of just GPU. They're able to shrink models so that you can start with a 70 billion parameter model and shrink it to a 7 billion parameter model while losing very little. Then when we talk about just raw inference, that's asking the model a question they can get the most out of an Nvidia GPU or an AMD GPU or a Gaudi 3 GPU. That combination, that's where they fit in that stack. RHEL AI is the central piece to OpenShift AI and watsonx on that for running. Really excited to have them on board. That's an exciting fit to the portfolio. No, that's great. If we just go back a second to think about how you view the broader competitive landscape and maybe just like when you talk to customers, what is the most common maybe stack that you're seeing from an infrastructure layer up to application layers? Where do you think IBM has the most opportunity? The stack, what I would say is the most common starting point, the only real commonality is a lot of customers start with foundation models. When I say foundation models, think GPT-4 or Gemini or maybe Claude in this. These are the trillion parameter plus models on it. This stack above that of how they try to integrate it, one, it's pretty nascent. It's fragmented because they usually get stalled down that path for the cost of complexity of training or flexibility of, I know the technology works, but I can't run it. To be honest, I think the best IBM opportunity is can we bridge customers from the, I know this works. It actually works great in some domains. If I can be tethered to the cloud, if I have lower volumes, I can make it work. Being able to complement that when you're stuck, what options do you have? Which small models do you pick? Who can train them? You made the comment earlier of like, well, we need the infrastructure to be built. This is the missing piece of that infrastructure. I think if we're able to build that, so you have a fit for foundation models in some cases, but then you are also unlocked in your use of fit-for-purpose models commercially, I think the middleware journey on this and the power that watsonx brings to being able to integrate these with business processes, bringing data in, is a really strong advantage for IBM because they've built a great suite there. We know this works in companies like the NatWest example or US Open that have really adopted this at scale. It's about building out enough infrastructure so that we can bring that advantage to bear. I think that'll be the focus over the next couple of years to go through that. Maybe another adjacent question is just around semiconductors, right? Like IBM obviously has a long history there and obviously, like in the optimized for Z, a lot of capabilities for running AI-based workloads. I'm kind of curious as to how you think about the longer-term view of whether you would be using more third-party kind of chips. Would you continue down this road of having more custom silicon? How do you think about that? Does it matter? I think it does. When we talk about optimizing unit prices and costs, infrastructure is a huge component of that. You see custom processors being built basically by all the clouds at this point. I actually really get excited. You mentioned IBM has a long history. If you look at Albany and the semiconductor partnerships they've done, dating back to Samsung with memory, to the options they have of NorthPole collaboration in AI processing going forward, especially when you get to Edge in use cases where you can't run off traditional hardware combinations, you might run IT off. It is a great tool in IBM's toolbox to be able to bring that they've known how to do this for a long time. The nice affinity with Red Hat is because we already work closely with IBM on IBM Z. We're going to have enablement for Telum and for Spyre and for NorthPole. If you look at Red Hat's strategy of taking our operating system and pushing it to the Edge, whether that is OpenShift running on the International Space Station or whether it's our in-vehicle operating system suite, there's a natural synergy there for customers that do invest in the semi side to build their own optimized infrastructure. Knowing that they will have a software stack that can run on that, I think, is a really powerful combination. That's earlier in the market, but I do think that's a relevant strategic asset to have, especially as things evolve. That makes sense. As you think about just within the Red Hat framework, if you think about where the infrastructure is kind of maybe not just infrastructure, but where you're seeing workloads within public cloud versus on-premise, how would you say has that-- The last kind of 15 years have been a big march towards public cloud, but I think that's decelerating fairly strongly now, at least in growth rate terms. Obviously, we've gone from zero to $400 billion of spend on public cloud in 15 years. When we think about how you are seeing workload movement, because you have that visibility in some ways, how would you characterize at the high level, what is actually happening with the use of multi-cloud architectures and what kind of dynamics could you share over there? Yeah. To your point, there's been a hard march to public cloud. I usually tell people if I went back five years, seven years, most customers would say, "I'm not focused on hybrid because I'm going to get everything to one cloud. That's my goal with it." That goal wasn't realized by a lot of customers, whether it was they continued to have on-prem infrastructure, whether it was other clouds developed new things that they had to participate in, so now they were multi-cloud, or they're big, and they acquired companies that had other footprints. I would say that trend plus sovereign cloud, geopolitics, there are pressures to where you have to be comfortable with hybrid at this point. You can't only have one answer if you're an enterprise or a global enterprise today. This is, I would say, directly attributed to why we've seen such strength in platforms like RHEL and OpenShift. The minute you have three clouds or two clouds in on-prem or a sovereign cloud and two public clouds, you need some commonality across them to get the best use. That has been a very strong trend. To be honest, I think AI only amplifies this because you have the problem with training, where a lot of the enterprise data still sits on-prem, and you have the problem of deployment, where if my end destination is a car or a factory or a retail branch, Edge really becomes part of my core strategy at that point. That I think you're right on this trend. We have seen the acceptance of hybrid really strong over the last couple of years. I think the work we've done with Ansible and the upcoming acquisition of HashiCorp gives us a real management strength in that space. AI is probably even more hybrid than the core traditional applications, because by design, you're training it on data that is yours and tends to be behind the public cloud move curve, and you are deploying it as close to your customers as possible for that unit cost efficiency. That brings us into some really exciting strategic conversations of how hybrid becomes an advantage for you, not just a reality that was thrust upon you because you couldn't get to a single cloud with it. Yeah. No, that's great. I know we're out of time, I'm going to try to sneak in two questions as one really quickly, if I can. Sure. One is just what's your longer-term view on training versus inference in the sense that, as these enterprises train based on their proprietary data, how much of workload or volume do you think ultimately or time spent would be on training versus inference in the enterprise world? Maybe just to wrap it up, what are you most excited about IBM's positioning here in AI? Thank you so much again for taking the time. No problem. Training versus inference. I do think in the next couple of years, we'll see the inference load eclipse training on it. At the same time, to be honest, we try to push training down to. There are multiple types of training. There's build a new large language model, which requires a supercomputing data center. When we look at RHEL AI, we train on a four-way or eight-way single machine. We try to keep training costs fundamentally lower than the mean aspect of, it's going to take us $1 billion to train. We are pushing that down. To your point of 2025 is the year for production workloads. By design, you want to be doing more inference. You want to be asking the model more questions than you are feeding it with data. That's something we watch there, in terms of making that shift. What I'm most excited about for IBM's positioning is when you have any technology disruption, I think the most challenging thing is can you make it work for enterprises? There are 1,000 things that have made really exciting technology to me not be able to make it forward in the industry, whether it's from cost perspectives, application of it, the ability to structure it, trust it. IBM has the entire portfolio from a software stack that is flexible and gives you those starting points, optionality on infrastructure, all the way to the skills that can help you get there faster. I think that's a really exciting position to be in to help bring customers to a point of success with the technology, not just stall out on one of those constraints and get frustrated. Yeah, we'll see what we can do in the coming years here. Amazing. Well, thank you so much, Matt. It was a pleasure to have you here and join us at our A.I. conference. We look forward to catching up with you soon again. Thanks everyone for joining us today. If you have any questions, please shoot us an email, we'll try our best to get it answered. Thank you again. Thanks, Matt. Thanks, y'all.