Good morning, everybody. I'm Billy Cladek, SVP of Firework. It's my pleasure to welcome you to How to Better Love Your Customer by Firework and Victoria's Secret & Co. Today, we have Chris Rupp, Chief Customer Officer of Victoria's Secret & Co., and Vincent Yang, CEO of Firework. They'll explore omni-channel success, the impact of evaluating customer experiences, and Chris's dedication to infusing AI into Victoria's Secret & Co.'s roadmap. Please join me in welcoming Chris Rupp and Vincent Yang. Thank you. All righty, so, good morning, everyone. Thank you for being here with us, and I think we'll get right into the content. So, Chris, so we got a lot of questions to you. Great. So the first one I'd love to dive in is: Love the Customer is one of your core values in Victoria's Secret. So can you tell us why it's so important and what it means for the business? Absolutely. Love the Customer is one of our core values at Victoria's Secret because we think of it as one of the four most important things that we do, and something that drives all of the decisions that we make. Our value system really sets up for us what is the key focus of what executives at our company seek to deliver. And, you know, that's probably not unlike most of the people in this room. Retail is a business mostly dedicated on how you can connect consumers to your products. So I think there's probably a lot of people in this audience that identify with that. So, you know, a lot of people talk about customer matters, et cetera. So I'm very keen to know, so how do you approach customer relationships? Mm. And also, has it changed than before? And how do you approach it differently than many other retailers? Yeah. I think the most important thing about the way you approach a customer relationship is how you seek to get to know who your customer is. And, you know, if you, if you try and guess, if you try to go into a store and just watch what customers are doing, you're going to get it sort of right. But I think if you pair all of those great anecdotes you get by watching consumer behaviors, talking to your friends, and learning that way, if you pair all of that information with the great data that you can get on how your consumers are behaving, I think those two things woven together make a great picture of what customers want and how they're behaving. The more you know about that, the better chance you have of influencing their behavior in order to change it just enough to perhaps buy one of your products. So, you know, it's really about, like, how are you bringing together that customer need and your products, playing the role of matchmaker? Very good. So I'm sure everybody here talk a lot about Omni-channel, whether it's, say, omni-channel commerce or unified commerce. So could you tell us a little bit about how you are approaching Omni-channels? Yeah. Omni-channel, I think, is probably the answer to a question, and the question is: how are you gonna provide customers with convenience? I think, everybody out there is looking for an easy button when they go shopping. I think Omni-channel just means you can buy what you want, when, and where you want. What's great about being in a job like mine is that we can constantly look for all the new technologies out there that can help us do this in new ways. Omni-channel is, of course, anything from Buy Online, Pickup In-Store, or Buy Online, Return In-Store. It even means going on Google, for example, to find what store you might wanna go to. Anything that moves you back between online and offline commerce is really Omni-channel. But I think it's really when a marketer thinks about doing a great job, they think about being contextual and relevant for a consumer at the moment they're interested in a product and just being right there for the customer. And I think Omni-channel allows you to do that. You can reach your customer, whether it's in the shopping mall, on Google, in social media, any one of those locations. But then as they're coming through the shopping funnel, it doesn't have to be linear. You know, they may start online and realize that they really wanna get a personal fitting for a bra, for example. And then you can have an Omni-channel experience that moves the customer back and forth between online and in-store to serve their different needs. Also, I noticed that you guys have an app. So could you talk a little bit... Is that part of the Omni-channel shopping experience? Absolutely. Yes. You know, our app is something that our best customers tend to download and use often. So the way I've been thinking about this is, what your app experience is really your best customer experience, and so you wanna make sure that your app has, I'll say, all the bells and whistles. But what I really mean is that it's the most convenient experience for those who wanna get something done fast. Our bOst customers shop with us quite often. But it's also a great place to build browsing experiences, where the customer can have fun learning about your products and experiencing all the different things you have to offer. So I think the app is a place where you wanna build your best experience and really experiment with a whole lot more, ways to build relationships with customers. I see. I see. So just let's shift out a little bit from omni-channel. Let's. One thing that I'm sure everybody, if they look at your LinkedIn, will ask the same question. So your previous job was the CDO for Albertsons Safeway- Yes. which is food. Yeah. What prompted you to shift from food industry to Victoria's Secret? Yeah, well, you know, I have worked in a few different industries along the way. I've worked in gaming, electronics, home fashions, tractors, food, as you mentioned, and now bras. So, you know, I remember at the very beginning of my career, somebody saying, you know, "Would you ever move out of a category you didn't love?" Actually, what I love is the spirit of discovery when you go into a new category and you learn how a customer shops, and it's very different between a tractor and a bra. Mm. But actually, the discovery of that shopping—this discovery of how customers shop that I think is so exciting. And so, so I've been fortunate to work in a few different places, but what's really interesting is when you go to one place and you learn. So in the food service business, recipes are a big thing, and using video in that experience to help a customer shop for food is quite natural. You see it all the time online. But then when you come to a place that sells bras, you start to think to yourself: "Wait, is there a place for video in this shopping experience?" And it's really not the same thing at all. We're not gonna put recipes on the Victoria's Secret site. But, you know, what we do need to look for is ways that the digital shopping experience becomes more personal and emotional, the way that in-store shopping feels for a customer. The digital experience can be quite flat. But I think starting to bring in elements of things that feel more personal and emotional, like they do in a physical store, really start to put the digital shopping experience into the emotional arena instead of a transactional arena. So, talking about that point, you know, we were having a discussion earlier. I think what's very interesting is you spend a lot of time observing how people are doing it in the physical store. Yes. Could you tell us more about- because I find that very interesting because you spend a lot of time on digital. We talked a lot about it, but many executives I talk to, they focus only on digital. Mm. They don't focus too much time on physical. Right. Right. So, you know, I, I love the digital experience because I love bringing technology to bear on a consumer experience, and putting those things together is so much fun. But, you know, I think, I would say this, and probably most of the people in this room, have learned this lesson along the way as well. You can't bring technology to your experience for the purpose of technology's sake. You can't just say, "I got to have AI. We're putting AI on my site." That doesn't make any sense. What you wanna do is figure out what customer need you're serving and how the technology plugs into that perfectly. And, my own practice for this, which I just consider to be fun, is to go to a Victoria's Secret store- Mm ... and to watch how our sales associates are interacting with customers. You know, I like to think about it this way. One of the reasons I went to work at Victoria's Secret is I've been a fan of the products for decades, and they have such a great selling experience in the store. And so when you go to watch what's happening between that associate and that customer, even from the moment the customer walks in the door, you know, the sales associate is looking to take cues from what she's doing or what she's wearing or who she's with to determine what her objective is for the day, and they'll even greet her and ask her questions about her objective for the day. And that helps qualify her so they can figure out, is she here shopping for a bra? Because if she is, 80% of the women shopping for a bra are wearing the wrong size. So we like to start by offering her a chance to try a new size and to help her get fitted. So wait a minute, when you go online to shop for a bra, is somebody asking you if you want a fitting? No, there's nobody there to ask you if you want a fitting. In fact, we have created a digital fit solution that we're really excited about, but it's still flat on a screen that you discover by clicking and going somewhere. I think using more of the technology that's available to us to have that leap off the screen, to invite you in and to help you discover what it is you're there to do today, will help continue to transition the digital experience to be a more emotional one. Got it. Got it. I think it's what we similar talked about. Digital feels more like a read-only mode. It's more like, as a consumer, I can only read, but physical is more like we can have a dialogue, we can have a conversation. Right. So, feel like we thinking about how do we bring those two experience all together? Talking about technologies, I think from a lot of your former experience, you're very good at being a pioneer about bringing technologies- Thank you ... into digital experience. So do you have a methodology or philosophy there? And I'm sure you experiment so many new techs. What is the right way to adopt a new tech? Mm. And what about some of the failures, the wrong examples you implemented? Yeah. I would say, I'll just go down the path that I was going down before because, you know, it starts with understanding what is the consumer need that you want to fill, and then finding the best technology that will do that job. So, I'll give you another example of watching customers as they come in through the front door. Another example is a customer will come into a Victoria's Secret looking for a bra that she bought before that she loves. Now, for anyone that has ever bought a bra, let's see if you remember the specific name of the style of the bra that you bought. Nobody does. Nobody does, including me. And so when you come into the front of the store, a sales associate will say, "What are you here to do today?" If a customer says, "I have a bra that I love. I just wanna buy another one," what I watch the sales associates doing is saying, "Oh, are you wearing that bra right now?" If she is, she asks her to pull her strap out. She pulls the strap out, and the sales associate's like, "Oh, I see. That's Body by Victoria. Let me take you right back over there to the Body by Victoria section."... and I thought: "Oh, my gosh! We could do this online. Hmm. I know. I'm gonna get a picture of every Victoria's Secret bra ever made, and I'll take pictures of the straps and have the customer choose her strap." What a terrible idea. No one's going to do that. No, but you ideate through that, and you come to the conclusion, you know, visual search is a perfect example of a technology that can solve this question. So now we have a feature that a woman can take a picture of any bra that she loves, and we will return back to her everything in our catalog that looks like that bra, which is a smaller selection of things in the perfect size for her. Wow, that takes the heartache out of shopping. That's very interesting about the visual search. Mm-hmm. I'm sure a lot of people here are also trying to learn. You know, is there any other formats of experience that you use or new technology that you're willing to share that, you know, to help you to connect better with your consumers? You talk a lot about sales associate, whether offline, online, anything you can share today. Absolutely. You know, there's probably just such a number of things that are important. For example, I know everyone is here at this conference to talk about AI. That has been such a quickly rising technology. I think everybody would say, "Wait, a year ago, we weren't here talking about that," but here we are today. So I think I'll maybe leave the conversation about AI aside for a moment because, you know, I do wanna talk about Firework and some of the great technology that we've worked on with you. But, you know, the video technologies that we've been working on with you, I think, really humanize the experience and bring a whole different dimension to the website. So I think that's a really important example of how you're taking a new technology, like short-term, short-form video, and using that as a way to create a better connection with the customer. And so this is a place where you start to experiment with, where can you put this on your site? What kind of content do you put on it? You know, at what moment is it right for you to interact with a customer? We started on our homepage, and it felt like more like a banner ad or something like... It didn't feel natural or interactive. But as we've moved this to new locations, as we think about interacting with our customers on a detail page when they're already very interested in a specific product, this is where more engagement happens because now they're really vested in the products they wanna learn about, particularly if it's from maybe an influencer or someone else whose opinion they care about. So now it's a matter of we've made the experience come off the page and to be more interactive and with richer content, and so now we're just trying to figure out the right places to put that right content to make a better connection and more conversion. Yeah, this is something that I learned as well after, you know, the working on this sector for a long time. A lot of people do videos, but we all think, like, which is formerly talked about, it's for branding, it's for content on a website, but that's all wrong. The real reality of video is it contains a human connections. Mm. It contains conversations, as you were leaning towards. You know, I think this is a very good takeaway for everybody, that video is not just for content, it's for conversations. So, you talked a little about AIs, about 50 questions about AI to ask me after this. Oh, good. Now, let's talk about that, for AI. Yes. Everybody talk about AIs. What's your take about AIs? Hmm. How are you thinking about for Victoria's Secret, you can, you know, apply those in your business? Yeah. We've got a lot of ideas. I'm sure everybody has a lot of ideas. Well, one of the things I'm interested in exploring very early on is how can you use AI to make the in-store shopping experience better? You know, the reason I'm thinking that way is that when you think about working in a retail store environment, you have so many team members that are new, and our assortment is so complex. 50,000 SKUs in a store, 50,000 SKUs in a store. Lots of sizes, lots of colors, lots of padding levels, and shapes, and all kinds of things. How do you make the right matchmaking experience between those products and customers? You know, I think it comes down to, you know, being able to use all of the data and information that you can bring to bear on the situation. So think about, for AI, for example, then, if a sales associate could just use the mobile phone in the store that, you know, she may already be using for mobile POS or something else, but, you know, if a customer comes in and is looking for a demi-padded bra in black with lace in size 36DDD, wow, do you know how many drawers in the store she has to go to, to try and figure that out? If she could just ask Victoria, "Do we have a product like that for our customers?" And the AI could return: "Why, yes, we do. There are three of them. This collection, this collection, this collection." And it's much easier then for the sales associate who's new on the floor that day to make a great connection with the customer. So you think about how do you put together a solution that will help store sales associates that over time then, the sales associates can actually help train the model so it answers the questions perfectly, and then maybe it could be customer-facing or at least some element of it. So we're thinking more along those lines early on in terms of the large language models and how we'll use those. But, you know, we're already using AI, for example, in email and other technologies that we're using in order to build marketing experiences and other things. And so, look-... Everybody says this, you know, AI is going to be as big as the Internet is. Boy, we are just in the opening innings, and so I think by the time we're sitting here next year, we will have imagined a whole lot of other things we can do with the technology. Also, a lot of people combining AI and personalizations together. So I'm just curious to that, how was your take about personalization now with AI there? Are you thinking about doing things differently? Yes. So, you know, the way I think about personalization is similar to how you're thinking about it. I love what you say about video being really a two-way conversation for customers, because content has always been sort of directionally just one way. And this whole conversational approach to retail, you know, thinking about it, the conversion in the retail store is way higher. They say seven times higher in a retail store than it is online. And you've made the observation before, you know, in the store, it's a conversation. Online, it's one way. What if we could take our 500 million visits a year for Victoria's Secret and convert just a little more? It's incredibly impactful, and humans are used to working this way as a conversation, right? So, I think it's really important that we're looking for technologies that bring more of that human and conversational and personal element to the shopping experience. And then the other aspect of personalization, which is: how do we know you and return things that are just right for you? I love thinking about how it's not just one product at one moment, but for example, you know, our customers will say, "Festival season is coming up, and I want to decide what I'm wearing for festival season." Instead of saying, "Do you have a black corset?" which is sort of a search response. What you'd love to be able to help someone through is, "Here's all of the outfitting solutions we have for festival." And I can tell you which one of those two things would sell a whole lot more if you have the right things for the right person at the right moment. Mm-hmm. It's really about thinking about the customer as a whole and what she's trying to get done. Yeah, I think you use a lot of words about customer. It's customer-centric, not the brand-centric. Mm. And then we remember, we talked about the first letter of personalization that people didn't focus on is person, right? We tend to focus a lot about personalization, but they don't realize they start with a person. So I'm sure everybody here about Chris talk about is focused on a customer. That can have a different perspective on personalization. So, final question here is, I'm sure in the audience, a lot of people are trying to learn from you as another retailer. A lot of people are tech solution vendor, wanting to sell products to you. So in your perspective, what's your view about expanding your ecosystem? Is that how do you choose and find different partners to work with? Oh, yeah. Well, I guess maybe a good way to think about it is looking for a partner where our goals are aligned, you know, and that could be because we are both interested in figuring out how to make a digital selling experience come off the page and grab a customer in a different way than what it does today. And then it's really exciting when a partner has lots of great ideas. We know our customer well, and those things come together in a great new experience. So I think it's really about goal alignment, both from a customer experience perspective and then also, of course, not forgetting all of the difficult stuff on the back end with the technology and making sure all that works. But, you know, just finding great partners that are trying to do the same thing, you know, make the digital experience a better one. That's great. I think that will wrap up all of our questions. We still get about seven minutes. Great. Would you be open we take some questions? Yeah, that'd be great. All right. Anyone has any questions for Chris? Feel free. We got the mic around. We can pass. Any questions, whether it's around shopping experience, AIs, customer-centric experience. Over here, there's- Yeah, really. There we go. Hi, Chris. I'm SJ from AllSaints. Thank you so much. Really insightful session. What I wanted to ask is, we're very interested in leveraging generative AI, bringing that in-store sort of knowledge and personalization experience online. What advice would you give us sort of as we sort of, you know, work towards that? Well, I'll tell you, I'm as new at that as everybody else is, but, I guess what we're thinking about, because we've been out talking to everybody about this, is if you're going to start with online, you just need to think about how you're training the model to be great so that you don't end up feeling like the model is training on customers and not so great. And what's the difference? You know, at what point is it not so great versus great? You know, I think every company has to decide what great is or not. But I think, you know, when you launch in a large language model, in generative AI, you're gonna find that 80% or 90% of what you do looks great, I don't know, I'll say out of the box, but then there's all of these edge cases that you're training it on, and I think that's just the place to be careful if you're going straight to the digital experience. That's brilliant. Thank you so much. Mm-hmm. Coming on over. Hold on. ... Hi, my name is Ivana Hamidi. I'm right here. Oh, in the middle. Yep. Hello. Hi. With Accenture. You mentioned the experience of coming off the page in a digital, your digital landscape. Have you already been testing this, like, trying to- Oh, yes. Get proof of concept? And, and in the testing, obviously, like, have you experienced some positives, negatives? Like, where are you in that testing phase? Yes. So the experiences that I was referring to have come from our testing. So, when we have been using video in our shopping experiences, you know, we've been looking at everything from, like, what are they doing in China? To what are other retailers doing? But what are other retailers doing in-store, online, you know, whether it's the home improvement space or... And so there's lots and lots of cues to take to figure out what experiments you should do. If the video feels like it's a billboard or out of place or not part of a story, it's not working for us. If a video looks like it's part of a story and helps answer a specific question that's natural at that moment, the customer dives into it and actually spends quite a bit of time there. So, we're looking for those places and exactly what the right content is. Because now, once you discover a great location, now you've got to try and scale up on the content side of it. So that's where we're at with it. Hello. Right here, on this side. Oh, hi. Hi. I'm Vinutha. I'm from India, Capgemini. You know, you spoke of Generative AI, you spoke of digital experiences, and this is the best space probably to ask this question: Have you thought of biases and how a Generative AI solution could deal with biases, especially the product that you are in, where even human biases are so strong and so difficult to contend with, right? Right. Yes, exactly, Capgemini, and we do work with you in India, so nice to meet you. Yes. So, you know, we're worried about that, and we're just learning about that along with everybody else. But, you know, I think whenever you're worried about something like that, it's sort of a you get what you measure. And so what you're gonna have to do is measure the outcomes that you're getting from the technology and how different they might be from outcomes that you would desire or be proud of. And so the more you measure, it's like I know it's going to be in those edge cases that we don't like the outputs that we're getting, that we're gonna have to train the models to do something else. And that's really I can only think about this philosophically at this moment because we're just kind of going into it ourselves. But, you know, I think it's a great question for everybody. We're all gonna have to work on that together. Yeah. Yeah. Hi. Hi. Bill Scott, I'm with VTAP. Can you tell me how does your loyalty program- Oh! fit into or complement some of the other initiatives that you've talked about today? Oh, gosh, I'm glad you asked that. We, I will say our company was late to the party with a loyalty program because we had such a great credit card program for so many years that we built rewards into, that we felt that that was really the way we engaged with our core customer. But we came to realize that there were so many great benefits for having a relationship with customers outside of the credit card, that this year we launched a loyalty program. And, we launched it in June. We have 24 million+ signed-up members just in the six months that we've had it live, which I think attests to the power of the brand drawing customers in. And, so many customers that have shopped with us for a lot of years are just signing up for it very quickly. Now, what does it do for us? Well, you know, we can start thinking about how we use our loyalty program and now the email address and the other methods of communication to be very targeted about direct relationships with those who have signed up with us, and, you know, predicting what behavior we think is most natural that they might demonstrate next. I'll give you an example of this. When customers start buying with us to begin with, they're almost always buying mist or panties. They buy something that's less than $10 in our last decile of purchases. And so what we think about is: How will we walk them up the loyalty ladder to... Can we get them to try one bra? Just try a bra with us and see if you love it. So using then, this information we get about our customers through the loyalty program, to then guide them on shopping experiences for things that we think that they're going to like based on their behaviors and their, you know, psychographic segments and all those kind of things. So, so, you know, at our last decile, people buy panties and a bra. Our top decile, they buy everything in the store. So we're just gonna figure out how we deepen our relationship with them over time based on, you know, this loyalty program that we'll be able to communicate with them, actually as much as they want to, depending on how much they buy from us. All righty, Chris, I think we've run out of time. Great. Thank you, everyone, for listening. Thank you. So please give Chris Rupp applause. Thank you. Hello, everyone. I'm Amy Eschliman. I'm Managing Director of Retail Strategy for Google Cloud. We have an amazing group of panelists with us today. I'm gonna go through a little bit of context about what we're gonna be talking about, and then we'll get right into the questions. So to start, I want to address the elephant in the room, generative AI. You might have heard of it while you're wandering the floor of NRF. Retail has been through this before. These massive transformations is nothing new to retail. The Internet is a great example, changed the way we shop. Mobile phones, same thing, changed where and how we shop. And now we're generative AI, which has the capability of transforming everything from the customer experience to the associate experience. It's got tremendous potential, and it's a really exciting time in retail for because of this technology. possibilities of generative AI in retail are really everywhere. It's the ability to synthesize and analyze information that we did not have the ability to do before. So think about handwritten forms coming from the stores, product imagery. We're able to use that unstructured data in a way that we, again, have not been able to do before. Then the ability to generate content. Massive implications in the world of retail marketing when you think about not only creating campaigns, but then also being able to iterate on those campaigns, improve them with the insights that you get. And then, creating and automating processes. Again, every part of the retail value chain has the ability to have generative AI affect and really change the way that we do things. And then finally, engaging through conversations. We're not in the same world before with robotic chatbots. These can be incredibly natural conversations that make a customer feel like they are having a very personalized, seamless experience. So the possibilities are really endless. This is some use cases. It's certainly not an exhaustive list of use cases, but if you look behind me, these are the use cases that we at Google feel like really have the ability to drive more immediate value within retail. And within retail, time to value has always mattered. When we talk to our retail customers and look at the retail industry, 2024 is going to be a year of action. Of the survey, we had 81% of retail decision makers say that they feel urgency to adopt generative AI, and 72%, a pretty massive number, are looking at implementing it in 2024. So the time is now. When we ask how they plan on using it, you can see a variety of use cases, everything from the customer experience to the associate experience and making processes more effective. So customer service, the primary use case mentioned. Product descriptions, how do you speed the categorization of product, the, the description of a product? How do you personalize that to different sectors and segments? Creative work is a huge opportunity. Conversational commerce was listed. And then also interesting, as almost equal amount of, of, survey respondents mentioned the associate experience as well. So not just the customer experience, but the associate experience. I know many of you in the room are at different places in your AI journey. And I think one thing stands true, no matter what type of AI we're talking about, strong data foundations are absolutely critical. Before you can move on to those really innovative use cases, you need to have that data behind you in order to use it in an effective way. So, with that, I'm gonna actually turn it over to our panelists, which I am really excited to speak to. I'm gonna have each of you introduce yourselves. Murali, I'll start with you, and we'll go down the line. Okay. My name is Murali Sundararajan. I'm the Chief Information Officer at Victoria's Secret. Hi, my name is Jessyn Katchera. I'm leading e-commerce for the group Carrefour and the innovation as well. Hey, my name is Chandu Nair, SVP of Technology for Data, AI, and Innovation at Lowe's, a home improvement company. Thank you guys very much for being here. So I'm gonna actually ask the first question of you, Murali. You at Victoria's Secret are leveraging AI and generative AI in lots of exciting ways. I'd love you to talk about how you're leveraging generative AI in the digital commerce space. Can you share more details? Yeah, I think like any other retailer, Victoria's Secret is focused on going through the journey with the AI and generative AI. So typically, we focus on three categories. One is the customer experience for the customers and the associate experience, both in the store as well as for the corporate office, and improving operational efficiency across various functional areas. So the use case is the one we started with last year, was the focus on a customer experience for the digital one. We wanted to bring the customer experience, what the customer typically go through in the store, and we want to mimic the same operations in the digital because that's what we were focused on, how we can able to fill the gap. The example is when the customer walk into the store, and then if she's interested in buying a bra, if the bra, the one she has used it for five years before, she wanted to come back and say: "I want to buy the same bra. Can you help me out?" So the associate who is in the store are experienced, they know what exactly the question they need to ask. They know how to exactly work through the customer to navigate exactly what she's looking for across all the 50,000 SKUs that we have. But if you want to simulate the same experience in the digital, we're just exploring how we can able to do. So that's when we had a conversation with Google, and Google was helping us to say: "Can we leverage the Vertex AI using the Vertex platform?" So we started the journey sometime in March. The use cases came in early in the March, and then we could able to work through the prototype. We put it in production in June of 2023. So the reason I'm mentioning the timeline is because this is how we can able to quickly do the prototype, see what the value is, and then put them into production, and then get some feedback from the customer. Because AI or generative AI, the feedback is very, very critical, and as, as you talked about, data is also critical. We need to understand what kind of the data we get, what kind of experience we are trying to solve. So this one, we could able to literally see. We, we solved the use case, and we can able to bring it to fruition pretty, very fast. And then that created a lot of momentum in terms of how do you build the use cases, and now we are focusing on a lot more on customer experience, a lot more on creating more productivity and efficiency for the associate, so that the associate can reduce their mundane work in the store- ... focus more with respect to the associate, for the customer experience, and we can able to improve their, the work output. Great, thank you. You guys, I don't know how you did it, but you sat in order of the questions I wanted to ask you. Jessyn, this next one's for you. I know Carrefour is doing some pretty amazing things with Generative AI, from marketing use cases to even HR use cases. For the purposes of today's conversation, I'd love to have you talk to us a bit about the marketing use cases you've started, and any results that you've seen thus far would be great. Yeah, and, just to play it back a little bit, I mean, similar to you, as soon as we heard about, you know, all the buzz that Gen AI was carrying it around, we knew we wanted to be part of that journey. Why? Not because it's fun or because it's sexy, but because we saw the potential it had to reinvent the way we operate, to transform our operations. And so the first thing we did to come up with, you know, what are those right use cases that we developed with Google for some of them, we started to talk with all of our operational teams to really understand what are the pain points today, what are the use cases we really need to solve, and where Gen AI can really play a role to transform those operations. By doing so, we realized there were three types of use cases that could be relevant. There are the everyday use cases, like all the low-value-added tasks that you can automate, that you can, you know, realize some small productivity gains. Think about, you know, sending minutes automatically, following up on actions, smart composing your emails. And for all of them, to be fair, that's not where we wanted to focus, because we thought there are a lot of brilliant players like Google and many of your competitors that will solve that for us. We just have to wait for the next release of those innovations in the tools we use on the day-to-day to bring that to the best for our employees. Then there was the second types of use cases, which was around: where can we get a competitive edge? Because we're reinventing part of the experience where Gen AI can help augment the way we serve our customers, by being more personal, by being more tailored, by better empowering our employees to do the right thing. And so that's one of the use case we developed there, is around, shopping assistant. And then the third thing, which I think is the truly crux of the issue for us, is: how can we reinvent end-to-end some big chunks of functions that we have in our business? And that's where marketing played, entered in our reflection. We realized that there are a lot of functions which you can really radically transform by using Gen AI to better empower your employees, to better serve our customers, to better propel our growth, and that's where we played in what we call the marketing studio. The goal there was really to think about: how do we use Gen AI to be smarter, faster at generating assets for our marketing campaigns? Whether it's audio, whether it's text, whether it's visual, these are some of the use cases we're exploring. Why does it matter? It depends on the companies. For us, it's really about accelerating the agility, the time to market. Instead of having to wait for many weeks to get a customer-ready product from a marketing standpoint, we can develop that in a matter of hours or a matter of days. Which means you can suddenly decide to expand and tailor the content you have to really fit the media you want to use, to start to deploy local nuances to the content you develop globally. Because instead of having to wait for weeks or months to deliver the one asset at a time, you can have done that in a few days. So you can start to really expand the reach and the personalization of your marketing campaign. And that's what we're really proud about. And some of the use cases we developed with Google on that front and that are displayed in the booth are really exciting for us. Thank you. I've been in retail for 20 years, and I think we've been talking about personalization all that time. It's really exciting to see some of these use cases come to life because they truly change the way that we deliver personalized content. Okay, Chandu, I'm going to send this to you next. You have been an early adopter of Google Retail Search. I know you're using generative AI on some merchandising use cases. I'd love to hear your perspective on kind of what how it's going, any results that you've seen thus far. Sure. Thanks, Amy. Absolutely. But I'll start with kind of the common theme that's there. I think, you know, I think generative AI for me, is not kind of the differentiator by itself. Your data is, right? So that's, you know, I think what... You know, how I think of generative AI as to how it'll unlock value for, for retail especially, is, you know, you got to think about now data that we could not process before. Think about all of your operating procedure documents, the PDFs and things like that, that is out there, that is typically hard for you to process and understand and really kind of build out experiences for your associates, for your customers, or kind of help improve in general, your productivity as a company, right? So, I think that's kind of the broader premise by which we started looking across the entire, you know, retail value chain, and see where, you know, generative AI use case could apply. So, and merchandising was definitely one of the areas that we picked on. And the reason why I picked on is, if you look at, you know, home improvement, we sell anything from appliances, you know, you sell appliances very differently, from lumber to paint. I mean, people curate. I mean, if you're a merchant, you curate those assortment very, very differently compared to, like, you know, your next category of assortment, right? That need, that you need to bring to the table, right? So what we started was with very basic stuff. We noticed a growing problem that we had was when it come to product onboarding, like, you know, onboarding a SKU or, you know, onboarding a vendor, it was a laborious process, very manual process. A lot of the data quality that is involved in capturing them upfront, it was not there. So it was, you know, kind of exposed to the customer or the store associate to explain that. It was not that we never tried to solve that problem. We have applied a lot of AI models before to solve that problem, but now with generative AI, we, you know, almost saw about 60% reduction in, you know, like, the amount of manual labor that is needed to upload these, you know, initial product, you know, product descriptions. You know, think about all of the attachments, the images, all of that, that needs to go into that, right? So that's one part. Then we thought about, okay, how do we productize this? And that's a theme that we were working on because, you know, any other thing. And it does require a different way to think about productizing generative AI products. But now think about how you shop, you know, for a vanity you know, in the bathroom, right? So, if you go to a store, you kinda see a curation of things that actually tells you how it could look like. The backsplash, the faucet, I mean, everything that is in there. How do you kinda bring that? How do you curate that in a very dynamic way? You know, usually it is all very manual, where somebody is, you know, a marketer or merchant is working together to kinda pull all of that together. But with Generative AI, you can actually curate that with a lot of. Not just with Generative AI, I should say. We use Generative AI alongside a lot of our model because it has to understand our data, and we bring that together to curate what that variety could look like. What are the options, right? So and we are in that process to kind of evolve, and that would be a customer-facing experience. So we have both associate-facing experiences, like helping onboard, you know, products with better data quality, better images, better graphics to kind of help sell, help kind of, you know, help tell the message to the customer the right way. At the same time, we also wanna kind of have the right experiences to our customers with curated collections that can actually enable sales digitally or in store, right? You know, in store, our associates are doing that. So that's been the path. Now, I'll kinda share some of the learnings, though, as far as to how we applied it. To me, like, I think large language models, all the buzz around generative AI, large language models are in a way commoditized. I mean, there is a lot of options, open source or otherwise, to go and get different large language models. Like I said, the data is foundation. You know, I think my job from a technology perspective is to look for what is the best and the most accurate model at the lowest price, right? Because these are expensive ones to run, you know, and you have to kinda figure out what are the right use cases to apply. So you kinda have to think about what is the most accurate model at the lowest price. Two, like we said, your data is your foundation, and you have to have an ecosystem where your models will work in, you know, in tandem with the LLMs to kinda give the right experience that is in there. The third learning is really in terms of when we talked about. You know, we are a product-driven, you know, driven structure in terms of our technology organization. But when we started to bring in generative AI, we had to kind of disrupt our model to kinda make sure that we can build the right experiences for the customer or our associates. The reason why I say that is these models are not are not good at giving you definitive answers, right? They are 80% good, 20%, they can do things that we kinda don't know or they are not accurate. So you have to now work with, you know, you have to change an engineer's mind to work with, you know, something that is not very definite, but it's 80% there, right? So how do you bake that into your process of rolling out experiences to your customers? Now, you have to also make sure that the customer is not exposed to that stuff. So those are some of the things, both from how we rolled out the different products, you know, from the use cases to different products, and how we thought and continue to learn as we evolve this, you know, evolve into this journey, if that makes sense. Great. Thank you. So you each talked a bit about prioritization and as you were describing those use cases, which is great. Thank you. I'd love to hear a little bit more, like any advice you have for prioritization across an organization, how you get the organization on board with the use cases that you've decided. I know you kind of addressed it in each question, but I'd love to go a bit deeper into that question of all the use cases you can do, how do you pick the one that you wanna start with? Murali, I'll start with you again. I, I think, I think the right way to do is what you said is correct. I think we have to understand the use case scenario is what makes more sense. I think the reason we picked up customer experience as a critical one for Victoria's Secret is because that's where the value for what we are trying to do, and that's where we can able to learn a lot of the insight from the customer, learn from them for certain experiences. But the other experiences, we are focused on associate focusing, because specifically when you go to the store, there are multiple situations where we can able to have an impact of the associates. Because in retail, the labor at the store is always the premium. How much can they be able to reduce their work behind the curtain, put them in front of the store, work with the associates with the customer to make the process easier, that's going to make the life of the customer much, much easier. So we looking for the business value, we looking for where all we can add value back to the customer. So that's where that's the priority that takes precedence over the other one. Doesn't mean there are other areas like supply chain or merchandising or finance, it doesn't make some sense, but we do them, but it's going to be a little different priorities. But for all of them, the focus is going to be the data and the experience that's going to have add value. Great. Thank you. How about you, Jessyn? I mean, for us, it was, I don't think prioritization for Gen AI is very different to any form of prioritization of projects that you have. We approach it in a very pragmatic and humble approach. We don't know everything. That's okay. And at the end of the day, it was really about, you know, what is the size of the prize? Like, what is the total amount of value that you can address with your Gen AI project or theme? How much savings do you anticipate? What's the technical feasibility given your org readiness, your talent pool, your ability to, to embrace that challenge? And the only additional lever is how advanced is the AI technology itself to serve that need today. Is it fully ready? Is it, still exploratory? Is it, like, very, very early stage? I think you need to be very realistic about it because it changes every day. But it's not true that, you know, today, Gen AI can do anything. I mean, it can do a lot of things, but can it solve any problem to the same level of precision, same level of scale, same level of effectiveness from a cost-saving perspective? No. And so you need to be very particular about this. And when I said, you know, we want it to be pragmatic and humble, it was really around not being paralyzed because you don't know everything. I mean, Gen AI is an area where you need to test fast. You need to. You can test fast, you can fail fast, and that's okay. And so you need to have that to embrace that mode of thinking into your organization. One of the things that our CEO helped us to do is to circumvent a lot of the traditional decision-making process to test our toes in the water, so we can actually develop use cases in a matter of four or five weeks for some of them, really see the results, and if we, you know, we have to make adjustments, so be it. We're learning, and that's okay. On the opposite side, you need to be very disciplined because you don't want to start to be, you know, to, to think that Gen AI is gonna change everything day one, and to have 100 use cases that you can't afford, you can't, you know, be disciplined about. And you need to be able to say, "You know, that doesn't work. That's okay, I'm gonna change it, or I'm gonna kill that idea, and I'm gonna move to the next one." And so I think that's really important to be ruthless in your prioritization every day and to think about, you know, where do you have enough proof of concepts, so you really want to scale to the next approach. And I think the last thing regarding your question about learning is really about how do you embrace that into your ecosystem? And that's why we want it for us to enable all of our people, whether they're on the frontline or whether they're at the headquarters, to be part of the solution of identifying what use case we want to tackle. Because if you do that, you know, with just an innovation team and not, you know, connected to the business, you're gonna move fast, but you're not gonna, you're not gonna be able to scale. If you do that just with embracing everybody in the organization, you might be slow. And so you need to find that balance. So, when you want to scale, people feel they were part of the ideation in the first place, so you have the right success factors to be able to scale. And what I truly believe, and that's where I'll end, is there is a question: how do you move from a moment to movements in Gen AI? It's very easy to have a lot of wins that are, you know, moments of innovation, but what you really need to do is to create a movement in your organization. So everybody thinks at their level, "What can I do with Gen AI?" You have the right expert that says, "Yes, it's realistic today. No, it's not realistic. Yes, we need to prioritize that because the size of the prize warrants that effort." And then being very ruthless at picking a few use cases to go at scale, so then you can find your next wave of innovation, and you can really build a movement into your organization that's gonna be lasting. So that's the way we're trying to transform function by function at Carrefour on our end, with the support of our CEO. That's great. Thank you. And Chandu? Yeah, I'll echo on some of the themes that were shared here, but I think generative AI is one of those things where it's like a death by a thousand use cases. Like, you know, every engineer, every product manager, everybody in the business team has a use case that they think can apply, you know, you can apply generative AI. You know, when we started off, and Google helped us with the hackathon event, and we've had a few hackathon events across to kind of get ideas around it. And soon we realized, obviously, like, Jess was saying, you know, it's a massive list of things. How do we go after it, right? The approach that we took is we looked not just from a generative AI, from a productivity play standpoint. We looked at both from a sales enablement, productivity, and experience standpoint, all three things in the mix. Then we created a value-to-risk framework. So there is value in terms of, like, the financial outcome or other outcomes that it can drive. And then what is the risk? You know, risk in meaning, you know, is the technology ready for that particular use case, too? Is there an adoption risk from the users because of concerns around the technology? You know, there are other risk around brand risk. You've heard a lot of, you know, bad PR that can come around. So from a brand perspective, how do we protect that? So we mapped out that across the value to risk framework. Then what we did is we identified core areas like marketing, merchandising, et cetera, and then we looked to productize these use cases. So, very definite set of high-value, low-risk use cases. For example, an associate-facing use case is a lower risk that we can pilot it before we, you know, put something out in front of a massive set of customers, right? So, and then we mapped those functional areas and created products like, you know, what would a Lowe's.com AI product look like to support all of the digital activities, and map the top uses around it. So that way, you can manage it like, throughout a roadmap, again, aligning that to the risk and the value it creates. So that was kind of how we adapted, and we're continuing to fine-tune and adjust to it because we're all learning. Great. Thank you. So in our last almost five minutes, you guys are each poised, your companies are each poised to drive a lot of value with generative AI in 2024. Any advice for the audience on how they can get started and really put generative AI to work with that short time frame? I think start with the data. Go where the data is. So wherever you have more confident the data is because, A, model is common. I think A model, you can use the model, which one you want, you want, whether it's a traditional AI, generative AI, depending upon the use case. But if the data is not the clean one, then what the outcome you get out of the generative AI is not going to be the one you want to trust, right? So focus on the data, focus on the learning. When I say the learning, you want to keep iterating the model. The model what you generate on day one is not going to be perfect. You have to feed the data back in so that the model learns more and more, and then you come to a point where it's just going to be more reliable. So that's what you have to do. So it's the iterative process. That's where it's just going to be more focused on the data, the process, and the evolution. How about you? I would say, don't get paralyzed because you don't know everything. Just get started. The good thing is you can, in a matter of weeks, you can see the results of what you wanted to develop. At the same point of time, stay focused. So start with data, I agree with you, and start with true pain points you want to solve. I used to be in the startup community, and it's always about solving a problem. Don't try to do Gen AI because it's fun or because it's cool. It doesn't matter. If it has an impact, it matters. If it doesn't have an impact, do something else. And so really think about what are the foundational pain points you really have to solve, and whether Gen AI is the response that is the more suitable for what you want to do. Maybe the third thing to add, just an additional dimension is, be responsible because, Gen AI is a fantastic machine, and it opens a wide range of possibilities. But at the same point of time, we all have a corporate responsibility, we have a responsibility for our customers, we have a responsibility for our employees, and so it's important to know in advance, like, what are the red lines you don't want to cross, and how do you, animate that into your organization? Because at some point, it's gonna be a spread movement, and so it's important to have guidelines, to have guardrails, to be compliant with GDPR, to, think about data privacy, to think about, also, like, what are the HR implications down the road? Because we all know that Gen AI, in a way, if you use it for associates processes, is gonna make performance converge, right? And so how do you... how are you gonna reinvent the way you do performance management to distinguish, you know, the top players, the lower-performing players? How do you make sure that your employees get smarter at using Gen AI? Not only by using, you know, better prompts, 'cause that's gonna be automated at some point, but also how do you make sure that they have the self- they're able to step back to understand the technology, to make sure that on the 20% cases where Gen AI is not the best answer or where Gen AI hallucinates, they can identify it by self-criticism, by being able to self-reflect and say, "Here, there might be something I need to do or to adjust," because otherwise you're gonna be much more powerful on the 80%, but you're gonna be less powerful on the 20%. So some of those problems are gonna be, you know, you don't have to say to solve them day one, but you need to think about them so you can find the right answers for the company and build the right processes around it. Great, thank you. And we'll finish with you, Chandu. Mm-hmm. Yeah, no, I'll... You know, obviously, we covered data and the guardrails that needs to come into play. Those are fundamentals, in my opinion. Really, data is the differentiator. The only two adds that I would do is, in terms of driving, it's a, it's a—to me, it's like a fast race on a tightrope. For driving adoption, it's gonna be super, super key in a technology that we are still trying to understand, right? So bringing in your business partners who are gonna be your biggest change management agents in the, into any of the initiatives that you're trying to drive, is gonna be super, super critical. That is one. Two is, you know, involve a human in the loop. You know, it's like the first time you get to... You know, back in the days when elevators kind of came into existence, there was a person inside who was pressing the button to get you up and down, just to make you comfortable in that. So, because there could be things that could go wrong, and that is the human-in-the-loop part of how you instrument this technology is good. Over a period of time, that human will not be there in that elevator to kind of bring you up and down. There is a voice that tells you you're going up, going down, you know. Over a period of time, even that doesn't exist. You can walk out, and you can just press a button, and you walk in, you trust it. But it is very important to get kind of that tribal knowledge and the support system that is there in a lot of the employees and the associates that's working. It's not a technology that has that view because it is not trained on tribal knowledge. It's trained on things that it can read off the Internet or what you're telling them as to what it is. So having human in the loop and really thinking about change management and adoption right upfront, that's the only way I believe value realization will happen out of this technology, right? So, yeah, that... Those are some of our learnings, and still evolving. Thank you, each of you. We're out of time. I really appreciate all of you spending the time with us. You gave great advice and great insights into what you guys are each doing, what your companies are doing. So thank you very much. Thank you.