Why AI ROI Still Feels Unclear

SPEAKER_00 0:02

And welcome back to episode three of That Martech Girl, the podcast where Martech meets culture, the business podcast with personality. My name is Rachel Gray. I'm your host, and today we're going to be talking again about AI. Surprise! Now we've been talking a lot about AI recently, and it's for good reason because I think everyone wants an AI strategy. Companies are investing a lot of money today in these new tools, and it seems like every day there is a new AI tool. But here's my question: Is any of this actually delivering ROI? And that's what we're talking about today. I'm so excited to welcome back Jorge Tittinger. Jorge is actually a former CEO of Silicon Graphics International, the founder of Titingger Consulting, an author, and an experienced board director and technology executive. Jorge is going to bring a really interesting perspective because he's not only spent years leading companies and making these types of investments, but he's also navigated some major technology transformations. So today we are going to be talking about what it actually takes to move beyond the pilot stage in AI and how companies should be measuring success with these AI transformation projects, and then what it actually looks like to scale AI in a way that delivers on real business value. This is going to be a great conversation, and I'm excited for you to hear it. Let's get into it.

SPEAKER_01 1:50

Hi,

Jorge’s Update On AI And Leadership

SPEAKER_01 1:51

Jorge. How are you doing today?

SPEAKER_03 1:53

I'm doing great, Rachel. Thanks for having me again. Appreciate it.

SPEAKER_01 1:57

Thank you so much. You know that actually you ended season two and now you're starting season three. So we appreciate you so much. Um yeah, but it's been a year since we last caught up and talked, and um you were just about to give me just a spiel of everything you've been up to. So I would love to catch up and just understand what has been going on in your world.

SPEAKER_03 2:21

Yeah, no, I appreciate that opportunity. So quite a bit, actually. I continue to serve on corporate boards as I used to, and I'm working with CEOs and executive teams on strategy, leadership, and organizational performance. But there's two areas that have become particularly exciting for me. The first one, as you can imagine, is AI, right? I'm spending considerable time thinking about how AI and specifically agentech AI is going to transform how companies operate, how leaders lead, and how boards govern. It's not simply another technology implementation. I believe we're at the beginning of a complete redesign on how work gets done. The second area is leadership development. I've been building a framework that I call executive intelligence to accompany artificial, uh, focused on helping leaders to develop capabilities so they can try thrive in a world of accelerated change, right? And uh of course, it's comprehensive of what AI will do to the workplace. And then I just published my new book. I don't know if you saw that.

SPEAKER_02 3:33

Yes, I think I saw that on LinkedIn.

SPEAKER_03 3:36

Yeah, so it's uh Yeah, it took a while, but it's I'm pretty proud of it. It's it connects the lessons from my soccer career to my business experience and my leadership career in both as a CEO and a chairman of a board. And the funny thing is, all three areas come together around one central question, which is how do we get people and organizations to perform at their best? And in today's world, including what AI can do to businesses.

SPEAKER_01 4:07

Right, because you still definitely need people. Yes, AI is great, but we still need people. So I do love that connection. Um,

From AI Washing To Real Value

SPEAKER_01 4:16

I've been hearing the same things that you have where AI is front and center of every single conversation, but I've been noticing as I'm talking to these Fortune 100 companies that there's a disconnect between and anxiousness where they feel like, oh no, something is happening. I need to, you know, run and catch up, but there's no real strategy behind that. So I wanted to ask your perspective as to like how do you see that gap being filled between, you know, just AI washing and actually getting real value from AI.

SPEAKER_03 4:56

Yeah. Yeah, I I see three categories of companies. Okay. And uh the first category is companies that are just experimenting with AI. So they have pilots, they have licenses, they have demos, lots of enthusiasm, but very, very little measurable impact.

SPEAKER_02 5:16

Right.

SPEAKER_03 5:17

The second category is companies that are generating productivity gains in very specific functions, right? So customer service, development, marketing, you name it, finance, etc. And the third category, which is still very small, they're beginning to redesign their entire business processes and business models around AI. And I think that's where I believe the biggest value will come from. But, you know, very few companies are doing this yet, right? We've seen the pattern before, very similar with other major technological transitions. So installing the technology doesn't automatically produce economic value, right? The value comes when people change how they operate to take advantage of that. So you know, I have a phrase that I'm telling people is AI adoption is not the same as AI transformation.

SPEAKER_02 6:13

Right.

SPEAKER_03 6:14

So the people who are just adopting it without changing anything, they're not getting the value they expected, right? Um, so you need to really actually change the way you're doing things to leverage your new technology. Right. I mean, imagine, I mean a very simplistic example. You know, a bank that installed all ATMs, right, but still kept the clerks inside, and the the customers still go inside, they don't use the ATM. You know, that didn't change anything, right?

SPEAKER_02 6:47

Right.

SPEAKER_03 6:47

It just cost them money. So there's a progression. I think the companies that are just experimenting will start using it for real, and then those that start using it for real will see the value and start transforming.

SPEAKER_01 7:00

I I was going to say, because you can have the tools and and the people like experimenting with the tools, but if there is no real training and change management, then it won't get scaled at a global level because many of these organizations are global, right? So how how exactly is that happening? And I guess um that leads me to my next question. So when you speak to these companies and they go, okay, well, my first AI initiative has been successful. What is the next step? What do you tell them that is the path for?

Measuring Success Beyond Time Saved

SPEAKER_03 7:40

So yeah, so so to some extent, you know, the world success, the word successful could mean very, very different things, right? So what I first ask is what changed in the business, right? Because successful could just mean you know, I bought all these tools, I have people using it, that they could define that as success. But if it's not changing something in the business, you know, it's interesting, but I don't I don't consider that a success for the corporation. So if somebody tells me that, for example, AI saved employees 20% of their time, that's interesting. But what they do with the time, right? Did you increase output? Did you improve improve customer response time? Did you accelerate product development, reduce costs, increase revenue, improve quality? Those are real tangible business results that these things ought to be producing. Um or did you simply give people 20% more time, which nothing inherently wrong with that, but that hopefully improves their experience and creates capacity. But you shouldn't count all every hour saved as return on investment, right? So, and I like to distinguish corporate value in three areas. So, first one is efficiency. Are we doing the same work with less resources or less time? The second one is effectiveness. You know, are we doing the work better, higher quality, better outcomes? And the third one is transformation. You know, are you now able to do things that you were not economically or operationally able to do before because of the impact that this AI project, successful AI project had? Um, that's the third category is where things can become really exciting about it, right? Um, but that's what I would ask is okay, did you define success as improved business results? What are they?

SPEAKER_01 9:45

Because it's really you know great to experiment, as we said before. It's nice to get the tools up and running, it's nice to get people using the tools. But did you define your success criteria prior to this experimentation in order to test if that pilot is actually going well and if it's actually you know feasible for the organization to be able to scale it? Because if not, then it's just more tools that you're just using.

SPEAKER_03 10:14

Yeah, yeah.

SPEAKER_01 10:15

Yeah, yeah, yeah.

SPEAKER_03 10:17

Yeah, it could be worse, really.

SPEAKER_01 10:20

But I feel you know that organizations are so anxious that they skip that step, the step of defining the success all the time, because they're just like, oh, we have to do something and we have to do it now. But instead of really sitting down and having these conversations where they're like, okay, we need to define how this is going to have outcomes for the business, they're just like, let's implement, let's get going, etc.

SPEAKER_03 10:46

Yeah, you there is that pressure for sure, right? And you know, leaders can respond to that pressure in a you know, rushed and planned manner, or they can respond properly and define you know, success criteria, define the impact, define how you know all the things that need to be defined, just like with any other technology, right? I mean, that's the thing that's interesting, is the biggest difference with AI is the ubiquity of it and the speed at which it moves. But just like without any other technology, you would not go change your ERP system without a plan. You wouldn't. And so why are you adopting AI without a plan?

unknown 11:36

Yes.

SPEAKER_01 11:38

That's so true. It's also technology as well, like any other you know, tech project. So I want to get a bit granular. If you can share um, if you have an example of an AI pilot that you've seen really change and and and speak to that transformation that you were talking about in the business. And you share like how was that done?

SPEAKER_03 12:04

Yeah.

A Governance Model That Prevents Chaos

SPEAKER_03 12:05

So yeah, I'll give you an example. Real life, one of my companies that I'm on the board, um, you know, this company, one of the things they do is they develop software. Okay. And uh, but it isn't it isn't like application software, software to run a machinery, right? That is used to produce other things. Um, sorry, I'm being a little vague on the company, but but uh to protect the innocent. And um, like most companies, we started with that approach, right? It's like, oh, we need to use AI. People started using AI, and we very quickly created what I call the tenth city of AI, right? So everywhere in the company, little AI projects were being done without any governance, without any understanding, and um we stopped it. We we created uh you know an AI subcommittee of the board, number one, um, and worked with leadership to have a council where people had to submit their plans, they needed to get vetted and approved, et cetera, et cetera, et cetera. And as you can imagine, and by the way, I I was the one, the board representative to that, to that committee. And so we got discipline quickly, right? And so people, you couldn't start something without uh you know, a project plan that clearly identified what were you gonna do, what was it gonna touch, what was the the hopeful value, we didn't know, right? But one of the areas where we had the most impact was in we started using it to do the first rev of the software code that we needed to generate. Okay. Um, and first rev is very important, right? Um, we first, this is pretty esoteric code. This is not just like regular, you know, IT. Um so we had to teach the AI from all the prior designs that the company had done for their products. So we fed that, we made sure it was the data was clean, curated, etc. And uh over time the agent has gotten really good and it's saving at least 30, if not 40% of time that used to take to go from you know, first in engineering doing the first lines of code to getting a first re a first draft that then we have the engineers review and make sure there's no errors, et cetera. And then we also have another AI that's more like a quality control uh agent that's looking for are they way are there ways to make the code more efficient, right? Because the lighter the code, the easier it's to debug, the easier it's to fix, to upgrade, et cetera. So really, really tangible ROI. Um in this company, you know, this this software needs to be developed for each pro each type of product that gets shipped. So it's accelerated a time to market significantly, and more importantly, it's accelerated the time to revenue for our customers by an amount, right? So that helps a lot. So very real, tangible example. You know, there's other examples, you know, in marketing, you know, very easy to get faster, better uh marketing material. Um, but the impact is for me, the biggest impact I've seen in the companies that I'm involved with is this thing that's saving months. So well, it was well done, right? And and it was like, you know, we made sure we have the human in the loop at the right place, because that's one of the dangers with agents, you know, it's like, oops, oops, we made a mistake. Who's responsible? Nobody.

SPEAKER_01 16:14

So um speaking of that human in the loop, I think that goes to my next question, which is like, how do you identify the problems with these AI pilots?

Diagnosing Pilot Failures And Human Review

SPEAKER_01 16:28

Where it feels like you know, you kind of have this pilot going and you're excited about it and it seems to be doing something, but as you said, oops, it made a mistake. Okay, where is the problem? Like, where can we fix the problem? Is the problem with you know the workflow? Is it with data? Is it with you know how you've been able to set it up? Is it with the success criteria? How do you start to think, how can I break this apart to diagnose that problem?

SPEAKER_03 17:00

Yeah, great question. I I think there's probably four sources of problem or failure, if you will, right? So the first one is the technology itself, right? Did it do what it's expected to do, right? The second could be the data, right? Was the information that it learned from accurate, accessible, and sufficient? Because one of the problems with agents is they will complete the task you ask them to do.

SPEAKER_02 17:30

Right.

SPEAKER_03 17:32

So if it's missing data, and you know, we've seen the example, it'll it potentially will make it up.

unknown 17:38

Right?

SPEAKER_03 17:39

So yeah. And um the fourth area could be, you know, are we solving for a real problem with real assumptions, right? Um but I think if you think if you look at this from a governance perspective, first of all, you know, there's been a big shift in the last couple of years from it from what I call co-pilots, where you had to tell it what to do, and they give you a response, and you told it something else, and they gave you a response, and you told it something else. So it was very clear who was telling the AI to do something. When you set up an agent, you can lose that, right? So I think it's super important to very clearly define at which points in the flow of the whole process that you've asked this agent to do, you need to get a human in the loop and review.

SPEAKER_01 18:31

And I think that goes back to your point just about the setting up of these pilots, like the governance that you explained even in the successful pilot, it's so important. Being able to have these boards, being able to lock everybody in a room and say, okay, what are we going to do? Let's talk about the success criteria, the workflows, etc. Because without that, it's not gonna work.

SPEAKER_03 18:54

Yeah. So when I when I get presented pilots, I I ask for three things, right? A defined business problem, measurable success criteria, an accountable owner, right? One accountable owner. Too many pilots start with the technology, right? We have this great AI, let's find something to do with it, right? Instead of, I would reverse it, I go, let's start with a meaningful business problem, establish the baseline, you know, define what success would look like, define how you will measure it, and make sure somebody with operational responsibility owns it. It's not the CIO, it's not the IT group all the time, unless the pilots are in their groups, right? And you know, I establish conditions for scaling or stopping the pilot before it begins.

unknown 19:50

Right?

SPEAKER_03 19:51

So before it begins, you know, if this happens, we're scaling it. If this happens, we're stopping it. It's not good practice to, you know, all of a sudden you start running into issues of, oh, should we stop, should we not stop without having defined that criteria? And then if you don't do that, you run the risk of what I call pilot purgatory, right? Lots of interesting experiments that never become meaningful business capabilities. And that's, you know, I heard a number that I actually find it hard to believe that 95% of people who are using AI say they're not getting the value. You know, I would think there's more than 5% that are getting value. But then to satisfy users, a pretty high number. It's not in the certainly not in the single digits, you know, and definitely high double digits.

SPEAKER_01 20:44

So it's surprising, but I can see how without the defined criteria and workloads, it would be disappointing.

What It Takes To Scale AI

SPEAKER_01 20:52

Um moving to something you were talking about with scale. I, as I have conversations with the companies I work with, this is something that's on everybody's mind. Like it's fine to do a pilot that's worked, yes, great. But moving from that in order to implement across like a Fortune 100 company to all of these different teams is another story. And so I wanted to ask you, you know, why do you think this is so difficult? Number one, and like in your experience, what are you doing about that?

SPEAKER_03 21:26

Yeah. Yeah, I know you're full of great questions, Rachel. There's another great question. And the reality is, I think there they're you know, a pilot proves that something can work under a particular set of conditions, right? So scaling needs to prove that the organization can make it work repeatedly, reliably, and economically. So they're very, very different challenges, right? So in a pilot, you might have very enthusiastic employees. Right? They're doing it. You have clean data, you have a lot of attention from leadership, and to some extent, a controlled environment. Scaling requires integration with existing systems, consistent data quality, cybersecurity, training, governance, process redesign, and change management, right? And there's a huge human dimension in this. So the change management is mostly where things go awry, right? Because I really think people need to understand why the changes make matter and how it affects them. And the phrase that I use is technology scales through systems, but transformation scales through people. So that's the biggest difference between a pilot and scaling. And the other thing is, you know, technology can change quickly. I mean, you can you know install something, boom, you have it. People change slowly.

SPEAKER_01 23:03

That's true.

unknown 23:04

That's very true.

SPEAKER_03 23:05

That's a reality, right? And so even the fast movers compared to the technology velocity is very slow. So you need to plan for that.

SPEAKER_01 23:16

Well, I think the million dollar question that I am I am wondering personally is you've been a CEO, you've been a CMO. What do you need to see in order to say I check off on this project and I am going to allocate this much of my budget to this AI pilot and going from pilot to actually scaling throughout the organization? Because we know it's not cheap, right? It's not cheap at all. Um, I would love your perspective on that.

SPEAKER_03 23:50

Yeah, uh, so so you're basically saying, why would I bet on it?

SPEAKER_01 23:57

Right. Right?

SPEAKER_03 23:58

Why would I invite okay?

SPEAKER_01 23:59

So you can say yes and sign.

SPEAKER_03 24:01

Yeah. So I would like evidence in five areas.

unknown 24:04

Okay.

SPEAKER_03 24:05

First, meaningful business problem with credible economic output or opportunity, right? I'm gonna assume that the pilot already happened. So a demonstrated pilot against an established baseline. So we can say the pilot was successful. Realistic scaling plan, including integration, cost, risk, change management, clearly accountable executive, not just the CIO, unless the project is in their uh group, and then a measurement system that tracks actual value after deployment. So with those, you can actually truly assess, okay, should I uh continue to invest in this, right? So the business owner should be accountable for the output and the outcome. And you know, all the other roles need to contribute, but I would not spread ownership across multiple people because if too many people own it, nobody owns it, right? So one owner again, right? Exactly. And then I also want clear decision points of where management can accelerate, modify, or stop the investment, just like a product life cycle, right? Where you have these gates, and at those gates you go, do we go on or do we stop? I think this the same thing needs to be done with AI projects to be treated like that, and the biggest challenge is much faster.

SPEAKER_01 25:43

What I'm hearing is that you are taking out the anxiety and the pressure, and you're going back to just the tried and true that has worked in other transformation projects, and you're applying it to today's world. That's what I'm hearing via the experience and what you're saying today. Is that correct?

SPEAKER_03 26:08

Would you say mostly?

SPEAKER_01 26:13

Okay.

SPEAKER_03 26:14

And I think the reality is that technology is often the easiest part, right? And what's harder is changing the behaviors, changing the workflows. You need to realign the structure of incentives for the way that work needs to be done going forward. And then you need to challenge deeply held assumptions about how work should be done. So you know you need to slay sacred cows when you're doing AI because of its potential impact and velocity, right? And like I said a little earlier, people don't automatically change because you give them a better tool. Right? So leaders need to demonstrate and explain the purpose, the value, the opportunities. They need to create a safe environment for experimentation. Address legitimate concerns about security, accuracy, accountability, and but also very importantly, they need to model their behaviors, right? So if they're demanding that everybody uses AI, guess what? You better be using it too in the right way, right? And this is where I think culture becomes critical, because in organizations where people are afraid to experiment, to admit mistakes, to challenge the status quo, that transformation will be way slower. And you know, as I said before, the the culture is an organization's operating system, really, right? So AI doesn't eliminate the importance of that operating system. It makes it actually even more important, right? And um I think the the companies, the few that I mentioned earlier that are truly changing their business model, that are changing their processes and their strategy to include AI, in my opinion, it ought to be always starting from the human. And how does the how can the human leverage it to make the human more effective, more efficient, etc. etc. Um, those are the ones that are going to be successful. So, yes, treat it like other tech, except for be aware that it really is going to change the way you work.

unknown 28:32

Right. Right.

SPEAKER_03 28:33

We're in we're in a without a technology, it might not have, right? This one, it will.

SPEAKER_01 28:39

My final

A 90 Day Plan For Marketing Leaders

SPEAKER_01 28:40

question has to do with the CMOs, the CEOs, maybe the VPs of marketing listening today that are just in this state of anxiety and pressure. What would you tell them in the next 90 days that they should do in order to implement some of these strategies we talked about today?

SPEAKER_03 29:06

Yeah. I, you know, the the the biggest thing is I would encourage marketing leaders to speak the language of business, not the technology features. You know, and I've seen it too much where they say, oh, this, you know, it can do this, it can do that, it can do that. You know, so don't come to the board with a presentation of how many AI tools you've deployed, how much content you're generating. Show us that AI can improve business outcomes, right? Customer acquisition, conversion rates, retention, customer lifetime value, uh, effectiveness of your campaigns, speed to market, those are the kinds of things that the CMOs ought to bring to the board. It's not how many pilots. It's you know, we're improving these areas that really matter to the business. So, for example, if AI allows the marketing team to produce five times as much content in the same amount of time, that's a great metric, but it's an operational metric. Now, if that content produces higher quality leads, improves conversion, reduces acquisition costs, those are business values. So shift your lingo from features and functions to outcomes as a CMO. Um, and I would also want to understand the risk to brand reputation, privacy, IP that comes with using AI potentially in an ungoverned manner, right? So, but if you really have set up the governance well, those risks go way, way, way down. So don't sell me AI capabilities. You know, show me how they change the economics of the business. To me, that is the best advice to CMOs. And it's a different language than they're used to using.

SPEAKER_01 30:58

So yeah, this is great advice. We

New Leadership Book And Closing

SPEAKER_01 31:02

thank you so much, Jorge. Thank you so much for your time. Um, I know you were super busy. Everybody, I need you to go check out his book. Um, can you give more information about that? Where can they find the book?

SPEAKER_03 31:14

Sure. So the book is it's a leadership book. You know, for many people that don't know this, you know, I had a career in professional sports, soccer specifically, before I started working in business. And the lessons from that part of my life are so applicable to leadership. So the book's called The Beautiful Game of Leadership. Most people know that the beautiful game typically refers to soccer. So that's the play on words. And the subtitle is Lessons from the Soccer Field to the Boardroom. And um it just got released on Amazon on the 6th. So it's you know, it's been out for only a few days, um, getting some really terrific um reviews, uh, you know, many of them from friends of mine. So I didn't I expect that, but there's also a lot of people who I don't know who are reading the book and and reviewing it. Um and I think I think the purpose of the book was really to shift the way a lot of people think about leadership and make it um much more attuned with today's world instead of the way it used to be, where you know there was a lot of um leadership by command and the power of office, et cetera, et cetera. I think in today's world you need to have a shift from pure IQ-led leadership to IQ plus EQ, um, where you're using both sides of your brain. But it's uh according to the read, it's a fun read. It's uh um, you know, there's a lot of uh things you could apply the next day in your work.

SPEAKER_01 33:01

So even in the conversation today, it sounds like it would be relevant to uh leaders looking to implement AI in today's world as well.

SPEAKER_03 33:11

Yeah, so if they look for that title, you know, Beautiful Game of Leadership and My Name, uh, they'll find it on Amazon. It'd be great. I appreciate the opportunity to do a little plug for my book.

SPEAKER_01 33:23

Of course, of course. Thank you, Jorge. Um, I hope you have a wonderful day, and thank you to the listeners for listening in. Thank you.

SPEAKER_00 33:32

Thank you so much for joining me for this episode of That Martech Girl, and a special thank you to Jorge again for joining me and sharing his insights. I always enjoy our conversations. He is so inspiring. If you enjoyed today's episode, please make sure you're following ThatMart Tech Girl on your favorite podcast platform. And make sure to come hang out with us on LinkedIn and Instagram for real-time updates weekly. And of course, if there's someone who you think could benefit from this conversation, be sure to share it with them. Thank you so much again for tuning in. I'm Rachel Gray, and I'll catch you on the next episode. Bye now.