A CTO’s POV: Speaking With Your CEO About Agentic AI

Arvind Mathur:
Thanks for joining us on this episode of Executive Insights. My name is Arvind Mathur, and I’m an enterprise strategist at AWS. Today I’m thrilled to be speaking with my colleague, Matthias Patzak. Matthias recently wrote a blog on LinkedIn that’s generated a lot of discussion. Today we’ll be covering the content of that post and why it’s sparked so much debate. Matthias, welcome to the show.  

Matthias Patzak:
Yes, thank you very much. Yeah, actually it's sparked a lot of conversation. I got a lot of comments.

Arvind Mathur:
And more than anything else, it's the conversation that these thought-provoking pieces trigger, which is so valuable.

I think that's what created a lot of engagement, because in all of our lives as tech leaders, we faced that situation where we knew we had to watch this new trend and we were not paying enough attention to it. And the CEO or some executive comes and asks, and you're always feeling on the back foot then. I think that's what really appealed to a lot of people and created a lot of engagement. 

Matthias Patzak:
Yes, it's true. And I guess this is what sparked a lot of impressions on this LinkedIn post with the CEO having to ask the CTO about, "Hey, what is this agentic AI thing?" And this is what I still see in a lot of organizations that the CTO, the CIO, the VP Engineering's are too busy with the day-to-day business, with keeping the lights on, with spreading out the capacity across all the current priorities. And they don't really have capacity, mental capacity, but also physical capacity and budget to really play around with new technologies. 

Arvind Mathur:
Totally agree. And I think now more than ever, it's important for CTOs and technology leaders to invest more time in this because things are changing so fast. And more importantly, many of these capabilities have been so consumerized, and business leaders are so much more aware of it that you cannot afford to be on the back foot when these questions pop up. You have to invest some capacity in scanning this, and actually introducing your business leaders to this, like you suggested very well as well, right?

Matthias Patzak:
Yes.

Arvind Mathur:
So let's perhaps step into your recommendations how to actually approach this. I know that generated some conversation as well. I loved you were very provocative with some of your suggestions, and I think it'll be good to jump into some of that and explore those deeply.

Matthias Patzak:
So the first step that I recommended actually was turn it into a business conversation. Because most of the time still, CTOs and the CIOs, they just want to understand the new technology. They want to understand the guardrails, they want to understand the cost, and they're not really thinking how we would call it at Amazon, working backwards from the customers. So they're not really getting out of the building, they're not really observing their customers and their users. They're not really understanding the customer needs and the customer problems, and they're not able as IT experts to have a business conversation with their peers in business. And this is the first point that I made in this LinkedIn post, to turn the discussion into a business conversation.

Arvind Mathur:
Totally agree. I think it has to be a business conversation because this is an opportunity to make business impact. This is not just some backend improvement. I think the conversation that the piece generated some discussion in this, was this whole find the 10X ideas and not just 10% ones. And I tell you, I love this one because it's so provocative. And especially with Agentic, it is possible now to find these 10X opportunities more so than many other technologies in the past. But the reality is that in our lives we do work on maybe 90 10% improvement ideas, and then five to 10 of these 10X ones. Why are you pushing on focus on the 10X ideas?

Matthias Patzak:
In general, I believe these incremental iterative improvements in organization, this is what really, really makes a successful organization, because then you are learning organization. If you're aware about your business, about your technologies, and then week by week, day by day, improve by 1, 2, 3, maybe 10%. But what I observe in the discussions about Generative AI, AI in general and now with Agentic AI, that a lot of organizations just think about improving their internal efficiencies, and they're not really thinking about how to pivot the business model. And from my perspective, what's going to happen is, and this is what we saw 10 or 20 years ago when digital native companies emerged, and then we had mobile first companies. We are going to see generative AI native companies and we are going to see Agentic AI organizations emerge. Then if you just worked inside your current business model, you might be lost.

Arvind Mathur:
Yeah, I agree. This is the sooner you can start looking for the 10X opportunities, the better it is. The only watch-out I allow for up in situations like this, the way you set it up, if you are on the back foot and a business leader has come to you, sometimes they often come with an idea as well. And what I've observed, and this is really the dynamics of the situation, you have to read how the situation is, sometimes you have to quickly work and show results on the idea the CEO might have brought to you that, "Hey, Agentic, I've always worried about this problem. Can we solve it with that?" It's usually very useful to quickly figure out a way to deliver on that. And then in parallel, once that builds the confidence and the credibility, then start talking about, "Hey, this is not just a small improvement opportunity. There's actually a lot more we can do with this." I know there was a good discussion on this topic in the chat conversation as well, but love the idea. Let's move to the second one.

Matthias Patzak:
So the second one is the traditional advice on, I rephrased it here as form a transformation squad, not a tech team. But actually the message is that you should not just work in the traditional IT silos. For sure, it needs to understand the core concepts of the new technology first. But if you really want to deliver prototypes, and if you want to really learn about the business opportunities, you need to have your Head of Sales, operations people, finance people, legal people in the room as well. And not just as an oversight, but really as a cross-functional transformation talk. Because it's not just a technology project, it's business reinvention.

Arvind Mathur:
And I think there was complete alignment and agreement on this in the conversation. I feel same way strongly as well, especially with something like Agentic. And if you want to focus on the 10X ideas, those are not internal technology improvement ideas. Those are opportunities to reimagine how our customers experience our products and services, what our products and services are capable of, et cetera, all of those things. So that requires broad transformation.

Matthias Patzak:
How do you perceive it and observe it in your customer conversations? So do you see really organizations working in cross-functional teams in these transformation squads, or do you see IT silos?

Arvind Mathur:
It's a wide distribution obviously. So there are the ones which are more mature, who understand this, have experienced the fact that just doing this as a technology project doesn't work. But then there's a vast number of organizations that are still dealing with some of these projects like technology. And that's the role we play as we have conversations with the CTOs or the business executives, is to help them understand, to look at this as a multifunctional cross-silo opportunity. And I've had great learnings on this personally from my own experience as well. In my time at an insurance company, we were trying to transform processes like underwriting and claims, and while they belong to the ops organization, it's not just ops, it's also sales, and product, and risk and customer experience. All of those have to be involved if you're doing any changes. In our case, we were trying to change claims time from three to four weeks to less than an hour. And that's a dramatic change, which doesn't come from just improving internal operational processes. So totally agree.

Arvind Mathur:
All right, so let's jump to step three.

Matthias Patzak:
So step three, what I recommended was pick two to three relevant use cases and move fast. And it's really about moving fast and having these pilots run in parallel and not sequentially, and speed with guardrails. And guardrails are important if you run such experiments, this beats perfect planning. Because what I see, and this is a quote that one of our team colleague Jake Burns used to make is, "Don't start when you're ready but become ready by starting." And this is what I really like and recommend with Agentic AI in especially. And there is another aspect in moving fast and running pilots in parallel, because it's a new technology, it's a new use cases. And because it's an experiment, you need to expect that 70% of the experiments are going to fail.

And so if you run your pilots sequentially, the first, the second and third one might fail and just the fourth iteration, the fourth experiment might really succeed. So this is why I recommend that you need as a CTO, have capacity, mental capacity, but capacity also from a budget perspective, from a people's perspectives available so that you can run your pilots in parallel. And this was discussed a lot, because it looks like some organizations don't have this appetite and don't have this capacity available.

Arvind Mathur:
And you were asking earlier about my conversations with customers, this is probably the number one challenge part. The reason is that most CTOs when they're put in a position like this, don't have the capability, and therefore the confidence to run multiple large projects in parallel. They find comfort, and even I used to at times when I was in confident about something new, would be more comfortable with experimenting with a few things, oftentimes actually not in the limelight, to learn and build confidence that you can then start going faster.

And I think that is the point which is really important to bring out here, is hopefully you've done that already before you have that conversation with the CEO. Because if you've done the experiments which are smaller, then you have the confidence then to go and find those 10X ideas and start running them in parallel. Which is I think the central point of the article, which I loved so much, was you should not be having this conversation with the CEO initiating this. You should have done the experiments and gone to them with the ideas, and then you'd be ready to start pushing this forward with confidence.

Matthias Patzak:
Yes, it actually must be part of your operating model and it must be part of your culture.

Arvind Mathur:
One of the questions that came up in the chat was, "Hey, there are so many new things that are coming out, there's Agentic, robotics are moving in a different direction, there is core AI, there's data and new ways to drive data and analytics capabilities, et cetera. How do you keep an eye on everything that's going on, and experiment with it sufficiently so you can proactively go to your business leaders? What are some best practices you've seen?

Matthias Patzak:
So actually you cannot play around and experiment and get knowledge about every single trend. And this is why you need to have an innovation or technology radar capability in your organization as well. And there is a thought leader, Pascal Finette, he wrote a book, it's called Disrupt Disruption, and he talks about how to spot signals, technology, market, customer change of signals. And then the question is how often do you encounter signals? If some signals in certain technologies show up a lot, then you need to jump on it and experiment on it. And this is why you also need to have different ways of working for experimentation.

So on some signals you might just have an individual developer, or a designer or product manager on a Friday afternoon experimenting with a prototype and with real customers. For some other experiments, you'll need to have free capacity for a team for two weeks with a time box. So it will be two weeks and they need to achieve something in these two weeks, and you're not going to invest more money, or more capacity or more time. But you need to be able on spotting signals and acting on signals. And also this is maybe the most important aspect, saying no to some of the signals in some of the aspects. So being able to prioritize this is key.

Arvind Mathur:
Yeah, and I think something you said earlier is another really critical aspect of this that I've personally practiced and advised folks, is the culture of curiosity, experimentation and which leads to innovation ultimately. And what I've noticed is in my own personal practical experience as well, is that if you create that culture in your team, in your organization, they will be doing experiments and keeping their eyes on what's happening, what's emerging on the radar, even without specifically setting a strategy or an action plan for it. My most useful situations were when a team came to me and said, "Hey Arvind, we've been exploring this thing, I think there's potential here." And that gives me the opportunity to go to a business team and say, "Hey, we figured this out. Is there a way to act on it?" So if you've got that culture, awesome. If you don't, start building it today. Because in today's world, if your team is not autonomously going and scanning and experimenting, then it's very hard to do this when the opportunity shows up.

Matthias Patzak:
Yeah, fully agree. And key for this is that you can show the successes and the failures of your experiments, because otherwise it will be just finally your product managers or your business colleagues are going to complain, because part of the capacity part of the teams are playing around with new technology and taking away capacity from actually delivering business value or feature. And this is why you really need to celebrate success. And a failed experiment is also a success of an experiment, but you really need to celebrate this inside of your organization. And then you create this culture of, it's okay taking away capacity and experiment with some new technologies or business ideas.

Arvind Mathur:
Terrific. Let's move to step four.

Matthias Patzak:
Yeah, step four. From my perspective, this was where I found the comments really, really interesting. And the advice is pay for scale while you pilot. So don't prototype just cute demos, but build production-ready solutions that can handle enterprise volume from day one. And from my perspective, so there are different type of agents, there are agents for the knowledge workers. So if you have a customer service agent who's going to use an agent on her desktop to automate one of her workflows. But as a former CTO, I find the agents that are microservices with a brain much more interesting. But having these agents inside of your architectures, into your applications, this is not easy.

So you need to have event-driven architecture. You need to be able to have proper observability in place, and at least at the beginning of a prototype, you need to be able to have humans on the loop, in the loops, sometimes off the loops. And you can just learn about the capabilities of agents if you have this agent in productions, maybe not with 100% of your traffic, but maybe 1% of your users, 1% of your traffic so that you really limit your blast radius while trying out 10X ideas. But this was really hard for a lot of people to understand and to adopt in their environment because they're missing probably the failure culture.

Arvind Mathur:
And I think what you see here links nicely to the previous point as well, that actually even within experimentation, there are stages. There could be experiments you're doing completely out disconnected with your enterprise. And if some of those start showing promise, then you start doing experiments internally in IT first. As you were describing just actually I love that point, you already have business processes which are being served through services and APIs, put a brain inside them internally, and that gives you that experience and comfort building production-grade agentic capabilities, that when you have that conversation on the 10X business cross-silo opportunity, you will have the confidence to go ahead and start doing this in production in parallel, across multiple ideas. So great point. It's a step in that experimentation to innovation journey to start doing this internally within technology first.

Now, I do understand the comments that were being made on this one that again, a lot of people are uncomfortable with this. They feel that it's important to do some experiments early on, like you said, reduce the blast radius. And I think the point we are really making here is that you have to do that before you have the business conversation. All of those small blast radius, cute pilots and projects and experimentation, if you've done that already, that then sets you up to move fast when the business opportunity shows up. And you're not experimenting, and hopefully at that stage you've gone past the situation where 70% of things are going to fail, and you can deliver more consistent capability at that stage.

Matthias Patzak:
Yeah, I fully agree. And this is why people need to get hands-on on new technologies early on, and this is what I recommend is include partners, include the AWS account teams, let them show you how to get hands-on with new technology.

Arvind Mathur:
All right, let's do step five.

Matthias Patzak:
Yeah, step five. It's I would say a no-brainer, but still you can debate it. So step five is measure business impact, not technical metrics. So track revenue increase, cost reduction, cycle time improvement, change in your conversion rate, and not model accuracy or user adaption rates. For sure, you need to have these leading indicators on adapting rates. You need to know some quality metrics, but you want to introduce new technology not for the sake of new technology, but for either having a 10% improvement in your business processes, or being able to come up with a 10X use case. And this is why you really need to measure every solution into production with real users and measure the real business impact. You don't want to tick a box on a feature list. You really want to say, "Hey, with this new Agentic AI, we were able to reduce exit rates on the website, by minus whatever percentage."

Arvind Mathur:
And I think there was universal agreement on this point. I wanted to share quickly just my personal experience on this as well in that same claims process acceleration one. Initially when we were doing that project, we were making the mistake of measuring different people in the team. So our operations team had one operational measure. Our tech team had another operational measure of delivering features, and our sales team had some other measures. It was not reaching that 10X level because of that. It was only when everyone got aligned to the same goal, which is move this from three weeks to five minutes, that not only just aligned everyone, but actually that's what drove the creativity, the barriers to what's stopping us from delivering the 10X and really creative solutions to that came out cross-functionally. So actually this is not just a measurement thing. This is a great way to align teams and to get them focused on the really big barriers that stop us from delivering 10X.

Matthias Patzak:
Yes, you're right. The alignment part is really important.

Arvind Mathur:
Super great conversation on this. I hope we can continue to engage with folks and go get more inputs on it. But the number one thing I'm taking away from this conversation, my personal learning and I'm going to recommend this to everyone, is start scanning. There's a lot of change happening, and you have to be ahead of the game in understanding what's possible and take these ideas to your business leaders. So my suggestion based on this really would be, go into your AWS account today, talk to your account teams and figure out how to start using these tools and start building now.

Matthias Patzak:
Yes, fully agree. And the tool, or the Amazon service that you should use is Amazon Bedrock Agent Core. So we have different services around agents, but the core for having agents in your architecture is Amazon Bedrock Agent Core. And this is really the service I recommend organizations getting their hands on as soon as possible.

Arvind Mathur:
Brilliant. Let's go build.

Matthias Patzak:
Let's go build.

Listen now

Listen now

Listen now