brandon carlson lean techniques digital transformation ai business value practical applications

So our next speaker is Brandon Carlson, founder of Lean Techniques, a Des Moines-based consultancy focused on digital transformation, IT modernization, and software delivery. With more than 30 years of experience in technology, Brandon is known for his practical people-first approach, emphasizing curiosity, collaboration, and meeting organizations where they are. In today's session, Brandon will explore how lean thinking can help move AI from isolated experimentation into real, measurable business value with practical examples across marketing, operations, compliance, and sales. Please join me in welcoming Brandon. All right. Thanks, everybody, for being here. Thanks for the nice intro. Really don't like this microphone thing. It's not my style. But as she said, my name is Brandon. Hope everybody's having a good conference so far.

audience engagement lean practitioners ai adoption systematic usage sporadic usage

Good. You're going to take some stuff home already? Good. Then I won't let you down. So I like to say if this is the only lean and AI talk you've ever been to, it's likely to be the best you'll ever see. How many people are lean practitioners? Okay, a few of you. So how many people are using AI? More of you? That's kind of shocking, really? No. Okay, are you using AI in systematic ways? Or are you kind of, you know, like, oh, I need to draft an e-mail, I need to explore this topic, I need to do this or that. How's everybody using it today? Just have a couple of people throw out. Sporadic? or systematic? How many people are systematic? Okay, sounds good. So this is kind of an interesting talk in my opinion because we are really talking today about kind of new meeting old.

toyota production system lean manufacturing measurable impact systematic implementation process framework

So like Lean has been around for a long time, right? Originated in Toyota Production System, Lean Manufacturing, those kinds of things. And I think what's important with all this change that's going on and all the craziness that we're dealing with, it's really important to understand that these old tools are still valuable and they still work, right? Is this speaker like squeaking and stuff or is it just me? You have the feedback. Like, how do I... like to walk around, so this is going to be trouble. Okay. So what we're going to try to do today is we're going to try to help you all see a process that you can take home and use today to start implementing more AI within your organization and create a measurable impact from it. My little tagline here is just rub a little AI on it. That's what we hear a lot at work. And they're like, well, can we just AI this? Can we get 30% improvement with AI?

change management okrs kpis transformation continuous improvement

Just rub some AI on it. So That might work, but it's not very effective. So what we'll do today is talk about how to make it more effective. Okay, so who's this talk for? This talk's for people who think good enough isn't enough anymore. It's for you if you're a leader that you don't want to keep doing the same things, you want to get better and you want to improve. And it's also for you're a team member. How many people in this room have been told by somebody that thou shalt be using AI and making things better? We see that a lot. People are coming to us all the time now. What do I do? I've been told I have to do this, and we've got these OKRs or KPIs or choose your own adventure, their metric of choice. Now what? So that's what Lt. does. We transform technology from keeping the lights on to leading the way in organizations. So we're a transformation company. We do a lot of change management. We'll talk probably a little bit about this depending on time, but when we go to roll out some of these things, how do we get those changes to actually stick?

fomo competitive pressure ai anxiety fortune 500 china

Because that's probably if you have tried to operationalize some of this stuff, you've probably run into that. Well, we did all the things and nobody uses it. Anybody experience that? So yeah, I think that's honestly going to be one of the biggest skills in this transformation to AI that you can have as an individual. is understanding change management and how can you advance these kinds of things in your organization. All right, how many people feel like this? Like maybe I'm behind in the AI race? Yeah, we have a lot of that. Maybe you think other people are doing AI better than you. A lot of companies are kind of, with all the news cycle and everything, this FOMO is happening and they think that they need to run to this stuff and they think they're behind and in reality they're not. I was talking to a CIO from a Fortune 500 company a few weeks ago, a month ago, and their executive team was saying, oh my gosh, we're so far behind. We're so far behind.

mit study pilot failure 94 percent scaling challenges value demonstration

So he said, you know what? Who's ahead? And they said, well, China's ahead. And so the CIO I was speaking to said, that's perfect. Let's go visit China. And they went and visited China, and they found out that, you know, the businesses there in China, The businesses there in China are actually not as far ahead as they thought. And so they're actually on pace with a lot of the other companies that they were worried about outpacing them. So if you're feeling this way, you're not alone. So that's one thing that leads to the AI anxiety. Right? Anybody ever seen this? Like, hey, we've been putting all this AI stuff in place and we're not really getting any value out of it. How many of you have seen the MIT study that was all over the place? What was the MIT study? What did it say? Ninety-five percent. I think it's ninety-four percent, but you know they... say like 97% of statistics are made-up on the spot anyway. So I believe it was 94%.

agents agentic workflows complexity simplicity first incremental approach

They said 94% of pilots don't show any value and don't get scaled up into real programs. So that's a problem, right? Especially when we're sitting there being told we should be using AI. So this is another thing that adds to the stress, right? And what about this? Agents and workflows and agentic workflows, oh my, like there's so many different things. How many people have heard agents 47 times today, right? Agentic workflows, and that creates the FOMO too, right? Oh my gosh, we need to be doing agentic workflows. Why? Well, because it sounds really cool, right? So we're going to talk through these things today. And if you take a look at my slides, you'll notice they're pretty simple. And there's a reason for that. Because we like to focus on simplicity. I find starting things simple, simply, and building on simple things and making them more complex over time is usually an easier way to get traction.

lean tools hypothetical example methodology simple approach process improvement

It's an easier way to get buy-in. It's easier to understand for people. So what we're going to do today is we're going to start pretty simply. And we're going to take some lean tools And we're going to walk through a hypothetical example that's not super hypothetical. I think you'll all get it. You've been there. And we're going to walk through how we can determine where we can put AI into this mix and improve our organization. Okay, so I want to go through now, I'm going to talk through just what each of these things are. And these are my definitions. There might be other folks in the room that have different definitions, but that's okay. These are the definitions we're going to use for today. A workflow. A workflow is a predefined set of steps that you execute more or less in order. You know, if you've ever been in a business environment, everything that's shown as a straight line is never in a straight line. There's always all kinds of back channels and things, but more or less straight line.

workflow definition agents agentic workflows guardrails active directory

Okay, so it's there for a single outcome. An agent. So this basically takes it one step further, and you tell the agent this is what the outcome has to look like, and you tell it just go do the thing by itself until it achieves the outcome. This is where some folks get in trouble. Like we were just talking at lunch earlier about the person that had the agent that you may have seen on the news that dropped their entire production database, because they gave it the goal, and it was just trying to figure out a way to get that done. Or maybe this was probably closer to a year ago. Did you hear the one, it's almost like telling you, did you hear the one about, did you hear the one about they've tried to put good guardrails and said, you cannot move forward unless you get a decision from Sarah. And I told the agent that. You guys heard this one? It was great. Sarah wasn't getting back to the agent, so the agent went into Active Directory and it renamed somebody Sarah, and then went about its past because it got the yes answer it was looking for.

workflow focus complexity management implementation strategy process over technology incremental learning

So there's some stuff that can happen in here that are quite comical and scary. Finally, agentic workflow, this is where you have multiple agents executing on workflows, possibly multiple workflows, and you typically are going to have another agent that orchestrates over the top or an orchestration layer across the top. Now, as you can imagine, each one of these, the complexity gets higher, right, as we move down that stack. And I told you, I'm a pretty simple person, so I don't want to talk about agents or agentic workflows today. Let's just focus on workflows. Because that's easy for me. It's simple, but we can still make a difference in our organizations. Once you get workflows down, then maybe you start thinking about agents. How can I implement an agent to do this workflow and have it even more autonomously? But I'm more interested in the process than the actual implementation.

contract review project onboarding master services agreement statement of work legal documents

So Let's just go with an example. How many people have ever read a legal document in your life? Keep your hand up if you enjoy it. We got a couple. That's good. That's good. We need everybody. We need people like that. So the example I'm going to go through is reducing project onboarding time. And so what happens is, in this hypothetical example, what's happening is you got a new project, you probably need to sign like a master services agreement with whatever company you're working with. On top of that, you've got like statement of works, things like that, you have to get signed, and all the paperwork process that you need to get through that particular, get into that project, correct? How many people have been in at least some Length. OK, not as many as I thought. So first thing we're going to do is we're going to do a traditional A3.

a3 process problem statement lean methodology background analysis client experience

And so in Lean, A3 process is basically, it's named after the size of paper they do it on. And it's basically just these boxes, and you answer all of these questions. And what the goal is to frame the problem statement. And I'm not going to go through all these because that would be incredibly boring. But the first box here is the main thing that I want to talk about, the background, right? First 2, I should say. So it's the first experience clients have with us as an organization, right? So they're going through and this is how we really operate. So we want to put our best foot forward in this and we want to, you know, provide a good experience there. But it's also long. There's a lot of back and forth. If you've ever negotiated a contract before, it's not exactly one and done in a lot of cases. And the longer this takes, the longer we can get going on the project, which delays us getting paid.

bottlenecks limited capacity error-prone document review resource constraints

And last I checked in businesses, getting paid is important. So we like to get that done as quickly as possible. Now, the Current condition here, we have limited availability of people. Like if you're in one of these organizations where there's a lot of contracting and things, there's a lot of people that are dependent on the contract, but there typically tends to be a couple of people only that are the ones reviewing contracts. So it creates a bit of a bottleneck for those couple of people, especially if you have a lot of new contracts coming through. And then not to mention, we had a couple of hands raise their hand about enjoying reading legal documents somewhat? Well, most people don't enjoy it. So even the people that are assigned to read the contracts, they have a tendency to browse them quickly because they don't want to actually sit down and read them thoroughly. So there's some error-prone practices in here as well, right?

goal setting problem definition alignment cycle time a3 completion

So our goal is to reduce the time it takes to go from contract or discussion of contract to get everything signed and cash in the door. Does that make sense? So it's just a good old-fashioned A3 lean problem statement. But I think a lot of times we don't really know in organizations what problem we're trying to solve when we get told, implement AI. So I think it's a good practice to go in, do your root cause analysis, do your 5 whys, pop the why stack, whatever you want to call it, Get your A3 set up right, and then it gives your teams and the folks that you're working with something to align around, like this is our goal in implementing AI. Next, I like to use a fishbone diagram. And here, different people do these slightly differently, but these are the six areas that we are identifying problems in.

fishbone diagram root cause analysis specialized knowledge workflow standardization audit trail

So now we've got our general problem, but then if we think about how does this problem show up in the people side of things? and you can see a lot of pretty lengthy text, specialized knowledge required to actually read these things, right? Process, lack of audit trail, no standardized workflow. So we've seen in companies that like, well, who's the person that's supposed to review the contract? Like, and say, well, Bob is, but Bob's busy for the next X days. And so they say, well, who else can I send it to? Because I want to get this across the finish line. So now you have inconsistency across whoever happens to be available as the one that happens to review the contract, right? So now, no clear guidance. I guess that gets into the policies, the process of policies, like what's acceptable and what's not. So you could see all of these things kind of, they give us a little bit of the scope of what challenges are we going to face or what things do we need to look at when we look at our new process.

value stream mapping lead time wait time processing time value-added activities

Now, here's my value stream map. This is the next tool I'm going to use. So I've got my problem statement and then a little bit more detailed analysis of the problem. And I like to turn to value stream maps. You could probably imagine, since I'm a nerd, I had AI do all these images, and I did notice that it inverted the top line here. This guy here is supposed to be on the bottom, and this guy here is supposed to be above the line, but hey, just going to drive me nuts. So this is the project we're talking about, or the workflow we're talking about. Now, could you do this? with a standard business process map, a standard flow chart? Yeah, you could. Why do I like value stream maps for this kind of stuff? Because when we start looking at a value stream map, the way value stream maps work, you can see each step, MSA co-creation, MSA signature, legal review, all the way across.

value stream components queues wait times roi measurement process mapping

Those are the value-added activities that we have. in this value stream, because it's important that we co-create the MSA. It's important that we get the signature. Then when you look at the buckets in between, those are the queues in between, and this is the wait time between steps, right? And so what I like about this, when it comes to especially starting to talk ROI, if we actually have a value stream analysis, we can see the total lead time in this process, is 30 minutes if it's a slam dunk new SOW that all the terms are exactly the same as the last one, to five weeks when we start adding in all the delays, waiting on the customer, waiting on the signer, all the waiting, right? And you can see processing time on five weeks is like 3 days, and then the wait time is the rest of it, right? So I like these from the perspective of they give us the foundation for improvement.

workflow selection coordination costs regulatory considerations team boundaries starting point

Now, do they have to be perfect? No, they don't have to be perfect, right? But if you can get a good idea of how long each of these things step, or each step takes and ranges, then you can at least have a foundation for going and having a discussion on, hey, we did improve things. If you have just a standard business process model or a standard flowchart, you're not thinking about What are the improvements that we can make on each one of these processes much? You've got the time here. Does that make sense? Now, this is important when you're getting started, I think. The question becomes, what workflow do you start with? What do you think? What are the issues that you might run into, depending on what workflow you start with? Any thoughts? Highly regulated. Highly regulated, yeah, that's a good one.

msa co-creation contract negotiation payment terms governing law collaboration

Downstream impact. Downstream impact, that's another good one. Another thing I see a lot, how many different departments or different areas of the business are you touching? Because as you can imagine, the more different areas that you're touching when you're trying to do this, the higher the coordination costs are, right? And trying to get five different departments on page to automate a workflow using AI is a lot harder than getting your own team aligned on that, right? So I think to get started, I would pick something that you feel like is going to be impactful, but probably stays within your team as you learn and as you grow and you start learning the techniques, right? because that's going to make it easier and you're not fighting a lot of other teams and a lot of other ideas. You can control the situation a little bit better. Now, so let's start going through this process. So MSA co-creation.

internal review resource constraints multiple roles review delays legal assessment

So for us, this is where in our organization, this is where we would sit with the customer and we would say, okay, Whose paper do you want to use? Do you want to use our paper or do you want to use your paper as a starting point, right? And let's say for the sake of discussion, we're using their paper. Well, we're going to read through their paper and we're going to say, you know, things like, you know, is there, I'm trying to think of things that we have in ours, like what are the payment terms according to the master services agreement? Are they favorable for us? Do we like them? Do we want to change them? Those kinds of things, right? And then we might say, the governing law is California, and we know that's really difficult if there's a dispute. So maybe we try to negotiate a neutral state into there, things like that. So those are the things that we're looking for, and we're working with, going back and forth with the customer on that, right?

attorney review legal counsel response time delays external review

And so that's a very collaborative process. Finally, we get that figured out. And then we have to send that MSA off to somebody within our organization for review. Right? So now this zero to five days, we get this MSA created with the customer and we ship it off to our person for internal review. Well, we don't have a dedicated legal in our organization. So that means the people that review these contracts are also people that have other jobs. Anybody have situations like that where maybe people wear multiple hats in your organization, right? So that creates delays because they're busy, they can't get to it right away, those kinds of things. So we've got up to a week delay in getting that thing reviewed. Now, MSA signature, what are we doing here? we're going and we're saying, okay, based on my review of this, we are saying, I think all of this looks good, except there's this one clause in there.

sow review docusign customer signature finance notification document workflow

One clause that I don't know. I think we're going to have to get that reviewed by an actual lawyer, because that's something we haven't seen before, right? So then they go in and they say, hey, we're going to need legal to look at it. So they send it off to the attorney, and now Zero to 10 days later, how many people know responsive attorneys? Because I'd like a referral. I might talk to you later. Attorneys tend to be busy, too, and they tend to take a little bit of time. So we got to wait there. Now, legal review, I almost just put 0, 1 to 2, because, you know, like, if they take a look at it, you're getting billed an hour anyway, right? So, but we'll just say 0 to 2 hours. So if there was nothing nothing funky on that MSA. We're going to skip the legal review and we're going to sign it in that MSA signature block. Then we have the SOW that's attached to that.

process validation common workflows ai suitability step analysis improvement opportunities

Once again, those are a lot shorter typically, not as much language in them. It's like, here's what we want, here's the terms we want it on. Then Zero days to five days now, we send it, we put it into DocuSign, we send it back, right? And then the customer, whenever they get around to signing it, which we don't have a lot of, it depends on how urgent they are to get the project done, then we wait on them some more, right? They sign it in DocuSign, then we have to notify finance, and then we get to the end of the process. Does that seem like a fairly common type of process you might see in your organizations? Fairly simple, right? Nothing earth-shattering here. All right, so what do we do? We've got our current state. We've got our problem statement. We've got some of the things we want to look at. So next step, I'm going to start going through each one of these steps individually. And what I want to start doing is I want to start saying, okay, now, can AI give us a lift on each one of these steps?

suitability framework cost of error task analysis generative ai risk assessment

Okay. And the way we tend to do that, like I said, I like things simple. I guess I should clarify, we're really talking about Gen. AI here. I'm talking mostly about Gen. AI. So we start looking at suitability for task. And so I've got this little 2 by two here, and you can see on the y-axis we have cost of error. Is this a low stakes problem if something goes wrong? or is it a high stakes problem if something goes wrong? the classic example is like cancer diagnosis, you hear people talk about a lot, right? Low stakes versus high stakes. If it's a marketing e-mail, my apologies to the marketing folks in the room, but most of those are probably lower stakes. It may do some brand damage, but nine times out of 10, they're okay. Then on the x-axis, we have the nature of the task. Is it nuanced or is it algorithmic? Is it Probabilistic, or is it deterministic?

msa co-creation high stakes nuanced work human-led fighter pilot metaphor

OK, and what we're going to do is we're going to plot each one of these steps in one of these quadrants. OK, so MSA co-creation, where do you think that goes? Is that high stakes or low stakes? Yeah, I'd say high stakes too. By the way, a good consulting answer is it depends. So if you ever don't want to answer, but pretend like you're contributing, do like us consultants do, just answer everything with it depends. Yeah, so high stakes. And then is it algorithmic or is it nuanced? I'd say it's pretty nuanced, right? What are you doing? What the... Well, this is nice.

human control negotiation ai agents unpredictable outcomes no ai intervention

Now maybe? Hey, okay. So yeah, so MSA creation, I put up here in the fighter pilot category. And I name these categories. I don't know if they're good names, but I was inspired by Microsoft with the copilot thing. I don't know. I feel like fighter pilots probably, I bet AI eventually will be pretty darn good at like dog fighting and things like that. But right now, if you've ever used ChatGPT or Claude or any of these tools and you put something in it and it says like canoodling or whatever and it sits there and spins a while, I don't think that'd be good in a dog fight. So for now, we're going to use the metaphor fighter pilot. But we want this task to be human-led. We don't want AI in there, right? So if you're sitting with your client, imagine your client has an AI that does the MSA co-creation and you have an AI that does MSA co-creation.

copilot quadrant msa signature algorithmic tasks ai assistance human oversight

And they're both talking to each other. Heavens knows what's going to come out the other side, right? Like, I don't think we really want that. We want the control in this particular case. So in this case, I think we said it was zero to two days for that task. It's probably going to stay zero to two days because we don't want to introduce AI into this. So we take the next step is the signature. Here we have high stakes because we're having to agree to that. abide by the terms of the master services agreement. So it's high stakes, but a signature is a signature. You just go in and, you know, like it's pretty routine. Companies tend to look for the same things every time in the, you know, just to make sure those boxes are checked. It's fairly algorithmic in nature. So here we have the, this lands in that copilot quadrant. So here it's like high enough stakes that we want a human to look at it. but low enough stakes that we're comfortable with AI taking a shot at it, right? So AI can review it, give me a list of things that it sees and notices for consideration, maybe classify each step as like showstopper, take a closer look, never seen this before, who cares?

legal review sow review human-led copilot tasks billable hours

Something like that, right? All right, next step, go back to legal review. Again, I want a lawyer that I'm working with, if I've seen something new, I want to have a discussion with them about it so I understand the implications about it. I don't want my AI to go talk to my lawyer and just keep asking questions and then keep asking questions and running up our bill rate, right? I want that to be human-led too. So our legal process, our legal step in here probably isn't going to change much because we want it to be human-led. Next step, SOW review. Again, just like the last one, SOWs tend to be pretty standard once we have the type of work, whatever we're going to invoice them for. And so this is up here in that copilot space, too. So far, we've seen two steps of this workflow that we feel like AI can help us out with, and two steps that probably shouldn't, are probably bad ideas, right?

autopilot docusign finance notification scripting deterministic processes

So what's after that? Customer signature. Okay, so customer signature, this is where we sent it over with DocuSign and we did all that stuff. This lands in the autopilot zone. It's pretty algorithmic. You put it in DocuSign, you send it over, they click through the DocuSign, click sign, sign, sign, sign, do all the signing, and then they click it back, and then you get it back and you file it whatever you're going to, that's pretty much just a standard process, right? And so over here, We put that in the automate space. So this we could basically, we'll talk about this in a second, but we could basically have AI just go do this. Okay, but we'll get back to how we do this next. notify finance. We have a certain set of steps that we have to do. We have a form that we have to fill out. We have to do this stuff, send it over to finance. Finance gets the project created, does all the back office stuff.

pre-mortem test pilot scenario analysis exploratory tasks low risk

Again, We're over here in this automate space because it's pretty low risk. Documents already signed, everything. Just kind of going through the motions at that point. Any questions up to now? Cool. All right. Nature of task, test pilot, I mean, test pilot was where I was going to next. Notice we didn't have anything in there. Low risk and nuanced or probabilistic, I'm going to throw out there pre-mortems. So these are the things that you might do. Is anybody familiar with the pre-mortem? How many people have done a post-mortem before? Yeah, those are fun. Pre-mortem is the opposite. So maybe what you can do with a pre-mortem is you can go in and say, okay, well, here's the MSA. What happens if, and you can do some scenario stuff, and maybe you can explore and decide how comfortable you are with different terms in it.

context depth proprietary data technical complexity implementation matrix llm training

So these are more of your one-off things that you just kind of want to try. They're nuanced, low risk, so we don't have to worry about if it gets it wrong, because there's human judgment there. Okay. But now, I guess I'll go back to this real quick. We've got this, but now what do we do with this? That's the next step. Okay, how do we implement this thing? And here, consultants love two by twos, so you get tons of them in here. So here we have context depth on the left on the y-axis. And this is really like general information or does this require a lot of proprietary information? Right? Because if you think about it, all these LLMs are all trained on the general internet. So they're okay at all of the stuff that requires general knowledge because they've been trained on all of that. But now the more stuff that they need to understand that's data from your organization or maybe it's your industry specific, very detailed private information, the more of that there is, these general LLMs, generative, they They just aren't going to know that.

domain experts proprietary knowledge skills tools msa implementation

So they're going to make more mistakes, probably hallucinate more, that kind of stuff. Technical complexity, no code versus pro code. And we'll talk about this in a little bit. If it's high complexity, you probably need a code-based solution. Somebody's going to have to write some software if it's super high complexity. If it's low complexity, you could probably get away with prompting and other techniques like that to do the implementation. So let's go through the same steps here again. MSA signature. So I landed this, you know, the proprietary data, only we know what information that we are looking for and what we want in our MSA. That's proprietary. The general internet doesn't know what we care about, right? So that's up here in the proprietary data section. Technical complexity, I don't know what your MSAs look like, but our MSAs really share a lot of language from a lot of MSAs.

skills prompting contract review insurance verification jurisdiction checking

You wouldn't be surprised or would be surprised if you choose, like, how many times do you get a legal document that you notice somebody else's name because they just copied the last document and tried to change everything, right? So yeah, so not really a super highly technical situation, but but does require some of our proprietary information. So this is the realm of what I call domain experts, right? So you have somebody in your organization that understands what they're looking for in the MSAs, understands that particular structure, what you want, what questions you want to ask. And what you're going to do here is, how many are you using Copilot? Okay. ChatGPT? Okay. Other? Claude for the win. By the way, Claude. Anyway. So here you're going to use tools, you're going to use skills in order to implement this. So if you're not familiar with skills, that's outside of the scope of our talk today.

sow review template validation sla checking automated flagging version control

But skills are basically you give the LLM instructions on how to handle things. So you can write a skill that says, read in this MSA, look for the state that disputes would be settled in. look at this website or this data that says these are the states that are really hard on companies in litigation and don't let it, you know, raise a flag if we're on any of these tough states, right? Or we want our insurance requirements. Here's a certificate of insurance. Make sure our certificate of insurance matches what they're requiring in the MSA. And if not, make a note, tell us this is what they expect, this is what we have, and we can make a decision, right? So that's the domain expert side. Reviewing SOW, same thing. Maybe you need like SLAs or something in the SOW, and a lot of times, especially with older clients, maybe you've got a client and somebody's doing that copy and paste thing, and they're copying and pasting the next the last one, and you've updated your general template for SOWs, and maybe the SLAs are now wrong, something like that, right?

deterministic tasks scripting zapier token waste cost efficiency

So now you can have a skill that when it gets to that SLA step, it goes ahead and flags all of those things for you. Customer signature. So put this on the power user. So customer signature, this one is, you know, pretty straightforward process. But you might need to be able to automate, like using Zapier or any of these, like, or maybe if you're technically intrigued enough to use like Claude Code or copilot to write your own software to automate these things. But the thing about these things, remember they were fairly deterministic. So a good way to burn a lot of tokens and spend money is doing things that should just be a script through AI. Right? So I would not use AI on these. I would just create a script for these. So you see here we're saying like, hey, in the automation side, we don't need, we're still going to improve our process, but AI isn't going to be our solution.

engineering complexity demand forecasting human-only activities msa co-creation no ai needed

So that's the answer there. Next, notify finance, that's the same. Right? Our pre-mortem, over here, just use the AI tool. And then the engineering I skipped because this is when you have a pretty highly complex situation. we're doing a model where we're doing demand forecasting for beverages and things like that. That's going to be a much more complex task. So our legal process doesn't need any of that. Did you notice anything else missing in here? MSA co-creation isn't in here. Why is MSA co-creation not in here? Yeah, we said this is a human-only activity, right? So we only need to, when we're thinking of implementation, the implementation model for human-only activities is humans, right?

improved results reduced wait times automation benefits skills impact value stream improvement

Talking to humans. So let's do that. Any questions so far? Is this all making sense? All right, so now we come back. and I'm not going to spend too much time on this, but we can see after we made those changes, some of the things that I circled have changed. The automation on the top reduced the delay significantly because we automatically sent an e-mail and we automatically created a DocuSign and things like that. So we're able to really crunch that down. And the wait time, if you recall, was a big part of the overall time. So the more we can reduce the wait time, the better. Then the SOW review, because we had all of those things listed out in our skill that said these are the things you must look for in this SOW, and tell me, rate those, and give me feedback on those, we were able to cut that down significantly.

process efficiency 67 percent improvement quick implementation connectors mcp

Same with the general review of the MSA, that also cut down significantly. So if you look down now at the process efficiency, we went from 12% to 20%. And you might be like, so what? But that is a 67% increase on a couple of really simple tests. You should be able to probably implement most of this in a matter of a few hours, right? What we did do is I would probably, if I were doing this for real, I'd reimagine the value stream. Right now, we just copied what we're already doing today, but with AI, we can do a lot more. I'd probably reimagine the value stream. I'd probably combine like SOWs and MSAs into one step, so then there's no hands-offs. I'd probably, you can connect If you aren't familiar, there's a thing called MCPs. That's a technical term, but they're typically called connectors and things like that in Claude and in ChatGPT.

incremental improvement starting small connectors mcp actions

I think they're actions in ChatGPT. Can't remember all the details, but I'd put connectors into DocuSign, and then it can go back and say, oh, hey, I'm done with this. Send it for signing. And so then we don't even have to wait. for the review. So there are ways that you could do this, make even more impact. But I think it's more important to start small and grow into it than trying to go big with your first couple of shots. Okay, last lean tool, by the way, for the non-lean folks, the implementation model and the AI suitability, those are not lean tools. But last lean tool I want to talk about today is PDCA. That's plan, do, check, act. And it's basically your basic continuous improvement process, right? Start by look at the problem, find opportunities. That's what we did when we created our A3. Then we go and we do a first pass. We do the do, and we implement some changes.

pdca continuous improvement model drift evaluations evals

And then the thing I wanted to hit on this slide, check changes a little bit in AI. So you need to be tracking the results. You need to be tracking how much time you're saving. But the thing that I don't think people are thinking about a lot when they're doing a lot of these custom AI solutions, or not custom, but AI solutions using skills and other things like that, the speed at which the LLMs change is rapid. You hear of a new model coming out all the time, right? Well, if your skill that reads MSAs is working okay this time. Next time a new model might come out and it will behave completely differently. So what you really need to do, in the tech world we call them evaluations or evals. So you really need a process that you come back and you check on the results of these things every once in a while.

regular evaluation skill drift conversational memory performance monitoring consistency checks

Have you ever noticed in ChatGPT, Copilot, Claude, when you're typing about something and it pulls up something that you talked about like 3 weeks ago and it says, oh, like you were saying, you go, oh, wait, that was a long time ago. Well, that causes drift. Okay, so that causes drift. So your MSA, you might interact with this MSA, and over time, those interactions are affecting your output. because it remembers some of those conversations. So it's good, but it's also bad because it could cause your model to drift or your skill to drift and not provide the same results it used to. So you want to continuously be checking these things on a fairly regular cadence to make sure that they're still behaving as well as you'd want them to behave. Now, next step, agents, right? So get this working, do a couple iterations. When you feel confident, then you can start talking about agents.

agents roi demonstration cycle time cost reduction iteration

Until then, I think just the workflows is a good starting place. And you can go back and say, hey, it took me 3 hours. We had a 60 plus percent improvement on cycle lead times. And we reduced the cost of monitoring or reviewing MSAs and SOWs by X percent, 50 percent or whatever add up there, right? So real simple way to start getting actual realized value out of AI rather than just a bunch of messing around. If you want, got a couple minutes for questions, but if you want, e-mail me. I think I had a couple of requests from other people that I've given this talk at to do like a workshop. If you're interested in actually sitting down and going through this process with some of your workflows or one of your workflows, let me know. We were planning on doing it like early June. Just shoot me an e-mail and I'll add you to the invite.

workshop offer practical application free training audience questions session closing

Not a paid thing, just a, it's real easy to talk about. It always looks easy on the screen, right? But it's not necessarily always easy in practice. Questions? That was either incredibly useless or I'm such a great speaker, everybody completely understands how to take this home. Thank you all. Good job. Thanks.