1 00:00:00,792 --> 00:00:11,112 So our next speaker is Brandon Carlson, founder of Lean Techniques, a Des Moines-based consultancy focused on digital transformation, IT modernization, and software delivery. 2 00:00:11,432 --> 00:00:22,552 With more than 30 years of experience in technology, Brandon is known for his practical people-first approach, emphasizing curiosity, collaboration, and meeting organizations where they are. 3 00:00:22,952 --> 00:00:36,472 In today's session, Brandon will explore how lean thinking can help move AI from isolated experimentation into real, measurable business value with practical examples across marketing, operations, compliance, and sales. 4 00:00:36,672 --> 00:00:38,232 Please join me in welcoming Brandon. 5 00:00:38,232 --> 00:00:44,632 All right. 6 00:00:44,712 --> 00:00:47,032 Thanks, everybody, for being here. 7 00:00:47,032 --> 00:00:48,712 Thanks for the nice intro. 8 00:00:49,832 --> 00:00:51,752 Really don't like this microphone thing. 9 00:00:51,752 --> 00:00:52,792 It's not my style. 10 00:00:54,152 --> 00:00:57,912 But as she said, my name is Brandon. 11 00:00:58,312 --> 00:01:00,792 Hope everybody's having a good conference so far. 12 00:01:01,192 --> 00:01:01,512 Good. 13 00:01:02,552 --> 00:01:04,392 You're going to take some stuff home already? 14 00:01:05,672 --> 00:01:05,832 Good. 15 00:01:05,832 --> 00:01:06,952 Then I won't let you down. 16 00:01:07,912 --> 00:01:15,432 So I like to say if this is the only lean and AI talk you've ever been to, it's likely to be the best you'll ever see. 17 00:01:18,672 --> 00:01:21,112 How many people are lean practitioners? 18 00:01:22,312 --> 00:01:23,832 Okay, a few of you. 19 00:01:25,352 --> 00:01:27,912 So how many people are using AI? 20 00:01:29,032 --> 00:01:29,752 More of you? 21 00:01:29,752 --> 00:01:31,432 That's kind of shocking, really? 22 00:01:31,432 --> 00:01:31,752 No. 23 00:01:33,312 --> 00:01:36,872 Okay, are you using AI in systematic ways? 24 00:01:36,872 --> 00:01:42,552 Or are you kind of, you know, like, oh, I need to draft an e-mail, I need to explore this topic, I need to do this or that. 25 00:01:42,552 --> 00:01:44,552 How's everybody using it today? 26 00:01:44,552 --> 00:01:45,992 Just have a couple of people throw out. 27 00:01:46,152 --> 00:01:46,952 Sporadic? 28 00:01:47,512 --> 00:01:48,632 or systematic? 29 00:01:49,272 --> 00:01:50,552 How many people are systematic? 30 00:01:51,512 --> 00:01:53,032 Okay, sounds good. 31 00:01:53,592 --> 00:02:03,992 So this is kind of an interesting talk in my opinion because we are really talking today about kind of new meeting old. 32 00:02:05,192 --> 00:02:08,632 So like Lean has been around for a long time, right? 33 00:02:08,632 --> 00:02:13,192 Originated in Toyota Production System, Lean Manufacturing, those kinds of things. 34 00:02:13,592 --> 00:02:26,072 And I think what's important with all this change that's going on and all the craziness that we're dealing with, it's really important to understand that these old tools are still valuable and they still work, right? 35 00:02:27,112 --> 00:02:30,072 Is this speaker like squeaking and stuff or is it just me? 36 00:02:30,792 --> 00:02:31,512 You have the feedback. 37 00:02:33,272 --> 00:02:34,872 Like, how do I... 38 00:02:34,872 --> 00:02:37,032 like to walk around, so this is going to be trouble. 39 00:02:37,672 --> 00:02:38,152 Okay. 40 00:02:39,192 --> 00:02:52,472 So what we're going to try to do today is we're going to try to help you all see a process that you can take home and use today to start implementing more AI within your organization and create a measurable impact from it. 41 00:02:53,912 --> 00:02:57,512 My little tagline here is just rub a little AI on it. 42 00:02:57,752 --> 00:02:59,112 That's what we hear a lot at work. 43 00:02:59,112 --> 00:03:01,672 And they're like, well, can we just AI this? 44 00:03:02,232 --> 00:03:04,312 Can we get 30% improvement with AI? 45 00:03:04,792 --> 00:03:05,992 Just rub some AI on it. 46 00:03:06,472 --> 00:03:06,872 So 47 00:03:07,672 --> 00:03:09,672 That might work, but it's not very effective. 48 00:03:09,672 --> 00:03:12,152 So what we'll do today is talk about how to make it more effective. 49 00:03:13,352 --> 00:03:14,872 Okay, so who's this talk for? 50 00:03:15,752 --> 00:03:19,272 This talk's for people who think good enough isn't enough anymore. 51 00:03:19,832 --> 00:03:25,032 It's for you if you're a leader that you don't want to keep doing the same things, you want to get better and you want to improve. 52 00:03:25,912 --> 00:03:28,552 And it's also for you're a team member. 53 00:03:28,952 --> 00:03:35,272 How many people in this room have been told by somebody that thou shalt be using AI and making things better? 54 00:03:35,672 --> 00:03:36,712 We see that a lot. 55 00:03:37,752 --> 00:03:39,432 People are coming to us all the time now. 56 00:03:39,472 --> 00:03:40,312 What do I do? 57 00:03:40,312 --> 00:03:47,592 I've been told I have to do this, and we've got these OKRs or KPIs or choose your own adventure, their metric of choice. 58 00:03:48,952 --> 00:03:49,432 Now what? 59 00:03:51,992 --> 00:03:52,712 So that's what Lt. 60 00:03:52,712 --> 00:03:53,432 does. 61 00:03:53,672 --> 00:03:59,352 We transform technology from keeping the lights on to leading the way in organizations. 62 00:03:59,752 --> 00:04:01,272 So we're a transformation company. 63 00:04:01,272 --> 00:04:03,272 We do a lot of change management. 64 00:04:03,832 --> 00:04:10,792 We'll talk probably a little bit about this depending on time, but when we go to roll out some of these things, how do we get those changes to actually stick? 65 00:04:11,392 --> 00:04:16,472 Because that's probably if you have tried to operationalize some of this stuff, you've probably run into that. 66 00:04:16,472 --> 00:04:18,312 Well, we did all the things and nobody uses it. 67 00:04:18,632 --> 00:04:19,832 Anybody experience that? 68 00:04:20,472 --> 00:04:28,152 So yeah, I think that's honestly going to be one of the biggest skills in this transformation to AI that you can have as an individual. 69 00:04:28,792 --> 00:04:33,672 is understanding change management and how can you advance these kinds of things in your organization. 70 00:04:35,512 --> 00:04:37,192 All right, how many people feel like this? 71 00:04:37,192 --> 00:04:39,792 Like maybe I'm behind in the AI race? 72 00:04:39,792 --> 00:04:42,632 Yeah, we have a lot of that. 73 00:04:43,432 --> 00:04:45,832 Maybe you think other people are doing AI better than you. 74 00:04:47,832 --> 00:04:57,752 A lot of companies are kind of, with all the news cycle and everything, this FOMO is happening and they think that they need to run to this stuff and they think they're behind and in reality they're not. 75 00:04:58,192 --> 00:05:10,072 I was talking to a CIO from a Fortune 500 company a few weeks ago, a month ago, and their executive team was saying, oh my gosh, we're so far behind. 76 00:05:10,072 --> 00:05:10,952 We're so far behind. 77 00:05:10,952 --> 00:05:11,912 So he said, you know what? 78 00:05:12,312 --> 00:05:12,912 Who's ahead? 79 00:05:12,912 --> 00:05:14,752 And they said, well, China's ahead. 80 00:05:14,752 --> 00:05:19,632 And so the CIO I was speaking to said, that's perfect. 81 00:05:19,832 --> 00:05:21,192 Let's go visit China. 82 00:05:22,352 --> 00:05:27,272 And they went and visited China, and they found out that, you know, the businesses there in China, 83 00:05:28,472 --> 00:05:32,712 The businesses there in China are actually not as far ahead as they thought. 84 00:05:33,112 --> 00:05:39,192 And so they're actually on pace with a lot of the other companies that they were worried about outpacing them. 85 00:05:39,752 --> 00:05:41,992 So if you're feeling this way, you're not alone. 86 00:05:42,952 --> 00:05:45,432 So that's one thing that leads to the AI anxiety. 87 00:05:46,872 --> 00:05:47,272 Right? 88 00:05:49,272 --> 00:05:50,312 Anybody ever seen this? 89 00:05:50,312 --> 00:05:55,592 Like, hey, we've been putting all this AI stuff in place and we're not really getting any value out of it. 90 00:05:56,752 --> 00:05:59,992 How many of you have seen the MIT study that was all over the place? 91 00:06:00,272 --> 00:06:01,392 What was the MIT study? 92 00:06:01,392 --> 00:06:01,912 What did it say? 93 00:06:01,952 --> 00:06:02,632 Ninety-five percent. 94 00:06:03,192 --> 00:06:05,392 I think it's ninety-four percent, but you know they... 95 00:06:05,552 --> 00:06:08,712 say like 97% of statistics are made-up on the spot anyway. 96 00:06:08,712 --> 00:06:10,992 So I believe it was 94%. 97 00:06:10,992 --> 00:06:19,992 They said 94% of pilots don't show any value and don't get scaled up into real programs. 98 00:06:20,152 --> 00:06:21,752 So that's a problem, right? 99 00:06:22,712 --> 00:06:25,672 Especially when we're sitting there being told we should be using AI. 100 00:06:27,432 --> 00:06:31,112 So this is another thing that adds to the stress, right? 101 00:06:34,272 --> 00:06:35,272 And what about this? 102 00:06:36,152 --> 00:06:39,952 Agents and workflows and agentic workflows, oh my, like there's so many different things. 103 00:06:39,952 --> 00:06:43,672 How many people have heard agents 47 times today, right? 104 00:06:44,072 --> 00:06:47,032 Agentic workflows, and that creates the FOMO too, right? 105 00:06:47,192 --> 00:06:49,912 Oh my gosh, we need to be doing agentic workflows. 106 00:06:50,472 --> 00:06:50,952 Why? 107 00:06:51,432 --> 00:06:54,312 Well, because it sounds really cool, right? 108 00:06:55,512 --> 00:06:58,152 So we're going to talk through these things today. 109 00:06:59,352 --> 00:07:03,672 And if you take a look at my slides, you'll notice they're pretty simple. 110 00:07:04,752 --> 00:07:05,992 And there's a reason for that. 111 00:07:07,592 --> 00:07:10,552 Because we like to focus on simplicity. 112 00:07:10,792 --> 00:07:20,872 I find starting things simple, simply, and building on simple things and making them more complex over time is usually an easier way to get traction. 113 00:07:21,272 --> 00:07:22,872 It's an easier way to get buy-in. 114 00:07:23,592 --> 00:07:25,432 It's easier to understand for people. 115 00:07:26,312 --> 00:07:29,112 So what we're going to do today is we're going to start pretty simply. 116 00:07:30,672 --> 00:07:32,392 And we're going to take some lean tools 117 00:07:32,672 --> 00:07:37,352 And we're going to walk through a hypothetical example that's not super hypothetical. 118 00:07:37,592 --> 00:07:39,432 I think you'll all get it. 119 00:07:39,752 --> 00:07:40,712 You've been there. 120 00:07:41,432 --> 00:07:49,192 And we're going to walk through how we can determine where we can put AI into this mix and improve our organization. 121 00:07:51,032 --> 00:07:57,032 Okay, so I want to go through now, I'm going to talk through just what each of these things are. 122 00:07:57,592 --> 00:07:59,112 And these are my definitions. 123 00:07:59,112 --> 00:08:02,472 There might be other folks in the room that have different definitions, but that's okay. 124 00:08:02,472 --> 00:08:05,272 These are the definitions we're going to use for today. 125 00:08:06,312 --> 00:08:07,192 A workflow. 126 00:08:07,752 --> 00:08:13,192 A workflow is a predefined set of steps that you execute more or less in order. 127 00:08:13,592 --> 00:08:20,312 You know, if you've ever been in a business environment, everything that's shown as a straight line is never in a straight line. 128 00:08:20,312 --> 00:08:23,832 There's always all kinds of back channels and things, but more or less straight line. 129 00:08:24,472 --> 00:08:26,792 Okay, so it's there for a single outcome. 130 00:08:28,312 --> 00:08:28,912 An agent. 131 00:08:29,912 --> 00:08:39,592 So this basically takes it one step further, and you tell the agent this is what the outcome has to look like, and you tell it just go do the thing by itself until it achieves the outcome. 132 00:08:41,352 --> 00:08:43,072 This is where some folks get in trouble. 133 00:08:43,072 --> 00:08:55,432 Like we were just talking at lunch earlier about the person that had the agent that you may have seen on the news that dropped their entire production database, because they gave it the goal, and it was just trying to figure out a way to 134 00:08:56,392 --> 00:08:57,272 get that done. 135 00:08:57,832 --> 00:09:01,192 Or maybe this was probably closer to a year ago. 136 00:09:01,672 --> 00:09:15,912 Did you hear the one, it's almost like telling you, did you hear the one about, did you hear the one about they've tried to put good guardrails and said, you cannot move forward unless you get a decision from Sarah. 137 00:09:17,352 --> 00:09:18,552 And I told the agent that. 138 00:09:18,872 --> 00:09:19,912 You guys heard this one? 139 00:09:19,912 --> 00:09:21,672 It was great. 140 00:09:22,552 --> 00:09:34,152 Sarah wasn't getting back to the agent, so the agent went into Active Directory and it renamed somebody Sarah, and then went about its past because it got the yes answer it was looking for. 141 00:09:34,152 --> 00:09:39,192 So there's some stuff that can happen in here that are quite comical and scary. 142 00:09:41,032 --> 00:09:53,272 Finally, agentic workflow, this is where you have multiple agents executing on workflows, possibly multiple workflows, and you typically are going to have another agent that orchestrates over the top or an orchestration layer across the top. 143 00:09:54,072 --> 00:10:01,192 Now, as you can imagine, each one of these, the complexity gets higher, right, as we move down that stack. 144 00:10:02,432 --> 00:10:08,712 And I told you, I'm a pretty simple person, so I don't want to talk about agents or agentic workflows today. 145 00:10:09,752 --> 00:10:11,912 Let's just focus on workflows. 146 00:10:12,832 --> 00:10:14,152 Because that's easy for me. 147 00:10:14,632 --> 00:10:18,712 It's simple, but we can still make a difference in our organizations. 148 00:10:19,672 --> 00:10:24,952 Once you get workflows down, then maybe you start thinking about agents. 149 00:10:24,952 --> 00:10:29,992 How can I implement an agent to do this workflow and have it even more autonomously? 150 00:10:31,752 --> 00:10:37,032 But I'm more interested in the process than the actual implementation. 151 00:10:38,072 --> 00:10:38,472 So 152 00:10:39,192 --> 00:10:40,632 Let's just go with an example. 153 00:10:42,392 --> 00:10:46,472 How many people have ever read a legal document in your life? 154 00:10:48,632 --> 00:10:50,232 Keep your hand up if you enjoy it. 155 00:10:52,952 --> 00:10:53,832 We got a couple. 156 00:10:53,832 --> 00:10:54,392 That's good. 157 00:10:54,472 --> 00:10:54,872 That's good. 158 00:10:54,872 --> 00:10:55,992 We need everybody. 159 00:10:56,712 --> 00:10:57,832 We need people like that. 160 00:10:59,352 --> 00:11:04,632 So the example I'm going to go through is reducing project onboarding time. 161 00:11:06,232 --> 00:11:17,672 And so what happens is, in this hypothetical example, what's happening is you got a new project, you probably need to sign like a master services agreement with whatever company you're working with. 162 00:11:18,072 --> 00:11:29,992 On top of that, you've got like statement of works, things like that, you have to get signed, and all the paperwork process that you need to get through that particular, get into that project, correct? 163 00:11:30,632 --> 00:11:32,952 How many people have been in at least some 164 00:11:33,592 --> 00:11:33,912 Length. 165 00:11:34,312 --> 00:11:35,752 OK, not as many as I thought. 166 00:11:37,592 --> 00:11:41,432 So first thing we're going to do is we're going to do a traditional A3. 167 00:11:42,632 --> 00:11:50,072 And so in Lean, A3 process is basically, it's named after the size of paper they do it on. 168 00:11:50,672 --> 00:11:54,432 And it's basically just these boxes, and you answer all of these questions. 169 00:11:54,432 --> 00:11:57,872 And what the goal is to frame the problem statement. 170 00:11:58,712 --> 00:12:01,992 And I'm not going to go through all these because that would be incredibly boring. 171 00:12:02,392 --> 00:12:07,912 But the first box here is the main thing that I want to talk about, the background, right? 172 00:12:08,872 --> 00:12:09,992 First 2, I should say. 173 00:12:10,312 --> 00:12:15,912 So it's the first experience clients have with us as an organization, right? 174 00:12:16,392 --> 00:12:19,592 So they're going through and this is how we really operate. 175 00:12:19,592 --> 00:12:25,912 So we want to put our best foot forward in this and we want to, you know, provide a good experience there. 176 00:12:26,792 --> 00:12:27,912 But it's also long. 177 00:12:27,912 --> 00:12:29,112 There's a lot of back and forth. 178 00:12:29,112 --> 00:12:35,032 If you've ever negotiated a contract before, it's not exactly one and done in a lot of cases. 179 00:12:36,552 --> 00:12:43,272 And the longer this takes, the longer we can get going on the project, which delays us getting paid. 180 00:12:43,352 --> 00:12:47,032 And last I checked in businesses, getting paid is important. 181 00:12:47,192 --> 00:12:49,672 So we like to get that done as quickly as possible. 182 00:12:50,872 --> 00:12:52,032 Now, the 183 00:12:53,832 --> 00:12:57,152 Current condition here, we have limited availability of people. 184 00:12:57,152 --> 00:13:09,752 Like if you're in one of these organizations where there's a lot of contracting and things, there's a lot of people that are dependent on the contract, but there typically tends to be a couple of people only that are the ones reviewing contracts. 185 00:13:10,152 --> 00:13:15,512 So it creates a bit of a bottleneck for those couple of people, especially if you have a lot of new contracts coming through. 186 00:13:16,952 --> 00:13:23,032 And then not to mention, we had a couple of hands raise their hand about 187 00:13:23,872 --> 00:13:26,552 enjoying reading legal documents somewhat? 188 00:13:27,752 --> 00:13:30,472 Well, most people don't enjoy it. 189 00:13:30,472 --> 00:13:39,752 So even the people that are assigned to read the contracts, they have a tendency to browse them quickly because they don't want to actually sit down and read them thoroughly. 190 00:13:40,472 --> 00:13:46,072 So there's some error-prone practices in here as well, right? 191 00:13:46,552 --> 00:13:52,792 So our goal is to reduce the time it takes to go from contract 192 00:13:53,512 --> 00:13:57,712 or discussion of contract to get everything signed and cash in the door. 193 00:13:57,712 --> 00:13:59,592 Does that make sense? 194 00:14:00,232 --> 00:14:04,192 So it's just a good old-fashioned A3 lean problem statement. 195 00:14:05,352 --> 00:14:12,712 But I think a lot of times we don't really know in organizations what problem we're trying to solve when we get told, implement AI. 196 00:14:13,912 --> 00:14:21,592 So I think it's a good practice to go in, do your root cause analysis, do your 5 whys, pop the why stack, whatever you want to call it, 197 00:14:22,312 --> 00:14:31,112 Get your A3 set up right, and then it gives your teams and the folks that you're working with something to align around, like this is our goal in implementing AI. 198 00:14:35,592 --> 00:14:38,872 Next, I like to use a fishbone diagram. 199 00:14:39,512 --> 00:14:48,712 And here, different people do these slightly differently, but these are the six areas that we are identifying problems in. 200 00:14:49,192 --> 00:14:56,632 So now we've got our general problem, but then if we think about how does this problem show up in the people side of things? 201 00:14:57,152 --> 00:15:04,152 and you can see a lot of pretty lengthy text, specialized knowledge required to actually read these things, right? 202 00:15:05,512 --> 00:15:08,952 Process, lack of audit trail, no standardized workflow. 203 00:15:08,952 --> 00:15:15,592 So we've seen in companies that like, well, who's the person that's supposed to review the contract? 204 00:15:15,592 --> 00:15:17,032 Like, and say, well, 205 00:15:17,592 --> 00:15:21,832 Bob is, but Bob's busy for the next X days. 206 00:15:21,832 --> 00:15:25,392 And so they say, well, who else can I send it to? 207 00:15:25,432 --> 00:15:27,032 Because I want to get this across the finish line. 208 00:15:27,032 --> 00:15:35,432 So now you have inconsistency across whoever happens to be available as the one that happens to review the contract, right? 209 00:15:35,992 --> 00:15:38,152 So now, no clear guidance. 210 00:15:38,152 --> 00:15:43,192 I guess that gets into the policies, the process of policies, like what's acceptable and what's not. 211 00:15:44,232 --> 00:15:46,632 So you could see all of these things 212 00:15:48,392 --> 00:15:56,392 kind of, they give us a little bit of the scope of what challenges are we going to face or what things do we need to look at when we look at our new process. 213 00:15:59,192 --> 00:16:02,112 Now, here's my value stream map. 214 00:16:02,112 --> 00:16:03,552 This is the next tool I'm going to use. 215 00:16:03,552 --> 00:16:08,872 So I've got my problem statement and then a little bit more detailed analysis of the problem. 216 00:16:09,272 --> 00:16:11,672 And I like to turn to value stream maps. 217 00:16:12,072 --> 00:16:23,512 You could probably imagine, since I'm a nerd, I had AI do all these images, and I did notice that it inverted the top line here. 218 00:16:24,712 --> 00:16:33,592 This guy here is supposed to be on the bottom, and this guy here is supposed to be above the line, but hey, just going to drive me nuts. 219 00:16:34,472 --> 00:16:39,112 So this is the project we're talking about, or the workflow we're talking about. 220 00:16:39,112 --> 00:16:41,112 Now, could you do this? 221 00:16:41,672 --> 00:16:44,792 with a standard business process map, a standard flow chart? 222 00:16:45,992 --> 00:16:47,192 Yeah, you could. 223 00:16:48,552 --> 00:16:51,512 Why do I like value stream maps for this kind of stuff? 224 00:16:52,632 --> 00:17:02,232 Because when we start looking at a value stream map, the way value stream maps work, you can see each step, MSA co-creation, MSA signature, legal review, all the way across. 225 00:17:02,232 --> 00:17:05,512 Those are the value-added activities that we have. 226 00:17:05,992 --> 00:17:10,232 in this value stream, because it's important that we co-create the MSA. 227 00:17:10,232 --> 00:17:12,232 It's important that we get the signature. 228 00:17:12,792 --> 00:17:21,912 Then when you look at the buckets in between, those are the queues in between, and this is the wait time between steps, right? 229 00:17:22,712 --> 00:17:34,552 And so what I like about this, when it comes to especially starting to talk ROI, if we actually have a value stream analysis, we can see the total lead time in this process, 230 00:17:35,192 --> 00:17:47,992 is 30 minutes if it's a slam dunk new SOW that all the terms are exactly the same as the last one, to five weeks when we start adding in all the delays, waiting on the customer, waiting on the signer, all the waiting, right? 231 00:17:47,992 --> 00:17:57,352 And you can see processing time on five weeks is like 3 days, and then the wait time is the rest of it, right? 232 00:17:58,232 --> 00:18:03,272 So I like these from the perspective of they give us the foundation for improvement. 233 00:18:03,592 --> 00:18:04,952 Now, do they have to be perfect? 234 00:18:04,952 --> 00:18:07,752 No, they don't have to be perfect, right? 235 00:18:08,312 --> 00:18:21,032 But if you can get a good idea of how long each of these things step, or each step takes and ranges, then you can at least have a foundation for going and having a discussion on, hey, we did improve things. 236 00:18:22,152 --> 00:18:28,432 If you have just a standard business process model or a standard flowchart, you're not thinking about 237 00:18:29,312 --> 00:18:33,032 What are the improvements that we can make on each one of these processes much? 238 00:18:33,032 --> 00:18:34,072 You've got the time here. 239 00:18:34,992 --> 00:18:35,992 Does that make sense? 240 00:18:37,752 --> 00:18:40,712 Now, this is important when you're getting started, I think. 241 00:18:42,792 --> 00:18:47,752 The question becomes, what workflow do you start with? 242 00:18:49,952 --> 00:18:50,552 What do you think? 243 00:18:51,592 --> 00:18:57,272 What are the issues that you might run into, depending on what workflow you start with? 244 00:19:01,712 --> 00:19:02,272 Any thoughts? 245 00:19:02,272 --> 00:19:02,872 Highly regulated. 246 00:19:03,992 --> 00:19:06,352 Highly regulated, yeah, that's a good one. 247 00:19:06,352 --> 00:19:08,712 Downstream impact. 248 00:19:08,712 --> 00:19:10,552 Downstream impact, that's another good one. 249 00:19:12,752 --> 00:19:18,472 Another thing I see a lot, how many different departments or different areas of the business are you touching? 250 00:19:19,272 --> 00:19:23,832 Because as you can imagine, the more different areas that you're touching when you're trying to do this, 251 00:19:25,472 --> 00:19:27,392 the higher the coordination costs are, right? 252 00:19:27,392 --> 00:19:36,232 And trying to get five different departments on page to automate a workflow using AI is a lot harder than getting your own team aligned on that, right? 253 00:19:36,792 --> 00:19:50,072 So I think to get started, I would pick something that you feel like is going to be impactful, but probably stays within your team as you learn and as you grow and you start learning the techniques, right? 254 00:19:50,552 --> 00:19:56,872 because that's going to make it easier and you're not fighting a lot of other teams and a lot of other ideas. 255 00:19:56,872 --> 00:19:58,872 You can control the situation a little bit better. 256 00:20:01,592 --> 00:20:04,272 Now, so let's start going through this process. 257 00:20:04,272 --> 00:20:07,672 So MSA co-creation. 258 00:20:09,192 --> 00:20:19,272 So for us, this is where in our organization, this is where we would sit with the customer and we would say, okay, 259 00:20:19,912 --> 00:20:20,952 Whose paper do you want to use? 260 00:20:20,952 --> 00:20:24,232 Do you want to use our paper or do you want to use your paper as a starting point, right? 261 00:20:24,792 --> 00:20:28,792 And let's say for the sake of discussion, we're using their paper. 262 00:20:29,032 --> 00:20:43,632 Well, we're going to read through their paper and we're going to say, you know, things like, you know, is there, I'm trying to think of things that we have in ours, like what are the payment terms according to the master services agreement? 263 00:20:43,832 --> 00:20:45,032 Are they favorable for us? 264 00:20:45,032 --> 00:20:45,832 Do we like them? 265 00:20:45,832 --> 00:20:47,032 Do we want to change them? 266 00:20:47,432 --> 00:20:48,632 Those kinds of things, right? 267 00:20:49,152 --> 00:21:00,152 And then we might say, the governing law is California, and we know that's really difficult if there's a dispute. 268 00:21:00,152 --> 00:21:05,592 So maybe we try to negotiate a neutral state into there, things like that. 269 00:21:05,592 --> 00:21:11,032 So those are the things that we're looking for, and we're working with, going back and forth with the customer on that, right? 270 00:21:12,672 --> 00:21:14,472 And so that's a very collaborative process. 271 00:21:14,632 --> 00:21:16,072 Finally, we get that figured out. 272 00:21:16,632 --> 00:21:23,032 And then we have to send that MSA off to somebody within our organization for review. 273 00:21:24,712 --> 00:21:25,112 Right? 274 00:21:25,112 --> 00:21:32,232 So now this zero to five days, we get this MSA created with the customer and we ship it off to our person for internal review. 275 00:21:32,792 --> 00:21:39,272 Well, we don't have a dedicated legal in our organization. 276 00:21:39,272 --> 00:21:44,432 So that means the people that review these contracts are also people that have other jobs. 277 00:21:44,432 --> 00:21:45,112 Anybody have 278 00:21:45,592 --> 00:21:50,232 situations like that where maybe people wear multiple hats in your organization, right? 279 00:21:50,632 --> 00:21:55,672 So that creates delays because they're busy, they can't get to it right away, those kinds of things. 280 00:21:56,072 --> 00:21:59,112 So we've got up to a week delay in getting that thing reviewed. 281 00:21:59,832 --> 00:22:03,272 Now, MSA signature, what are we doing here? 282 00:22:03,792 --> 00:22:11,512 we're going and we're saying, okay, based on my review of this, we are saying, 283 00:22:11,912 --> 00:22:15,992 I think all of this looks good, except there's this one clause in there. 284 00:22:18,072 --> 00:22:20,952 One clause that I don't know. 285 00:22:20,952 --> 00:22:27,272 I think we're going to have to get that reviewed by an actual lawyer, because that's something we haven't seen before, right? 286 00:22:27,912 --> 00:22:33,272 So then they go in and they say, hey, we're going to need legal to look at it. 287 00:22:33,592 --> 00:22:36,232 So they send it off to the attorney, and now 288 00:22:38,152 --> 00:22:42,312 Zero to 10 days later, how many people know responsive attorneys? 289 00:22:42,992 --> 00:22:44,072 Because I'd like a referral. 290 00:22:44,632 --> 00:22:45,672 I might talk to you later. 291 00:22:46,552 --> 00:22:49,912 Attorneys tend to be busy, too, and they tend to take a little bit of time. 292 00:22:49,912 --> 00:22:50,952 So we got to wait there. 293 00:22:51,432 --> 00:23:00,552 Now, legal review, I almost just put 0, 1 to 2, because, you know, like, if they take a look at it, you're getting billed an hour anyway, right? 294 00:23:00,552 --> 00:23:03,552 So, but we'll just say 0 to 2 hours. 295 00:23:03,552 --> 00:23:04,792 So if there was nothing 296 00:23:05,592 --> 00:23:08,152 nothing funky on that MSA. 297 00:23:08,712 --> 00:23:13,432 We're going to skip the legal review and we're going to sign it in that MSA signature block. 298 00:23:14,872 --> 00:23:17,112 Then we have the SOW that's attached to that. 299 00:23:17,112 --> 00:23:21,472 Once again, those are a lot shorter typically, not as much language in them. 300 00:23:21,472 --> 00:23:23,912 It's like, here's what we want, here's the terms we want it on. 301 00:23:25,272 --> 00:23:25,752 Then 302 00:23:26,152 --> 00:23:31,512 Zero days to five days now, we send it, we put it into DocuSign, we send it back, right? 303 00:23:31,832 --> 00:23:43,032 And then the customer, whenever they get around to signing it, which we don't have a lot of, it depends on how urgent they are to get the project done, then we wait on them some more, right? 304 00:23:44,072 --> 00:23:50,152 They sign it in DocuSign, then we have to notify finance, and then we get to the end of the process. 305 00:23:51,152 --> 00:23:55,592 Does that seem like a fairly common type of process you might see in your organizations? 306 00:23:55,992 --> 00:23:56,952 Fairly simple, right? 307 00:23:57,112 --> 00:23:58,712 Nothing earth-shattering here. 308 00:24:00,072 --> 00:24:00,952 All right, so what do we do? 309 00:24:00,952 --> 00:24:02,472 We've got our current state. 310 00:24:05,192 --> 00:24:06,472 We've got our problem statement. 311 00:24:06,472 --> 00:24:08,072 We've got some of the things we want to look at. 312 00:24:09,032 --> 00:24:13,832 So next step, I'm going to start going through each one of these steps individually. 313 00:24:14,912 --> 00:24:23,032 And what I want to start doing is I want to start saying, okay, now, can AI give us a lift on each one of these steps? 314 00:24:24,232 --> 00:24:24,632 Okay. 315 00:24:27,512 --> 00:24:31,032 And the way we tend to do that, like I said, I like things simple. 316 00:24:33,032 --> 00:24:36,632 I guess I should clarify, we're really talking about Gen. 317 00:24:36,632 --> 00:24:37,112 AI here. 318 00:24:37,672 --> 00:24:39,992 I'm talking mostly about Gen. 319 00:24:39,992 --> 00:24:40,312 AI. 320 00:24:41,272 --> 00:24:44,712 So we start looking at suitability for task. 321 00:24:46,072 --> 00:24:50,712 And so I've got this little 2 by two here, and you can see on the y-axis we have cost of error. 322 00:24:51,112 --> 00:24:53,352 Is this a low stakes problem if something goes wrong? 323 00:24:53,832 --> 00:24:56,072 or is it a high stakes problem if something goes wrong? 324 00:24:56,952 --> 00:25:01,112 the classic example is like cancer diagnosis, you hear people talk about a lot, right? 325 00:25:01,352 --> 00:25:03,032 Low stakes versus high stakes. 326 00:25:03,432 --> 00:25:11,112 If it's a marketing e-mail, my apologies to the marketing folks in the room, but most of those are probably lower stakes. 327 00:25:11,832 --> 00:25:16,232 It may do some brand damage, but nine times out of 10, they're okay. 328 00:25:17,032 --> 00:25:19,392 Then on the x-axis, we have the nature of the task. 329 00:25:19,392 --> 00:25:22,392 Is it nuanced or is it algorithmic? 330 00:25:22,872 --> 00:25:23,352 Is it 331 00:25:23,832 --> 00:25:27,432 Probabilistic, or is it deterministic? 332 00:25:27,912 --> 00:25:36,072 OK, and what we're going to do is we're going to plot each one of these steps in one of these quadrants. 333 00:25:36,392 --> 00:25:41,792 OK, so MSA co-creation, where do you think that goes? 334 00:25:41,792 --> 00:25:46,872 Is that high stakes or low stakes? 335 00:25:50,712 --> 00:25:51,912 Yeah, I'd say high stakes too. 336 00:25:52,072 --> 00:25:54,392 By the way, a good consulting answer is it depends. 337 00:25:54,392 --> 00:26:00,712 So if you ever don't want to answer, but pretend like you're contributing, do like us consultants do, just answer everything with it depends. 338 00:26:02,072 --> 00:26:03,432 Yeah, so high stakes. 339 00:26:04,432 --> 00:26:07,512 And then is it algorithmic or is it nuanced? 340 00:26:11,672 --> 00:26:13,432 I'd say it's pretty nuanced, right? 341 00:26:16,672 --> 00:26:17,272 What are you doing? 342 00:26:22,352 --> 00:26:22,912 What the... 343 00:26:29,092 --> 00:26:30,132 Well, this is nice. 344 00:26:35,762 --> 00:26:36,482 Now maybe? 345 00:26:41,602 --> 00:26:42,322 Hey, okay. 346 00:26:43,042 --> 00:26:50,242 So yeah, so MSA creation, I put up here in the fighter pilot category. 347 00:26:50,802 --> 00:26:51,002 And 348 00:26:51,832 --> 00:26:53,592 I name these categories. 349 00:26:53,592 --> 00:26:57,232 I don't know if they're good names, but I was inspired by Microsoft with the copilot thing. 350 00:26:57,232 --> 00:27:00,072 I don't know. 351 00:27:00,072 --> 00:27:07,592 I feel like fighter pilots probably, I bet AI eventually will be pretty darn good at like dog fighting and things like that. 352 00:27:07,592 --> 00:27:17,432 But right now, if you've ever used ChatGPT or Claude or any of these tools and you put something in it and it says like canoodling or whatever and it sits there and spins a while, I don't think that'd be good in a dog fight. 353 00:27:17,672 --> 00:27:19,992 So for now, we're going to use the metaphor fighter pilot. 354 00:27:20,392 --> 00:27:22,552 But we want this task to be human-led. 355 00:27:23,112 --> 00:27:25,672 We don't want AI in there, right? 356 00:27:25,672 --> 00:27:37,512 So if you're sitting with your client, imagine your client has an AI that does the MSA co-creation and you have an AI that does MSA co-creation. 357 00:27:37,512 --> 00:27:38,792 And they're both talking to each other. 358 00:27:38,792 --> 00:27:41,112 Heavens knows what's going to come out the other side, right? 359 00:27:41,672 --> 00:27:42,952 Like, I don't think we really want that. 360 00:27:42,952 --> 00:27:45,112 We want the control in this particular case. 361 00:27:45,752 --> 00:27:46,792 So in this case, 362 00:27:47,752 --> 00:27:50,392 I think we said it was zero to two days for that task. 363 00:27:50,552 --> 00:27:54,952 It's probably going to stay zero to two days because we don't want to introduce AI into this. 364 00:27:57,112 --> 00:27:59,592 So we take the next step is the signature. 365 00:28:00,632 --> 00:28:05,112 Here we have high stakes because we're having to agree to that. 366 00:28:05,552 --> 00:28:07,912 abide by the terms of the master services agreement. 367 00:28:08,472 --> 00:28:12,472 So it's high stakes, but a signature is a signature. 368 00:28:12,552 --> 00:28:14,872 You just go in and, you know, like it's pretty routine. 369 00:28:15,032 --> 00:28:21,512 Companies tend to look for the same things every time in the, you know, just to make sure those boxes are checked. 370 00:28:21,672 --> 00:28:23,832 It's fairly algorithmic in nature. 371 00:28:23,832 --> 00:28:28,312 So here we have the, this lands in that copilot quadrant. 372 00:28:28,312 --> 00:28:31,352 So here it's like high enough stakes that we want a human to look at it. 373 00:28:32,552 --> 00:28:37,352 but low enough stakes that we're comfortable with AI taking a shot at it, right? 374 00:28:37,352 --> 00:28:52,312 So AI can review it, give me a list of things that it sees and notices for consideration, maybe classify each step as like showstopper, take a closer look, never seen this before, who cares? 375 00:28:52,312 --> 00:28:53,592 Something like that, right? 376 00:28:55,112 --> 00:28:59,192 All right, next step, go back to legal review. 377 00:28:59,272 --> 00:29:01,912 Again, I want a lawyer 378 00:29:02,552 --> 00:29:07,992 that I'm working with, if I've seen something new, I want to have a discussion with them about it so I understand the implications about it. 379 00:29:08,272 --> 00:29:15,352 I don't want my AI to go talk to my lawyer and just keep asking questions and then keep asking questions and running up our bill rate, right? 380 00:29:16,232 --> 00:29:18,232 I want that to be human-led too. 381 00:29:18,232 --> 00:29:24,872 So our legal process, our legal step in here probably isn't going to change much because we want it to be human-led. 382 00:29:27,112 --> 00:29:29,512 Next step, SOW review. 383 00:29:30,152 --> 00:29:31,992 Again, just like the last one, 384 00:29:32,552 --> 00:29:38,552 SOWs tend to be pretty standard once we have the type of work, whatever we're going to invoice them for. 385 00:29:39,032 --> 00:29:43,752 And so this is up here in that copilot space, too. 386 00:29:45,032 --> 00:29:55,552 So far, we've seen two steps of this workflow that we feel like AI can help us out with, and two steps that probably shouldn't, are probably bad ideas, right? 387 00:29:57,272 --> 00:29:58,072 So what's after that? 388 00:29:59,912 --> 00:30:01,352 Customer signature. 389 00:30:02,312 --> 00:30:07,512 Okay, so customer signature, this is where we sent it over with DocuSign and we did all that stuff. 390 00:30:09,672 --> 00:30:12,472 This lands in the autopilot zone. 391 00:30:12,632 --> 00:30:13,752 It's pretty algorithmic. 392 00:30:14,072 --> 00:30:27,992 You put it in DocuSign, you send it over, they click through the DocuSign, click sign, sign, sign, sign, do all the signing, and then they click it back, and then you get it back and you file it whatever you're going to, that's pretty much just a standard process, right? 393 00:30:29,192 --> 00:30:30,472 And so over here, 394 00:30:31,272 --> 00:30:33,392 We put that in the automate space. 395 00:30:33,392 --> 00:30:40,632 So this we could basically, we'll talk about this in a second, but we could basically have AI just go do this. 396 00:30:41,272 --> 00:30:45,552 Okay, but we'll get back to how we do this next. 397 00:30:45,552 --> 00:30:46,632 notify finance. 398 00:30:46,632 --> 00:30:48,432 We have a certain set of steps that we have to do. 399 00:30:48,432 --> 00:30:49,992 We have a form that we have to fill out. 400 00:30:49,992 --> 00:30:52,392 We have to do this stuff, send it over to finance. 401 00:30:52,392 --> 00:30:56,072 Finance gets the project created, does all the back office stuff. 402 00:30:56,632 --> 00:30:57,192 Again, 403 00:30:58,312 --> 00:31:02,712 We're over here in this automate space because it's pretty low risk. 404 00:31:03,432 --> 00:31:05,352 Documents already signed, everything. 405 00:31:06,072 --> 00:31:08,072 Just kind of going through the motions at that point. 406 00:31:09,352 --> 00:31:10,392 Any questions up to now? 407 00:31:13,032 --> 00:31:13,352 Cool. 408 00:31:14,472 --> 00:31:14,872 All right. 409 00:31:15,992 --> 00:31:20,552 Nature of task, test pilot, I mean, test pilot was where I was going to next. 410 00:31:21,192 --> 00:31:22,712 Notice we didn't have anything in there. 411 00:31:25,352 --> 00:31:35,112 Low risk and nuanced or probabilistic, I'm going to throw out there pre-mortems. 412 00:31:35,112 --> 00:31:36,552 So these are the things that you might do. 413 00:31:36,552 --> 00:31:38,712 Is anybody familiar with the pre-mortem? 414 00:31:40,632 --> 00:31:42,712 How many people have done a post-mortem before? 415 00:31:43,432 --> 00:31:44,312 Yeah, those are fun. 416 00:31:44,592 --> 00:31:45,832 Pre-mortem is the opposite. 417 00:31:46,152 --> 00:31:54,072 So maybe what you can do with a pre-mortem is you can go in and say, okay, well, here's the MSA. 418 00:31:54,752 --> 00:32:02,072 What happens if, and you can do some scenario stuff, and maybe you can explore and decide how comfortable you are with different terms in it. 419 00:32:02,312 --> 00:32:05,872 So these are more of your one-off things that you just kind of want to try. 420 00:32:05,872 --> 00:32:13,112 They're nuanced, low risk, so we don't have to worry about if it gets it wrong, because there's human judgment there. 421 00:32:15,352 --> 00:32:15,672 Okay. 422 00:32:16,232 --> 00:32:18,632 But now, I guess I'll go back to this real quick. 423 00:32:18,632 --> 00:32:22,232 We've got this, but now what do we do with this? 424 00:32:22,392 --> 00:32:23,352 That's the next step. 425 00:32:23,792 --> 00:32:25,432 Okay, how do we implement this thing? 426 00:32:27,832 --> 00:32:32,632 And here, consultants love two by twos, so you get tons of them in here. 427 00:32:33,112 --> 00:32:38,392 So here we have context depth on the left on the y-axis. 428 00:32:38,792 --> 00:32:45,752 And this is really like general information or does this require a lot of proprietary information? 429 00:32:46,312 --> 00:32:46,632 Right? 430 00:32:46,792 --> 00:32:50,592 Because if you think about it, all these LLMs are all trained on the general internet. 431 00:32:51,432 --> 00:32:57,752 So they're okay at all of the stuff that requires general knowledge because they've been trained on all of that. 432 00:32:58,192 --> 00:33:14,952 But now the more stuff that they need to understand that's data from your organization or maybe it's your industry specific, very detailed private information, the more of that there is, these general LLMs, generative, they 433 00:33:15,672 --> 00:33:16,952 They just aren't going to know that. 434 00:33:16,952 --> 00:33:21,752 So they're going to make more mistakes, probably hallucinate more, that kind of stuff. 435 00:33:22,952 --> 00:33:26,952 Technical complexity, no code versus pro code. 436 00:33:27,432 --> 00:33:29,112 And we'll talk about this in a little bit. 437 00:33:29,912 --> 00:33:33,352 If it's high complexity, you probably need a code-based solution. 438 00:33:34,072 --> 00:33:37,192 Somebody's going to have to write some software if it's super high complexity. 439 00:33:37,752 --> 00:33:45,192 If it's low complexity, you could probably get away with prompting and other techniques like that to do the implementation. 440 00:33:47,112 --> 00:33:50,792 So let's go through the same steps here again. 441 00:33:53,272 --> 00:33:54,312 MSA signature. 442 00:33:54,312 --> 00:34:04,872 So I landed this, you know, the proprietary data, only we know what information that we are looking for and what we want in our MSA. 443 00:34:05,112 --> 00:34:06,072 That's proprietary. 444 00:34:06,312 --> 00:34:09,032 The general internet doesn't know what we care about, right? 445 00:34:09,672 --> 00:34:12,952 So that's up here in the proprietary data section. 446 00:34:13,352 --> 00:34:14,552 Technical complexity, 447 00:34:15,312 --> 00:34:22,712 I don't know what your MSAs look like, but our MSAs really share a lot of language from a lot of MSAs. 448 00:34:23,112 --> 00:34:34,872 You wouldn't be surprised or would be surprised if you choose, like, how many times do you get a legal document that you notice somebody else's name because they just copied the last document and tried to change everything, right? 449 00:34:35,352 --> 00:34:42,792 So yeah, so not really a super highly technical situation, but 450 00:34:43,592 --> 00:34:46,312 but does require some of our proprietary information. 451 00:34:46,712 --> 00:34:50,232 So this is the realm of what I call domain experts, right? 452 00:34:50,552 --> 00:34:59,632 So you have somebody in your organization that understands what they're looking for in the MSAs, understands that particular structure, what you want, what questions you want to ask. 453 00:34:59,992 --> 00:35:03,592 And what you're going to do here is, how many are you using Copilot? 454 00:35:05,312 --> 00:35:05,672 Okay. 455 00:35:05,912 --> 00:35:06,792 ChatGPT? 456 00:35:08,112 --> 00:35:08,392 Okay. 457 00:35:09,672 --> 00:35:10,072 Other? 458 00:35:11,032 --> 00:35:11,912 Claude for the win. 459 00:35:12,232 --> 00:35:13,112 By the way, Claude. 460 00:35:13,472 --> 00:35:13,712 Anyway. 461 00:35:15,832 --> 00:35:20,592 So here you're going to use tools, you're going to use skills in order to implement this. 462 00:35:20,592 --> 00:35:26,632 So if you're not familiar with skills, that's outside of the scope of our talk today. 463 00:35:26,872 --> 00:35:31,272 But skills are basically you give the LLM instructions on how to handle things. 464 00:35:31,272 --> 00:35:41,192 So you can write a skill that says, read in this MSA, look for the state that disputes would be settled in. 465 00:35:41,672 --> 00:35:56,392 look at this website or this data that says these are the states that are really hard on companies in litigation and don't let it, you know, raise a flag if we're on any of these tough states, right? 466 00:35:56,792 --> 00:36:00,712 Or we want our insurance requirements. 467 00:36:01,112 --> 00:36:03,432 Here's a certificate of insurance. 468 00:36:03,432 --> 00:36:07,912 Make sure our certificate of insurance matches what they're requiring in the MSA. 469 00:36:08,152 --> 00:36:08,872 And if not, 470 00:36:09,512 --> 00:36:13,992 make a note, tell us this is what they expect, this is what we have, and we can make a decision, right? 471 00:36:15,272 --> 00:36:17,032 So that's the domain expert side. 472 00:36:17,832 --> 00:36:20,072 Reviewing SOW, same thing. 473 00:36:20,312 --> 00:36:36,552 Maybe you need like SLAs or something in the SOW, and a lot of times, especially with older clients, maybe you've got a client and somebody's doing that copy and paste thing, and they're copying and pasting the next 474 00:36:37,272 --> 00:36:43,912 the last one, and you've updated your general template for SOWs, and maybe the SLAs are now wrong, something like that, right? 475 00:36:44,552 --> 00:36:51,832 So now you can have a skill that when it gets to that SLA step, it goes ahead and flags all of those things for you. 476 00:36:54,392 --> 00:36:55,512 Customer signature. 477 00:36:56,232 --> 00:36:58,152 So put this on the power user. 478 00:36:58,152 --> 00:37:04,792 So customer signature, this one is, you know, pretty straightforward process. 479 00:37:05,432 --> 00:37:22,072 But you might need to be able to automate, like using Zapier or any of these, like, or maybe if you're technically intrigued enough to use like Claude Code or copilot to write your own software to automate these things. 480 00:37:22,392 --> 00:37:27,032 But the thing about these things, remember they were fairly deterministic. 481 00:37:27,752 --> 00:37:34,552 So a good way to burn a lot of tokens and spend money is doing things that should just be a script through AI. 482 00:37:36,152 --> 00:37:36,472 Right? 483 00:37:36,472 --> 00:37:38,712 So I would not use AI on these. 484 00:37:38,952 --> 00:37:40,512 I would just create a script for these. 485 00:37:40,512 --> 00:37:52,552 So you see here we're saying like, hey, in the automation side, we don't need, we're still going to improve our process, but AI isn't going to be our solution. 486 00:37:52,552 --> 00:37:55,112 So that's the answer there. 487 00:37:55,672 --> 00:37:57,752 Next, notify finance, that's the same. 488 00:37:58,232 --> 00:37:58,632 Right? 489 00:37:59,672 --> 00:38:04,232 Our pre-mortem, over here, just use the AI tool. 490 00:38:04,952 --> 00:38:14,392 And then the engineering I skipped because this is when you have a pretty highly complex situation. 491 00:38:15,112 --> 00:38:21,992 we're doing a model where we're doing demand forecasting for beverages and things like that. 492 00:38:22,232 --> 00:38:24,152 That's going to be a much more complex task. 493 00:38:25,112 --> 00:38:28,712 So our legal process doesn't need any of that. 494 00:38:30,352 --> 00:38:32,152 Did you notice anything else missing in here? 495 00:38:35,112 --> 00:38:36,952 MSA co-creation isn't in here. 496 00:38:37,592 --> 00:38:39,512 Why is MSA co-creation not in here? 497 00:38:42,632 --> 00:38:45,592 Yeah, we said this is a human-only activity, right? 498 00:38:45,832 --> 00:38:52,632 So we only need to, when we're thinking of implementation, the implementation model for human-only activities is humans, right? 499 00:38:52,792 --> 00:38:53,832 Talking to humans. 500 00:38:54,152 --> 00:38:55,112 So let's do that. 501 00:38:56,792 --> 00:38:57,912 Any questions so far? 502 00:38:57,912 --> 00:39:00,552 Is this all making sense? 503 00:39:02,552 --> 00:39:05,472 All right, so now we come back. 504 00:39:05,992 --> 00:39:15,112 and I'm not going to spend too much time on this, but we can see after we made those changes, some of the things that I circled have changed. 505 00:39:16,232 --> 00:39:25,112 The automation on the top reduced the delay significantly because we automatically sent an e-mail and we automatically created a DocuSign and things like that. 506 00:39:25,352 --> 00:39:27,272 So we're able to really crunch that down. 507 00:39:27,912 --> 00:39:33,192 And the wait time, if you recall, was a big part of the overall time. 508 00:39:33,192 --> 00:39:35,352 So the more we can reduce the wait time, the better. 509 00:39:37,032 --> 00:39:53,832 Then the SOW review, because we had all of those things listed out in our skill that said these are the things you must look for in this SOW, and tell me, rate those, and give me feedback on those, we were able to cut that down significantly. 510 00:39:53,832 --> 00:39:54,712 Same with the 511 00:39:55,032 --> 00:39:58,472 general review of the MSA, that also cut down significantly. 512 00:39:58,872 --> 00:40:05,112 So if you look down now at the process efficiency, we went from 12% to 20%. 513 00:40:05,352 --> 00:40:08,712 And you might be like, so what? 514 00:40:09,432 --> 00:40:15,352 But that is a 67% increase on a couple of really simple tests. 515 00:40:15,832 --> 00:40:22,472 You should be able to probably implement most of this in a matter of a few hours, right? 516 00:40:23,552 --> 00:40:29,352 What we did do is I would probably, if I were doing this for real, I'd reimagine the value stream. 517 00:40:30,712 --> 00:40:35,032 Right now, we just copied what we're already doing today, but with AI, we can do a lot more. 518 00:40:35,272 --> 00:40:37,032 I'd probably reimagine the value stream. 519 00:40:37,032 --> 00:40:44,632 I'd probably combine like SOWs and MSAs into one step, so then there's no hands-offs. 520 00:40:44,632 --> 00:40:47,272 I'd probably, you can connect 521 00:40:47,992 --> 00:40:50,552 If you aren't familiar, there's a thing called MCPs. 522 00:40:50,552 --> 00:40:58,072 That's a technical term, but they're typically called connectors and things like that in Claude and in ChatGPT. 523 00:40:58,392 --> 00:41:00,152 I think they're actions in ChatGPT. 524 00:41:00,472 --> 00:41:09,952 Can't remember all the details, but I'd put connectors into DocuSign, and then it can go back and say, oh, hey, I'm done with this. 525 00:41:09,952 --> 00:41:11,032 Send it for signing. 526 00:41:11,632 --> 00:41:13,472 And so then we don't even have to wait. 527 00:41:14,072 --> 00:41:14,712 for the review. 528 00:41:14,712 --> 00:41:18,312 So there are ways that you could do this, make even more impact. 529 00:41:18,552 --> 00:41:25,272 But I think it's more important to start small and grow into it than trying to go big with your first couple of shots. 530 00:41:27,312 --> 00:41:35,432 Okay, last lean tool, by the way, for the non-lean folks, the implementation model and the AI suitability, those are not lean tools. 531 00:41:35,992 --> 00:41:42,632 But last lean tool I want to talk about today is PDCA. 532 00:41:43,032 --> 00:41:44,472 That's plan, do, check, act. 533 00:41:44,872 --> 00:41:48,552 And it's basically your basic continuous improvement process, right? 534 00:41:49,672 --> 00:41:52,472 Start by look at the problem, find opportunities. 535 00:41:52,472 --> 00:41:54,472 That's what we did when we created our A3. 536 00:41:54,952 --> 00:41:56,952 Then we go and we do a first pass. 537 00:41:56,952 --> 00:41:59,912 We do the do, and we implement some changes. 538 00:42:00,312 --> 00:42:04,712 And then the thing I wanted to hit on this slide, check changes a little bit in AI. 539 00:42:06,312 --> 00:42:07,992 So you need to be tracking the results. 540 00:42:07,992 --> 00:42:10,392 You need to be tracking how much time you're saving. 541 00:42:10,872 --> 00:42:30,152 But the thing that I don't think people are thinking about a lot when they're doing a lot of these custom AI solutions, or not custom, but AI solutions using skills and other things like that, the speed at which the LLMs change is rapid. 542 00:42:30,152 --> 00:42:32,712 You hear of a new model coming out all the time, right? 543 00:42:33,272 --> 00:42:38,632 Well, if your skill that reads MSAs 544 00:42:39,272 --> 00:42:41,752 is working okay this time. 545 00:42:43,592 --> 00:42:47,592 Next time a new model might come out and it will behave completely differently. 546 00:42:49,512 --> 00:42:56,552 So what you really need to do, in the tech world we call them evaluations or evals. 547 00:42:57,272 --> 00:43:04,232 So you really need a process that you come back and you check on the results of these things every once in a while. 548 00:43:04,632 --> 00:43:16,232 Have you ever noticed in ChatGPT, Copilot, Claude, when you're typing about something and it pulls up something that you talked about like 3 weeks ago and it says, oh, like you were saying, you go, oh, wait, that was a long time ago. 549 00:43:17,112 --> 00:43:18,632 Well, that causes drift. 550 00:43:19,312 --> 00:43:20,552 Okay, so that causes drift. 551 00:43:20,552 --> 00:43:31,832 So your MSA, you might interact with this MSA, and over time, those interactions are affecting your output. 552 00:43:32,472 --> 00:43:34,792 because it remembers some of those conversations. 553 00:43:36,232 --> 00:43:43,592 So it's good, but it's also bad because it could cause your model to drift or your skill to drift and not provide the same results it used to. 554 00:43:44,072 --> 00:43:52,872 So you want to continuously be checking these things on a fairly regular cadence to make sure that they're still behaving as well as you'd want them to behave. 555 00:43:55,512 --> 00:43:59,752 Now, next step, agents, right? 556 00:44:00,712 --> 00:44:03,392 So get this working, do a couple iterations. 557 00:44:03,392 --> 00:44:07,032 When you feel confident, then you can start talking about agents. 558 00:44:07,952 --> 00:44:11,672 Until then, I think just the workflows is a good starting place. 559 00:44:11,992 --> 00:44:14,792 And you can go back and say, hey, it took me 3 hours. 560 00:44:15,352 --> 00:44:19,832 We had a 60 plus percent improvement on cycle lead times. 561 00:44:20,632 --> 00:44:23,992 And we reduced the cost of monitoring or 562 00:44:25,512 --> 00:44:30,952 reviewing MSAs and SOWs by X percent, 50 percent or whatever add up there, right? 563 00:44:31,592 --> 00:44:39,112 So real simple way to start getting actual realized value out of AI rather than just a bunch of messing around. 564 00:44:42,392 --> 00:44:48,552 If you want, got a couple minutes for questions, but if you want, e-mail me. 565 00:44:49,192 --> 00:44:54,392 I think I had a couple of requests from other people that I've given this talk at to do 566 00:44:54,792 --> 00:44:55,912 like a workshop. 567 00:44:56,312 --> 00:45:04,352 If you're interested in actually sitting down and going through this process with some of your workflows or one of your workflows, let me know. 568 00:45:04,352 --> 00:45:06,472 We were planning on doing it like early June. 569 00:45:07,432 --> 00:45:10,792 Just shoot me an e-mail and I'll add you to the invite. 570 00:45:11,512 --> 00:45:15,272 Not a paid thing, just a, it's real easy to talk about. 571 00:45:15,272 --> 00:45:17,912 It always looks easy on the screen, right? 572 00:45:17,912 --> 00:45:19,832 But it's not necessarily always easy in practice. 573 00:45:20,712 --> 00:45:21,432 Questions? 574 00:45:26,552 --> 00:45:32,952 That was either incredibly useless or I'm such a great speaker, everybody completely understands how to take this home. 575 00:45:33,192 --> 00:45:33,832 Thank you all. 576 00:45:33,832 --> 00:45:37,592 Good job. 577 00:45:37,592 --> 00:45:37,912 Thanks.