AI Workflows: From Concept to Execution (for Project Managers): transcript

The assignment

0:00, on YouTube So Randy has three things to do by the end of the quarter. Every change going into a release window has to be in front of the people who approve it, before that window opens. He has to track what actually shipped. And the release notes go out Friday. That's the assignment. That's all of it. And look, he's done work like this before. None of these tools, same job. He knows what good looks like, he knows Friday is the part that slips first, and he could tell you right now which of those three is the hard one, before he writes a single thing down. He's not guessing. Here's why it landed on him, by the way.

Why it landed on him

0:40, on YouTube Last quarter, two changes went into the same window and they collided. Three people had approved them, separately, different threads, and not one of the three saw the other two. Randy is the one who pieced it together afterwards. Which is how it became his. And the thing he took from that wasn't that somebody was careless. It's that nothing actually went wrong on the Friday. Friday is just where it surfaced. He knows the ugly number too, because people who do this for a living have gone and measured it. Eighty-six percent of all incidents caused by change come from a change that closed successfully. It got approved. It got deployed.

The number nobody likes

1:21, on YouTube Somebody ticked it off as done … and then weeks later it took something down. So he isn't building an approval form. He's building something that stays interested after the deploy. Which is a different piece of software, and a harder one. What you're going to watch is somebody competent take an assignment all the way to something running. Concept, options, design, a prototype, a pilot, and then the unglamorous part, which is making it survive him. And the reason it's worth your time isn't that he gets things wrong. It's that he's holding a toolbox nobody handed him before, and the limits are not where his instincts say they are. So he goes and finds them.

2:04, on YouTube That's the part you can take. Which is the whole method, so let me just say it now. This isn't a session about prompting, and the thing you need isn't a better prompt. It's being able to look at a building block and see what it can't do … because what it can't do is the spec for the next thing you build. The limitation is the specification. And notice what somebody good actually does with that. He doesn't sit around waiting to be surprised by a limitation. He goes looking for it while it's still cheap. Concept is three questions, and most people only ask the first one. What problem are you solving, exactly. Who for.

2:42, on YouTube And who else runs this when you're not here. That third one is the one that gets left out. It's also the one that decides whether you built a workflow or a hobby. So Randy starts by refusing his own brief. Because "a release go, no-go workflow" isn't a problem statement, it's a department. He pulls it apart until every piece names one job. Intake, where a proposed change becomes a record. Window assembly, where those records group into a release window. Authorization, where a person says yes or no with the whole window in front of them. Outcome, where you find out if it worked. And publication, where it goes out on Friday. Five jobs. And exactly one of them needs a person to decide anything.

One sentence, five jobs

3:31, on YouTube That split is most of the design, and it's the first thing you should do to your own brief. The four that need no judgment are the four you can build yourself. The one that does need judgment stays a human decision, and your job there isn't to replace that person. It's to give them a better view than three separate email threads gave them. Now, here's the bit people in a hurry skip. Randy is not the first person who has ever needed this. Release governance is a worked discipline with decades behind it. So before he designs anything, he goes and takes its vocabulary. A change authority is the person or group responsible for authorizing a change, which is not the same thing as a committee.

Prior art

4:14, on YouTube And the same literature will happily tell you that review boards often turn into bottlenecks. A standard change is one that's low risk, pre-authorized, well understood and fully documented. He isn't inventing release management. He's implementing something that was written down long before any of these tools existed. And that hands him two things. The hard thinking about what should happen is already done, by somebody else. And the only genuinely new part of his problem is the toolbox. Which is a much smaller problem than the one he was handed. Whatever you're automating, go check whether that's true of yours. It usually is. Then he writes down the ways he could approach it.

Ten ways in, four ruled out

4:58, on YouTube Not one plan. Ten. Because your first idea is rarely the one that survives, and paper is the cheapest place you'll ever find that out. And then he rules four of them out. This one runs four times a year, so frequency kills it. A workflow you touch quarterly never gets learned and never gets improved. You rebuild it from memory every single time you come back. This one hands the model the decision about whether a change is risky, which is judgment he can't check. And if you can't tell whether an answer was right, you can't tell whether the thing is working. You'll end up defending it on faith, in a room full of people who don't share your faith. This one rolls back automatically the second something looks wrong, and the consequence outruns his ability to verify it.

5:48, on YouTube He's not saying never. He's saying not first. And not while he's still learning what these tools get wrong. And this last one needs evidence that lives in a vendor portal he can't reach, and in two people's heads. So the workflow would be guessing. Worse, it'd be guessing confidently, which is the failure nobody catches in a demo. Frequency. Checkable judgment. Reversibility. Reachable evidence. Four questions you can ask in a meeting without opening a laptop. They cost you nothing except being willing to say no to your own ideas. And then he writes down the thing he already knows is going to bite. Whatever he builds is going to run as him. His connections.

Writing down the risk

6:34, on YouTube His name on every message it sends. He's not ignoring that. He's logging it, on day one, because it's what decides when this stops being his and becomes the platform team's. He'd rather know that now than find out in a handover. Design is the afternoon that saves you the quarter. And almost nobody gives it an afternoon. What he does here isn't architecture diagrams. He reads what each tool promises, in the vendor's own documentation, and he writes down what it refuses to do. Because the refusals are what you build the design around. First real decision: where does the truth live. He's got three candidates. A spreadsheet, which everybody already has.

7:19, on YouTube A list, which comes with the licence. And a proper relational table, which is what he'd pick if money weren't a question. The spreadsheet goes first. And it goes without a single line of work. Because the connector locks that file for up to six minutes after it's used. It returns two hundred fifty six rows by default. When a key matches more than one row, it quietly updates the first one only. And then the sentence that settles it, in the vendor's own words: simultaneous modifications are not supported. A Friday job and a human being are going to reach for that file on the same afternoon. He didn't need an experiment to know how that ends. Somebody wrote it down, and he read it.

Ruling out the spreadsheet

Choosing the list

8:03, on YouTube So the list it is. And he reads its limits too, instead of celebrating. It's free with the licence he already holds, the connector reads and writes, it holds thirty million items, and nothing he builds on it costs anything per run. The catch is one number. The view threshold is five thousand items, and a change log passes five thousand way sooner than you'd expect. So he indexes the columns and filters the views on day one. Costs ten minutes now. Costs a weekend later. The relational table he prices, and parks. Deliberately. The connector is premium, so it costs money per user. And the environment that holds it cannot be created on a Microsoft 365 licence at all. The documentation is blunt about that, so somebody with Power Platform rights has to stand it up for him. So it simply isn't his to create.

Pricing the table he can't have

8:58, on YouTube And that goes in the plan as a known future step, with a known price. Which is a very different thing from a surprise. Then the question that looks like paperwork and turns out to be the entire design. What is a change? He has to answer that in columns. And columns are unforgiving in a way sentences aren't. What is it, who asked, which window, which systems it touches, who approves it, and what state is it in. And that last column makes him list the states. Proposed. Scheduled. Approved. Rejected. Released. Verified. Failed. And the second those are written down, the design asks him a question nobody in the room asked.

What is a change?

9:43, on YouTube Can a change go from proposed straight to released? He says no. Which means he's just specified a gate. And notice he didn't decide to build one. The table told him he needed it. That's what people mean when they say design surfaces requirements, and it's a lot less mystical than it sounds when it happens on your own whiteboard. Then the agent. Which is the part everybody wants to start with, and he gets to on purpose instead of out of enthusiasm. The distinction he needs is sitting in the documentation. A knowledge source grounds an answer. A tool takes an action. Those are not two words for the same thing.

Knowledge grounds, tools act

10:27, on YouTube And the agent builder he's got accepts knowledge sources, and does not accept actions. It does not support authoring actions that integrate external services. Vendor's own words. So this agent can tell anybody what's in Thursday's window … and it can't put anything into Thursday's window. Can't record a change. Can't collect an approval. Can't publish the notes. Which isn't an obstacle. It's the division of labour, handed to him for free, before he wrote any of it. The write path lives in the flows, and the agent is the window people look through. Knowledge grounds. Tools act. If you remember two words out of today, remember those. He finds the next one the same way. In a sentence he'd never have gone looking for, if he hadn't been looking for limits.

11:18, on YouTube A run of one of these flows can't last longer than thirty days. Now, the obvious build is: start an approval, wait for the answer. But a change window can sit open for five weeks when somebody's on leave, or a vendor slips. And on day thirty-one, that run doesn't fail loudly. It just ends. And the answer arrives to nobody. So he splits it before he builds it. One flow creates the approval. A different flow waits for the answer and acts on it. That's a design-time decision, invisible at concept time, expensive to retrofit. And there's one of those in every build you'll ever do. Finding it on paper costs twenty minutes.

12:01, on YouTube Finding it in production costs the quarter. And then he gets to the place where the manual runs out. Which is the most interesting moment in this whole thing. The approval records live in the vendor's store. Not in his table. And he needs to know whether one of them can be edited after the fact, because somebody in an audit is going to ask him that. So he reads everything. The documentation doesn't say. Not yes. Not no. So he does what you do when the documentation runs out. He tests it. As cheaply as he can possibly make the test, somewhere being wrong costs him nothing. And that right there is the engineering hat.

Where the manual runs out

12:39, on YouTube That's the one part of this the tools can't hand you. They'll answer any question you ask them. Knowing which question is load-bearing, and how to answer it for almost nothing … that's yours. What he builds from the answer is small. At the moment a decision gets made, his own flow writes his own row, into his own list, with versioning on. Who decided. What they said. Their comment. The timestamp. He doesn't replace the vendor's record. He stops depending on it. And he's careful about why, because the reason people usually give is the wrong one. Records that can't be overwritten are a real legal requirement in life sciences, where the regulation says record changes shall not obscure previously recorded information.

13:24, on YouTube Outside regimes like that? Nobody's making him do this. He does it anyway. For one reason. Attribution. Because if that row can be changed afterwards, then who said yes is a claim. And if it can't be changed, then who said yes is evidence. That's a design judgment, not a compliance one, and he can tell you which one he's making. Which is what an audit actually tests you on. Now he builds something. And the first thing he builds is deliberately not the design. A saved prompt and a spreadsheet. One week. Somebody proposes a change in chat, he pastes it in, the model turns it into a tidy row, and on Friday it writes the notes. And yes, that's the store he just disqualified. That's fine. He's not depending on it.

The prototype

14:14, on YouTube He's interrogating it. Because a prototype isn't a small version of the product. It's an instrument for answering one question. And his question is whether approvers will actually look at a window view when you put it in front of them. Answer comes back in a week. It's yes, with a condition. They look if it arrives where they already are. One finding. It reshapes the real build, and it cost him an afternoon. And he's clear-eyed about what that week could not tell him. Not whether two writers collide, because he was the only writer. Not whether it survives a month, because it ran for seven days. Not whether it works without him, because he was standing in the middle of it the entire time. A prototype tests the idea. It does not test the mechanism.

15:02, on YouTube And knowing the difference is what stops you shipping an afternoon's work to forty people. So he builds the design, and runs it as a pilot. Real flows, the real list, real approvals landing in mail and in chat cards. Three weeks. One team who agreed to go first. And the pilot teaches him three things design couldn't, because all three of them are about people. Approvers answer from their phones. In the evening. Which means the view has to be legible on a phone, and he'd designed it for a monitor. Nobody read the Friday notes until he moved them out of email and into the channel the team already lived in. At which point they got read by more people than had ever read them before. And that gate he was so pleased with?

The pilot

15:48, on YouTube Resented within nine days. Because he was gating certificate renewals with the same ceremony as database migrations. Now that third one is the interesting one, because he doesn't have to invent the fix. The discipline he borrowed from back in act two already named it for him. A standard change. Low risk, pre-authorized, well understood, fully documented. So he writes a bypass, and a rule about who's allowed to call something standard. And the gate stops being a tax everybody pays. Because a workflow that gates everything equally hasn't finished being designed. It's just said no to everything and called that governance. And what happens next is people route around you.

A hole in his own gate

16:32, on YouTube Which leaves you with a bottleneck and no record. Then he does the one thing that separates a pilot from a demonstration. He changes the design. Not the goal, not the scope. The phone layout, the publication channel, and the gate. Three changes, every one of them from evidence, none of them from opinion. Which is the whole reason you run a pilot in the first place. Production is where it stops being his, and starts being something a team depends on. And the work that gets it there is not glamorous. The Friday job runs on a timer. And that timer is the first thing in this build that runs when nobody's watching. So he asks the question that separates a script from a system. What happens if it runs twice? Because it will.

What if it runs twice?

17:22, on YouTube A run gets retried. Somebody tests it. The platform hiccups. And two sets of release notes go out, which is worse than none going out. Because now nobody trusts the set that was right. So the job asks what it already published, before it publishes anything. Same input, run ten times, one result. The word for that is idempotent, and it's worth learning the word, because every scheduled thing you ever build is going to need it. Then the collision check. Which is where this whole assignment started. Mechanically it's almost nothing. Query the list for windows that overlap, and branch when they do. And he can say why it matters without raising his voice.

Collisions and attribution

18:04, on YouTube Published guidance from people who run very large systems puts roughly seventy percent of outages down to changes in a live system. And on doing two at once, it says they strongly advise running only one canary deployment at a time. The reason underneath that is attribution. Put two changes in one window, and when something breaks, you can't tell which one did it. So you didn't just lose a release. You lost the ability to learn anything from having lost it. And here he's careful, in a way worth copying. There's no published figure for how often two changes in one window actually collide. He looked. It doesn't exist. So he doesn't quote one.

18:45, on YouTube The argument stands on attribution without a statistic. And if somebody hands you a percentage for this, ask them where they got it. That habit is most of your credibility in a room with engineers in it. Then the job the opening number demands. The one he'd have been forgiven for skipping. Approved isn't an outcome. Released isn't an outcome either. Because eighty-six percent of change-caused incidents come from changes that closed successfully. So he adds a verified state, a date, and one line of evidence, written by whoever would've been paged if it had gone wrong. And the moment that column exists, the workflow answers a question he has never once been able to answer in his career. Of the changes we approved … how many actually worked? He won't like the first number. Nobody does.

Verified

19:34, on YouTube The current industry survey has roughly thirty-eight percent of respondents reporting a change failure rate above sixteen percent, so he's in company. But it's a number now, instead of a feeling. And a number is something you can move. Now, cost. Because he priced this at design time, and the answer surprises most people. Everything you've watched carries no per-run charge. None. It draws on an allowance of six thousand actions a day that came with the licence he already had. And the read-only agent is zero-rated for people who already hold a Copilot licence. So the running cost of the whole thing is nothing. Which is why he never needed a business case to start.

What it costs to run

20:19, on YouTube He needed an afternoon. Where it stops being free is specific, and he knows the figures because he went and looked. A premium connector is fifteen dollars per user per month. A capacity licence attached to a single flow is a hundred and fifty. Relational storage past the first gigabyte is forty-eight dollars a gigabyte. Those are published list prices, checked on the sixth of October. And the vendor's own page says prices shown are for marketing purposes and may not reflect actual list price. Which is why that date stays on the screen. And why you check it yourself before quoting it to anybody who controls a budget. And now the two walls. And the thing to notice is that neither one is a surprise.

21:03, on YouTube Both have been on his plan since design. First one is that relational table he parked back in act three. Premium connector, and an environment his own licence cannot create, which means somebody with Power Platform rights has to stand it up for him. He isn't blocked by that wall. He's standing at it on purpose, with a price and a reason written down. Second one is identity. And that's the one he logged on day one. Everything Randy built runs as Randy. It breaks when he changes teams. It breaks when his licence moves. It breaks when a connection needs consent again while he's on holiday. And it breaks completely, every part of it, the day he leaves. The fix is real, and it isn't exotic.

Wall one, a real table

Wall two, identity

21:52, on YouTube A service identity can own and run these flows. But creating that identity happens in an administration console. The registration upstream of it happens in a different one. And decoupling a flow from the person who built it needs the relational store from the first wall. Every step is on the other side. He planned for this. There's one thing he can do with nobody's permission, and he does it the week the pilot starts instead of the week he leaves. He adds a trusted colleague as a co-owner. So somebody else can edit it, see the run history, keep it alive while he's away. That's continuity. And he doesn't oversell it as identity, because it isn't.

22:33, on YouTube Then he goes and has the conversation. And look at what he carries into the room. Not a request. A working system. Three weeks of pilot evidence. A verified column that says what his change failure rate actually is. A list of exactly what he needs from them, and the reason for each item on it. That conversation goes differently. Anybody who's sat on the other side of it will tell you the same. And the sentence he uses is the one for you to carry out of here. The workflow doesn't graduate because it got too big. It graduates because it got too important to depend on one person's token.

23:10, on YouTube Three things we didn't do today, and each one is its own session. Durable data. A real schema, real relationships, and the difference between a list that holds records and a table that protects them. Outcome tracking as a discipline of its own, because verified turns out to be a lot harder than approved. And a lot more valuable. And the door. Which is what you reach for when the system you need has no connector at all. There's a way to give a model its own door into a system, and that one's coming, because somebody asked for it. So look at what Randy ended up with. A list. Four flows. One agent that only reads.

What he actually built

23:53, on YouTube It's almost nothing. But look at what he had to do to get there. He refused his own brief and broke it into five jobs. He went and took the vocabulary from people who'd solved this before him. He wrote down ten approaches and ruled out four on paper. He chose a store by reading what it refused to do. He let a table tell him he needed a gate. He found out a window can outlive a run, and split the flow before building it. He got to the end of the documentation and tested the rest for almost nothing. He built a prototype to answer one question, and didn't mistake it for a product. He ran a pilot, and then actually changed the design. He asked what happens if it runs twice.

24:37, on YouTube And he left a deliberate hole in his own gate, so people wouldn't route around him. That's systems design. He's been managing people who do it for most of his career. And this quarter, he did it himself. Now here's what that costs you, because it's worth being honest about it. Every one of these little workflows is a small piece of software with a lifecycle. Somebody decides what it's for, designs it, builds it, tests it, watches it, fixes it, and eventually retires it. In a kiosk that size, that somebody is you. Wearing every hat. That used to be a team. And the only reason it can be one person now is that you've got an engineer available who doesn't say no.

25:20, on YouTube Which is the trade worth understanding. Because when you used to ask a busy engineer for something and they told you it wouldn't work … what you actually got was a free design review. They were handing you the limitation. And the limitation, as we said right at the start, is the specification. Nothing says no to you now. It just builds whatever you asked for. Confidently, and quickly, including the wrong thing. So the no has to come from inside the work. Which means it's part of the hat you're wearing, instead of something that arrives from somebody else's calendar. That's the hat Randy put on back in act three, and it's the whole skill. Not prompting. Not tools.

26:03, on YouTube Just the habit of asking what a thing can't do before you depend on it, going and finding out when nobody's written it down, and then designing around the answer instead of around the hope. So pick one question that belongs to you. Not the department's question. Yours. The one you answer by hand every week and have stopped noticing you answer. Give it an afternoon of concept and an afternoon of design before you build anything. Then build the smallest thing that answers one question. You already know how to do this. You've been doing it with other people's hands for years. The only new part is the toolbox … and the toolbox will tell you what it can't do, if you ask it first. That's the method. The limitation is the specification.