AI Workflow Design for Project Managers: transcript

The assignment

0:00, on YouTube Nora has a form to fill in. Forty fields, every week, sometimes twice. A vendor sends over a scope document for a site they have just surveyed, and those forty fields are what her own planning system needs before anybody can put a crew on a calendar. The vendor document is really two documents wearing one cover. The front half is a form. Dates, a contact, a site address, a reference number, boxes with answers in them. The back half is prose, because somebody walked the building and wrote down what they saw. The back half is where a lot of the answers are. Not all of them. But enough that she can't skip it, and enough that she can't get past it by being fast. That's the assignment.

0:45, on YouTube Forty fields, out of a document that is half form and half paragraph, every week. Now, what has she got. One chat assistant. The one the company already pays for, in a browser tab she already has open. No agents. No scripts, because she can't install an interpreter on that laptop and she isn't going to ask. No new licence, no new tenant, nothing that needs a ticket and a quarter. I want to say that out loud at the start, because everything in the next twenty minutes stays inside it. Nothing here needs a tool she doesn't already have. If it ever sounds like it does, I have lost the thread, and you should hold me to it. So here is what she built.

What she has to work with

The governance file

1:27, on YouTube A prompt, and underneath the prompt, a file. She calls it her governance file, and it is about sixty lines long now. Use the exact wording from the source where the source has it. Don't invent a date. If the floor isn't given, say the floor isn't given. Be concise. Always fill every field. If you haven't built one of these, you have certainly seen one. In ChatGPT it lives in a project's instructions, or inside a custom GPT. In Copilot it is the instruction box on an agent, or a saved prompt. Same artifact, different door. Every one of those lines went in on a Tuesday because something went wrong on a Monday.

2:08, on YouTube Which means the file isn't really a set of rules. It is a scar record. And it works. Most of the time. Let's not walk past that phrase, because everybody does. Most of the time means she still reads all forty fields before she sends it, every week, because she has no idea which ones are wrong this week. The thing that was supposed to save her the reading never actually saved her the reading. Then one week she is out, and Ben covers for her. Same prompt, same file, same vendor, a document that looks like all the others. The answers come back different. Not dramatically. Field eleven is phrased in a way Nora's version never phrases it. Field twenty-three is blank, and Nora's is never blank.

The week she was out

2:55, on YouTube Field seven has something in it that is plausible. So Ben asks the reasonable question. What did I do wrong? And Nora opens the file she wrote herself, reads it line by line, and cannot tell him. That is the actual problem. It isn't field seven. It is that the person who built the thing can't explain the thing. Hold on to that, because we come back to it at the end, and it turns out to be the test that matters. The normal next move here is obvious. Add a rule. Something like, match the phrasing in the examples. Maybe, never leave a field blank. That will fix field eleven and field twenty-three, and next month something else will be wrong, and the file will be sixty-two lines long instead of sixty.

Your prompt isn't broken

3:43, on YouTube So let me say the thing this whole session is built on, and then spend the rest of it earning it. Your prompt isn't broken. It's undesigned. Every hour spent on wording is an hour not spent on structure, and the wording is not where the failure lives. Here is where it lives. The rules are global. The defects are local. Every line in that file applies to all forty fields. Every line went in because of one field. So each fix helps the field it was written for and quietly makes some other field worse, and nobody connects the two, because they are a month apart. Watch it happen with two of her own rules.

Global rules, local defects

Two rules that cannot both win

4:26, on YouTube A field that needs interpreting comes back padded with detail the document never contained. Reasonable response. Be concise, only use what is in the source. In goes the rule, and the padding stops. Six weeks later a different field starts arriving clipped. It is a field that needed a full sentence copied across exactly as written, and be concise is now shortening it. Nobody suspects that rule, because that rule fixed something. Then the blanks start. Reasonable response. Always fill every field. That one is worse than it looks. For the fields where the honest answer is, the document doesn't say, she has just written a standing instruction to make something up.

5:10, on YouTube Both rules are sensible. Neither can be deleted, because each one is holding up a field. They cannot both be satisfied, and no amount of rewording resolves it, because the conflict isn't in the words. One instruction is being applied to forty different jobs. That is the diagnosis. Now the first piece of the design, and it comes before any rule gets written. Decide what shape the answer is. Here is a sentence out of a scope document. File Room printers on the second floor behind the cabinets, ports A one zero zero two B, A one zero zero three B, A one zero zero four B. Her target doesn't want a paragraph about that.

Fields, or records?

5:53, on YouTube It wants three rows, one per port, each carrying the floor, the room and the access note. Three rows. The prompt has been producing a tidy description. Sometimes one row. Sometimes three. Sometimes three with the location only on the first one. That is not a tuning problem. That is a type mismatch, and you can add rules to it forever. A target that needs records and a prompt that writes prose will never agree, and I would bet a good share of anybody's rules file is made of attempts to make them agree. Saying it out loud looks different depending on where you are sitting. In ChatGPT or Gemini you ask for a table.

6:36, on YouTube In Copilot Studio or Power Automate you hand it a schema. The move is identical. Say what shape you want before you say what goes in it. Now the part that does the actual work, and it is not a tool. It is twenty minutes with the target document and a pen. Sort every field into one of four classes. Move. Interpret. Look up. Consequence. That is the method. Everything else in this session is a consequence of having done that honestly, and almost nobody does it, because it feels like admin and it looks like something you can skip. Class one. Move. The answer is sitting in a field in the source, as a value. Site address. Vendor reference.

The four classes

Move

7:25, on YouTube Survey date. There is no judgment in it, it is a copy. And here is the thing about a copy. It shouldn't go anywhere near the model. A copy is a copy. Every Move field you identify is a field that can't be hallucinated, can't be reworded, can't be summarised, and can't be shortened by a rule somebody wrote for a different field six weeks ago. Nora has fourteen of those out of forty. The day she labelled them, fourteen of her forty failure modes stopped existing, and no prompting was involved. Class two. Interpret. The answer is in the prose. Somebody wrote it in a sentence and it has to come out as a value.

Interpret

8:10, on YouTube Behind the cabinets becomes an access constraint. Second floor becomes a floor number. This is the class the whole problem is made of, and it is the class where the model genuinely earns its keep. Two rules attach to it, and only to it. First, the sentence travels with the answer. Whatever gets extracted, the quote it came from comes along beside it. That one change costs nothing, and it changes what review is. Review stops being, re-read ten pages and see if this is right. It becomes, does this sentence say that. Which takes four seconds, and which anybody can do, including somebody who has never seen the site. Second, not stated is a legal answer.

8:56, on YouTube It has to be allowed, out loud, in writing. The dangerous failure here was never a wrong answer, because a wrong answer usually looks wrong. The dangerous failure is a plausible answer to a question the document never addressed, because the form then looks complete, and a complete form doesn't get a second look. Notice what just happened to the always fill every field rule. It didn't need to be reworded. It needed to stop applying to the eleven fields it should never have applied to. Class three. Look up. The answer isn't in the source document at all. Not in a field, not in the prose, nowhere. The vendor never knew it. Which panel do those three ports terminate on?

Look up

9:43, on YouTube The contractor has no idea. That lives in Nora's own system, and it always did. No amount of prompting fills that field, because there is nothing to prompt against. People spend months tuning a prompt for a field whose answer was never in the document, and it is the most quietly expensive mistake on this list. A lookup also doesn't have to be an integration. Not today. Today it can be Nora pasting in the twelve rows that matter, or keeping a short reference table beside the prompt. The plumbing isn't the point. The point is that the field gets answered by a table instead of by a reading, and we will automate it in a few minutes without changing the design at all.

Consequence

10:28, on YouTube Class four. Consequence. This is the one everybody drops. Behind the cabinets. That is not a field on her form. Nothing on that form asks about cabinets. But it means somebody has to move cabinets before anybody can work, which is a prerequisite with a cost and an owner and a date. A prompt told to fill the fields will throw that sentence away, correctly even, because it did what it was told. Then the crew arrives on a Tuesday, the printers are behind a cabinet nobody budgeted to move, and the day is gone. So the output needs somewhere to put things that are not fields. One list. Implications, each with the sentence that implied it, for a human to accept or reject. Not answers. Candidates.

11:19, on YouTube And now the most valuable move in the whole method, which only exists because the classes exist. Move a field down a class. Take something you have been interpreting and ask whether your own system already knows it. If it does, it was never an Interpret field. It is a Look up field, and you have just moved it out of the class where answers can be confidently wrong into the class where they can only be missing. That is the reliability gain, and here is the part I like. It is countable. Nora started with nineteen Interpret fields. After the pass, eleven. Eight fields stopped being able to lie to her, and that is a number you can put in front of a manager.

Reclassify downward

12:02, on YouTube So let's automate the lookup. Same design, now with a connection to the system of record. This is the one place in the session where you might need something you have to ask somebody for, so let me be straight about it. Everything up to here runs on what you already have. The classification does. The quotes do. The lookup does, with a table you paste in. The connection is an upgrade to a design that already works without it, so if you can't get one this quarter you lose the automation, not the method. In ChatGPT those connections are connectors or custom actions. In Copilot they are Graph connectors and Power Platform connectors.

12:45, on YouTube The job is the same in both. Fetch it, don't guess it. And with one in place, the lookup turns out to do three different jobs rather than one. Job one. Validate. Do ports A one zero zero two B, A one zero zero three B and A one zero zero four B actually exist? One wrong character in a port identifier is invisible. It is invisible in prose, it is invisible on the form, and it stays invisible all the way to a technician standing in the wrong room holding the right paperwork. A lookup catches it in milliseconds. Nothing else catches it at all until it costs a day.

The connector: validate

Enrich

13:27, on YouTube Job two. Enrich. Which panel do they terminate on? The target wants it, the contractor never said it, and her system has known it for four years. That field was never an extraction problem. Once it is a lookup, it is simply answered. Job three, and this is the one I would build the demonstration around. Conflict. The document says second floor. The system says A one zero zero two B is on three. Stop and notice what that is, because it is genuinely new. Not a better answer. A disagreement nobody could see before. The vendor couldn't see it, because they don't have her system. Her system couldn't see it, because it has not read the document.

Conflict

14:12, on YouTube And she couldn't see it, because catching it means cross-checking forty fields against a database by hand, every week, which nobody does and nobody should. The output stops being a filled form. It becomes a filled form plus a flagged disagreement, which is a better thing than either source had on its own. One more benefit arrives free. The connection brings your own vocabulary with it. Device types, work package types, risk categories, whatever your organisation's real list is. An open-ended interpretation turns into a choice from a fixed list, and the model phrased it differently this time disappears as a category of problem. Which, incidentally, is field eleven. The one Ben got differently. It was never a phrasing problem.

Your own vocabulary

15:01, on YouTube It was a missing list. Now, what comes out. Not a filled document. A filled document, plus an account of how every value got there. Let me be precise about what that account is not. It is not the model's confidence score. Asking a model how sure it is gets you a number it generated, about itself, with nothing underneath it. That is a guess in a lab coat, and people trust it because it has a decimal point in it. What we want is status derived from evidence, and we already have the evidence, because we did the classification. Moved. It came out of a source field, exactly. Nothing to check. Verified. It matched the system of record.

How every value got there

15:49, on YouTube Nothing to check. Needs eyes. It was interpreted from prose. Here is the sentence. Read it, accept it or fix it. Contradicted. The document and the system disagree. Resolve this before anybody schedules anything. Not stated. Nobody answered this question anywhere. Go and ask the contractor, don't guess. And consequence. Implied, not stated, with the sentence that implied it. Six statuses. If that is too many for a reviewer at eight in the morning, collapse it to three. Confirmed. Needs eyes. Missing. Here is what it does to Nora's week. Forty fields. Six need eyes. One contradicted. She reads six sentences instead of ten pages. And when somebody asks how long the review takes now, she has a number, and the number comes out of the work instead of out of a feeling.

Forty, six, one

16:47, on YouTube Forty, six, one. That is the slide. That is the whole business case, and it fits on one line. One more thing about that report, and it matters more than it sounds. It did not cost a second pass. There is no review step. No critique call, no self-check, no sending the document back through to grade its own homework. The status falls out of the same single output as the values, because it is only a record of which class the field was in and whether a quote and a lookup existed. Which means the long document gets sent once, not twice. That is worth sitting with, because one design decision bought two different things. The structure that makes the output trustworthy is the structure that makes it cheap. Classification isn't a tax you pay for reliability and then pay again at the till. Which is where we pick up next time.

One pass, not two

17:43, on YouTube What a run actually costs, and which parts of a design drive the bill. So. Back to Ben. What does done mean here? Not, does it give me the right answer. You can get the right answer by accident, and Nora did, for about eight months. Done is this. Somebody else runs it and gets my answer. That is the test. It is a brutal one, and it is the only one that catches what went wrong in week one. Think about why the governance file failed it. The file worked for Nora because Nora remembered which rule went in for which problem, and that memory was never in the file.

What done means

18:23, on YouTube It was in her head and it did not travel. So Ben inherited sixty instructions with no reasons attached, and the first time one of them misfired there was nothing to reason about. Now hand over the other thing instead. Forty fields, each labelled with its class. The Move fields with their source field. The Interpret fields with their rule and their quote requirement. The Look up fields with their table. The consequence list. Ben doesn't need to know why. He can see why. Every rule names the field it governs, so a rule that misbehaves points at exactly one place, and fixing it can't reach across and damage field twenty-three. That is the difference between a prompt and a specification, and it is the only difference that matters.

19:10, on YouTube Developers worked this out a while ago, by the way. They keep the specification in a file that sits beside the work, called agents dot m d, or claude dot m d, so whoever opens the project next inherits the reasons and not just the rules. Different room, same conclusion. Which brings me to the part I want you to take to work. A prompt that only works when you run it is a skill. A prompt that anybody can run is an asset. Your organisation cannot see the first one. It shows up as one person being slightly faster at something, which is invisible, and which earns you more of the same work rather than credit for it. The second one has a name on it.

A skill, or an asset

19:51, on YouTube It gets handed to a team. It gets pointed at in a review. Same forty fields. Same afternoon of thinking. Completely different thing to own. One last practical piece and then I will let you go. Before you change a single rule, go and find three documents you have already done by hand, with the answers you know are right sitting beside them. That is a regression set. It takes twenty minutes to assemble and it is the difference between tinkering and engineering, because now when you change something you can re-run all three and see what broke. Notice that the regression set has no product name attached to it. Every other piece of this is called something different depending on where you are sitting. That one is just a folder, in every tool, forever, which tells you which part of today was the durable part.

20:43, on YouTube Nora found hers in a folder she already had. Everybody has that folder. So here is the afternoon. Print your target. Sort the fields into four columns. Count what is in the Interpret column, then go through it once more and ask which of those your own system already knows. That number, how many you moved out of Interpret, is this whole session, measured. Your prompt isn't broken. It was never designed.