1
00:00:00,000 --> 00:00:05,840
Nora has a form to fill in. Forty
fields, every week, sometimes twice.

2
00:00:06,400 --> 00:00:10,640
A vendor sends over a scope document
for a site they have just surveyed, and

3
00:00:10,720 --> 00:00:15,120
those forty fields are what her own
planning system needs before anybody can

4
00:00:15,160 --> 00:00:19,360
put a crew on a calendar. The vendor
document is really two documents

5
00:00:19,400 --> 00:00:22,800
wearing one cover. The
front half is a form.

6
00:00:23,040 --> 00:00:28,240
Dates, a contact, a site address, a
reference number, boxes with answers in

7
00:00:28,320 --> 00:00:32,960
them. The back half is prose, because
somebody walked the building and wrote

8
00:00:33,080 --> 00:00:37,080
down what they saw. The back half
is where a lot of the answers are.

9
00:00:37,280 --> 00:00:41,200
Not all of them. But enough that she
can't skip it, and enough that she

10
00:00:41,280 --> 00:00:45,280
can't get past it by being
fast. That's the assignment.

11
00:00:45,360 --> 00:00:50,480
Forty fields, out of a document that
is half form and half paragraph, every

12
00:00:50,560 --> 00:00:54,720
week. Now, what has she
got. One chat assistant.

13
00:00:55,040 --> 00:00:59,360
The one the company already pays for,
in a browser tab she already has open.

14
00:00:59,840 --> 00:01:03,920
No agents. No scripts, because she
can't install an interpreter on that

15
00:01:04,000 --> 00:01:09,360
laptop and she isn't going to ask. No
new licence, no new tenant, nothing

16
00:01:09,440 --> 00:01:13,600
that needs a ticket and a quarter. I
want to say that out loud at the start,

17
00:01:13,800 --> 00:01:17,360
because everything in the next
twenty minutes stays inside it.

18
00:01:17,440 --> 00:01:20,640
Nothing here needs a tool
she doesn't already have.

19
00:01:20,680 --> 00:01:24,640
If it ever sounds like it does, I have
lost the thread, and you should hold

20
00:01:24,720 --> 00:01:27,440
me to it. So here
is what she built.

21
00:01:27,520 --> 00:01:30,800
A prompt, and underneath
the prompt, a file.

22
00:01:30,880 --> 00:01:35,360
She calls it her governance file,
and it is about sixty lines long now.

23
00:01:35,440 --> 00:01:38,880
Use the exact wording from the
source where the source has it.

24
00:01:38,960 --> 00:01:43,560
Don't invent a date. If the floor
isn't given, say the floor isn't given.

25
00:01:43,760 --> 00:01:46,960
Be concise. Always
fill every field.

26
00:01:47,520 --> 00:01:50,600
If you haven't built one of these,
you have certainly seen one.

27
00:01:50,800 --> 00:01:56,040
In ChatGPT it lives in a project's
instructions, or inside a custom GPT.

28
00:01:56,240 --> 00:02:00,720
In Copilot it is the instruction
box on an agent, or a saved prompt.

29
00:02:00,960 --> 00:02:05,760
Same artifact, different door. Every
one of those lines went in on a Tuesday

30
00:02:05,920 --> 00:02:07,920
because something went
wrong on a Monday.

31
00:02:08,080 --> 00:02:10,960
Which means the file isn't
really a set of rules.

32
00:02:11,120 --> 00:02:14,960
It is a scar record. And
it works. Most of the time.

33
00:02:15,440 --> 00:02:18,720
Let's not walk past that
phrase, because everybody does.

34
00:02:19,120 --> 00:02:23,840
Most of the time means she still reads
all forty fields before she sends it,

35
00:02:24,000 --> 00:02:28,000
every week, because she has no idea
which ones are wrong this week.

36
00:02:28,080 --> 00:02:31,920
The thing that was supposed to save her
the reading never actually saved her

37
00:02:32,000 --> 00:02:36,400
the reading. Then one week she
is out, and Ben covers for her.

38
00:02:36,520 --> 00:02:41,120
Same prompt, same file, same vendor,
a document that looks like all the

39
00:02:41,200 --> 00:02:44,040
others. The answers
come back different.

40
00:02:44,320 --> 00:02:49,200
Not dramatically. Field eleven is
phrased in a way Nora's version never

41
00:02:49,360 --> 00:02:55,000
phrases it. Field twenty-three is
blank, and Nora's is never blank.

42
00:02:55,280 --> 00:02:58,280
Field seven has something
in it that is plausible.

43
00:02:58,560 --> 00:03:00,880
So Ben asks the
reasonable question.

44
00:03:01,360 --> 00:03:06,400
What did I do wrong? And Nora opens
the file she wrote herself, reads it

45
00:03:06,480 --> 00:03:11,680
line by line, and cannot tell
him. That is the actual problem.

46
00:03:12,080 --> 00:03:16,160
It isn't field seven. It is that
the person who built the thing can't

47
00:03:16,240 --> 00:03:20,960
explain the thing. Hold on to that,
because we come back to it at the end,

48
00:03:21,000 --> 00:03:23,355
and it turns out to be
the test that matters.

49
00:03:23,395 --> 00:03:25,715
The normal next move
here is obvious.

50
00:03:26,195 --> 00:03:30,715
Add a rule. Something like, match
the phrasing in the examples.

51
00:03:30,995 --> 00:03:36,115
Maybe, never leave a field blank.
That will fix field eleven and field

52
00:03:36,195 --> 00:03:40,195
twenty-three, and next month something
else will be wrong, and the file will

53
00:03:40,275 --> 00:03:43,395
be sixty-two lines
long instead of sixty.

54
00:03:43,475 --> 00:03:47,235
So let me say the thing this whole
session is built on, and then spend the

55
00:03:47,315 --> 00:03:51,395
rest of it earning it.
Your prompt isn't broken.

56
00:03:51,475 --> 00:03:56,595
It's undesigned. Every hour spent
on wording is an hour not spent on

57
00:03:56,675 --> 00:04:00,675
structure, and the wording is
not where the failure lives.

58
00:04:00,875 --> 00:04:04,595
Here is where it lives.
The rules are global.

59
00:04:04,635 --> 00:04:10,595
The defects are local. Every line in
that file applies to all forty fields.

60
00:04:10,675 --> 00:04:13,635
Every line went in
because of one field.

61
00:04:13,755 --> 00:04:18,195
So each fix helps the field it was
written for and quietly makes some other

62
00:04:18,355 --> 00:04:23,315
field worse, and nobody connects the
two, because they are a month apart.

63
00:04:23,395 --> 00:04:26,035
Watch it happen with
two of her own rules.

64
00:04:26,115 --> 00:04:29,875
A field that needs interpreting comes
back padded with detail the document

65
00:04:30,035 --> 00:04:32,915
never contained.
Reasonable response.

66
00:04:32,955 --> 00:04:36,195
Be concise, only use
what is in the source.

67
00:04:36,275 --> 00:04:38,915
In goes the rule, and
the padding stops.

68
00:04:39,235 --> 00:04:42,915
Six weeks later a different
field starts arriving clipped.

69
00:04:42,955 --> 00:04:47,395
It is a field that needed a full sentence
copied across exactly as written,

70
00:04:47,515 --> 00:04:50,435
and be concise is
now shortening it.

71
00:04:50,635 --> 00:04:54,435
Nobody suspects that rule,
because that rule fixed something.

72
00:04:54,515 --> 00:04:57,928
Then the blanks start.
Reasonable response.

73
00:04:58,062 --> 00:05:02,475
Always fill every field. That
one is worse than it looks.

74
00:05:02,515 --> 00:05:06,835
For the fields where the honest answer
is, the document doesn't say, she has

75
00:05:06,955 --> 00:05:10,435
just written a standing
instruction to make something up.

76
00:05:10,515 --> 00:05:14,755
Both rules are sensible. Neither can
be deleted, because each one is holding

77
00:05:14,835 --> 00:05:19,555
up a field. They cannot both be
satisfied, and no amount of rewording

78
00:05:19,675 --> 00:05:23,395
resolves it, because the
conflict isn't in the words.

79
00:05:23,475 --> 00:05:27,275
One instruction is being
applied to forty different jobs.

80
00:05:27,315 --> 00:05:32,035
That is the diagnosis. Now the first
piece of the design, and it comes

81
00:05:32,115 --> 00:05:37,235
before any rule gets written.
Decide what shape the answer is.

82
00:05:37,315 --> 00:05:40,035
Here is a sentence out
of a scope document.

83
00:05:40,195 --> 00:05:44,755
File Room printers on the second floor
behind the cabinets, ports A one zero

84
00:05:44,955 --> 00:05:50,635
zero two B, A one zero zero
three B, A one zero zero four B.

85
00:05:50,675 --> 00:05:53,355
Her target doesn't want
a paragraph about that.

86
00:05:53,395 --> 00:05:58,355
It wants three rows, one per port, each
carrying the floor, the room and the

87
00:05:58,435 --> 00:06:04,302
access note. Three rows. The prompt
has been producing a tidy description.

88
00:06:04,488 --> 00:06:07,022
Sometimes one row.
Sometimes three.

89
00:06:07,208 --> 00:06:10,235
Sometimes three with the
location only on the first one.

90
00:06:10,275 --> 00:06:15,555
That is not a tuning problem. That is
a type mismatch, and you can add rules

91
00:06:15,635 --> 00:06:20,675
to it forever. A target that needs
records and a prompt that writes prose

92
00:06:20,835 --> 00:06:25,635
will never agree, and I would bet a
good share of anybody's rules file is

93
00:06:25,715 --> 00:06:28,035
made of attempts
to make them agree.

94
00:06:28,515 --> 00:06:32,315
Saying it out loud looks different
depending on where you are sitting.

95
00:06:32,595 --> 00:06:36,155
In ChatGPT or Gemini
you ask for a table.

96
00:06:36,435 --> 00:06:40,475
In Copilot Studio or Power
Automate you hand it a schema.

97
00:06:40,755 --> 00:06:46,035
The move is identical. Say what shape
you want before you say what goes in

98
00:06:46,115 --> 00:06:51,002
it. Now the part that does the
actual work, and it is not a tool.

99
00:06:51,042 --> 00:06:54,762
It is twenty minutes with the
target document and a pen.

100
00:06:54,842 --> 00:06:58,402
Sort every field into
one of four classes.

101
00:06:58,762 --> 00:07:03,442
Move. Interpret.
Look up. Consequence.

102
00:07:03,882 --> 00:07:08,282
That is the method. Everything else
in this session is a consequence of

103
00:07:08,442 --> 00:07:13,002
having done that honestly, and almost
nobody does it, because it feels like

104
00:07:13,082 --> 00:07:16,042
admin and it looks like
something you can skip.

105
00:07:16,522 --> 00:07:22,122
Class one. Move. The answer is
sitting in a field in the source, as a

106
00:07:22,202 --> 00:07:25,642
value. Site address.
Vendor reference.

107
00:07:25,802 --> 00:07:30,002
Survey date. There is no
judgment in it, it is a copy.

108
00:07:30,282 --> 00:07:32,322
And here is the
thing about a copy.

109
00:07:32,602 --> 00:07:35,242
It shouldn't go
anywhere near the model.

110
00:07:35,482 --> 00:07:41,002
A copy is a copy. Every Move field
you identify is a field that can't be

111
00:07:41,082 --> 00:07:46,122
hallucinated, can't be reworded, can't
be summarised, and can't be shortened

112
00:07:46,202 --> 00:07:49,482
by a rule somebody wrote for a
different field six weeks ago.

113
00:07:50,202 --> 00:07:52,762
Nora has fourteen of
those out of forty.

114
00:07:53,322 --> 00:07:57,402
The day she labelled them, fourteen
of her forty failure modes stopped

115
00:07:57,482 --> 00:08:00,282
existing, and no
prompting was involved.

116
00:08:01,242 --> 00:08:05,562
Class two. Interpret. The
answer is in the prose.

117
00:08:06,042 --> 00:08:09,722
Somebody wrote it in a sentence
and it has to come out as a value.

118
00:08:10,042 --> 00:08:12,922
Behind the cabinets becomes
an access constraint.

119
00:08:13,322 --> 00:08:15,762
Second floor becomes
a floor number.

120
00:08:16,122 --> 00:08:19,962
This is the class the whole problem
is made of, and it is the class where

121
00:08:20,002 --> 00:08:22,282
the model genuinely
earns its keep.

122
00:08:22,922 --> 00:08:25,842
Two rules attach to
it, and only to it.

123
00:08:26,122 --> 00:08:29,362
First, the sentence
travels with the answer.

124
00:08:29,642 --> 00:08:33,642
Whatever gets extracted, the quote
it came from comes along beside it.

125
00:08:34,202 --> 00:08:38,642
That one change costs nothing,
and it changes what review is.

126
00:08:38,922 --> 00:08:42,842
Review stops being, re-read ten
pages and see if this is right.

127
00:08:43,322 --> 00:08:46,122
It becomes, does this
sentence say that.

128
00:08:46,682 --> 00:08:50,842
Which takes four seconds, and which
anybody can do, including somebody who

129
00:08:50,922 --> 00:08:56,322
has never seen the site. Second,
not stated is a legal answer.

130
00:08:56,522 --> 00:08:59,642
It has to be allowed,
out loud, in writing.

131
00:09:00,202 --> 00:09:04,362
The dangerous failure here was never
a wrong answer, because a wrong answer

132
00:09:04,442 --> 00:09:09,162
usually looks wrong. The dangerous
failure is a plausible answer to a

133
00:09:09,362 --> 00:09:13,722
question the document never addressed,
because the form then looks complete,

134
00:09:13,762 --> 00:09:16,282
and a complete form
doesn't get a second look.

135
00:09:17,002 --> 00:09:20,442
Notice what just happened to the
always fill every field rule.

136
00:09:20,922 --> 00:09:25,082
It didn't need to be reworded. It
needed to stop applying to the eleven

137
00:09:25,242 --> 00:09:27,325
fields it should
never have applied to.

138
00:09:27,365 --> 00:09:33,365
Class three. Look up. The answer
isn't in the source document at all.

139
00:09:33,685 --> 00:09:37,485
Not in a field, not
in the prose, nowhere.

140
00:09:37,765 --> 00:09:42,645
The vendor never knew it. Which panel
do those three ports terminate on?

141
00:09:43,205 --> 00:09:48,725
The contractor has no idea. That lives
in Nora's own system, and it always

142
00:09:48,805 --> 00:09:53,845
did. No amount of prompting fills that
field, because there is nothing to

143
00:09:54,005 --> 00:09:59,125
prompt against. People spend months
tuning a prompt for a field whose answer

144
00:09:59,285 --> 00:10:03,845
was never in the document, and it is
the most quietly expensive mistake on

145
00:10:03,925 --> 00:10:08,085
this list. A lookup also doesn't
have to be an integration.

146
00:10:08,165 --> 00:10:13,365
Not today. Today it can be Nora pasting
in the twelve rows that matter, or

147
00:10:13,445 --> 00:10:16,405
keeping a short reference
table beside the prompt.

148
00:10:16,485 --> 00:10:20,965
The plumbing isn't the point. The point
is that the field gets answered by a

149
00:10:21,045 --> 00:10:25,205
table instead of by a reading, and
we will automate it in a few minutes

150
00:10:25,445 --> 00:10:27,365
without changing
the design at all.

151
00:10:28,325 --> 00:10:33,925
Class four. Consequence. This
is the one everybody drops.

152
00:10:34,245 --> 00:10:38,485
Behind the cabinets. That
is not a field on her form.

153
00:10:38,725 --> 00:10:41,285
Nothing on that form
asks about cabinets.

154
00:10:41,525 --> 00:10:46,005
But it means somebody has to move
cabinets before anybody can work, which is

155
00:10:46,085 --> 00:10:49,965
a prerequisite with a cost
and an owner and a date.

156
00:10:50,245 --> 00:10:54,485
A prompt told to fill the fields will
throw that sentence away, correctly

157
00:10:54,565 --> 00:10:57,005
even, because it did
what it was told.

158
00:10:57,285 --> 00:11:01,685
Then the crew arrives on a Tuesday, the
printers are behind a cabinet nobody

159
00:11:01,845 --> 00:11:04,965
budgeted to move,
and the day is gone.

160
00:11:05,045 --> 00:11:08,965
So the output needs somewhere to
put things that are not fields.

161
00:11:09,045 --> 00:11:14,245
One list. Implications, each with the
sentence that implied it, for a human

162
00:11:14,325 --> 00:11:18,965
to accept or reject.
Not answers. Candidates.

163
00:11:19,045 --> 00:11:23,365
And now the most valuable move in
the whole method, which only exists

164
00:11:23,445 --> 00:11:28,045
because the classes exist.
Move a field down a class.

165
00:11:28,085 --> 00:11:31,845
Take something you have been interpreting
and ask whether your own system

166
00:11:31,925 --> 00:11:36,525
already knows it. If it does,
it was never an Interpret field.

167
00:11:36,565 --> 00:11:40,325
It is a Look up field, and you have
just moved it out of the class where

168
00:11:40,405 --> 00:11:44,405
answers can be confidently wrong
into the class where they can only be

169
00:11:44,845 --> 00:11:49,565
missing. That is the reliability
gain, and here is the part I like.

170
00:11:49,605 --> 00:11:54,205
It is countable. Nora started
with nineteen Interpret fields.

171
00:11:54,245 --> 00:11:59,365
After the pass, eleven. Eight fields
stopped being able to lie to her, and

172
00:11:59,445 --> 00:12:02,525
that is a number you can
put in front of a manager.

173
00:12:02,565 --> 00:12:07,365
So let's automate the lookup. Same
design, now with a connection to the

174
00:12:07,445 --> 00:12:12,165
system of record. This is the one place
in the session where you might need

175
00:12:12,245 --> 00:12:16,645
something you have to ask somebody
for, so let me be straight about it.

176
00:12:16,885 --> 00:12:20,005
Everything up to here runs
on what you already have.

177
00:12:20,085 --> 00:12:22,885
The classification
does. The quotes do.

178
00:12:22,925 --> 00:12:25,925
The lookup does, with
a table you paste in.

179
00:12:26,005 --> 00:12:30,405
The connection is an upgrade to a
design that already works without it, so

180
00:12:30,485 --> 00:12:35,605
if you can't get one this quarter you
lose the automation, not the method.

181
00:12:35,685 --> 00:12:40,765
In ChatGPT those connections are
connectors or custom actions.

182
00:12:40,965 --> 00:12:45,525
In Copilot they are Graph connectors
and Power Platform connectors.

183
00:12:45,605 --> 00:12:49,965
The job is the same in both.
Fetch it, don't guess it.

184
00:12:50,245 --> 00:12:54,005
And with one in place, the lookup
turns out to do three different jobs

185
00:12:54,085 --> 00:12:57,685
rather than one.
Job one. Validate.

186
00:12:58,165 --> 00:13:04,165
Do ports A one zero zero two B, A
one zero zero three B and A one zero

187
00:13:04,245 --> 00:13:09,525
zero four B actually exist? One wrong
character in a port identifier is

188
00:13:09,605 --> 00:13:14,645
invisible. It is invisible in prose,
it is invisible on the form, and it

189
00:13:14,805 --> 00:13:18,805
stays invisible all the way to a
technician standing in the wrong room

190
00:13:19,045 --> 00:13:23,365
holding the right paperwork. A
lookup catches it in milliseconds.

191
00:13:23,845 --> 00:13:26,885
Nothing else catches it at
all until it costs a day.

192
00:13:27,845 --> 00:13:31,845
Job two. Enrich. Which
panel do they terminate on?

193
00:13:32,325 --> 00:13:36,325
The target wants it, the contractor
never said it, and her system has known

194
00:13:36,365 --> 00:13:40,805
it for four years. That field
was never an extraction problem.

195
00:13:41,045 --> 00:13:43,605
Once it is a lookup,
it is simply answered.

196
00:13:44,485 --> 00:13:48,485
Job three, and this is the one I
would build the demonstration around.

197
00:13:48,965 --> 00:13:52,205
Conflict. The document
says second floor.

198
00:13:52,405 --> 00:13:56,245
The system says A one zero
zero two B is on three.

199
00:13:56,965 --> 00:14:00,845
Stop and notice what that is,
because it is genuinely new.

200
00:14:01,125 --> 00:14:05,325
Not a better answer. A disagreement
nobody could see before.

201
00:14:05,605 --> 00:14:08,485
The vendor couldn't see it,
because they don't have her system.

202
00:14:08,885 --> 00:14:12,165
Her system couldn't see it, because
it has not read the document.

203
00:14:12,565 --> 00:14:16,405
And she couldn't see it, because
catching it means cross-checking forty

204
00:14:16,565 --> 00:14:21,925
fields against a database by hand,
every week, which nobody does and nobody

205
00:14:22,085 --> 00:14:25,365
should. The output stops
being a filled form.

206
00:14:25,765 --> 00:14:29,845
It becomes a filled form plus a
flagged disagreement, which is a better

207
00:14:29,925 --> 00:14:32,325
thing than either
source had on its own.

208
00:14:32,725 --> 00:14:37,445
One more benefit arrives free. The
connection brings your own vocabulary

209
00:14:37,565 --> 00:14:42,645
with it. Device types, work package
types, risk categories, whatever your

210
00:14:42,725 --> 00:14:47,525
organisation's real list is. An
open-ended interpretation turns into a

211
00:14:47,605 --> 00:14:51,445
choice from a fixed list, and the
model phrased it differently this time

212
00:14:51,765 --> 00:14:54,085
disappears as a
category of problem.

213
00:14:54,165 --> 00:14:56,805
Which, incidentally,
is field eleven.

214
00:14:57,285 --> 00:15:01,725
The one Ben got differently. It
was never a phrasing problem.

215
00:15:01,925 --> 00:15:04,793
It was a missing list.
Now, what comes out.

216
00:15:04,873 --> 00:15:10,553
Not a filled document. A filled
document, plus an account of how every

217
00:15:10,633 --> 00:15:15,753
value got there. Let me be precise
about what that account is not.

218
00:15:15,833 --> 00:15:18,873
It is not the model's
confidence score.

219
00:15:18,993 --> 00:15:24,073
Asking a model how sure it is gets you
a number it generated, about itself,

220
00:15:24,193 --> 00:15:29,513
with nothing underneath it. That is a
guess in a lab coat, and people trust

221
00:15:29,593 --> 00:15:32,233
it because it has a
decimal point in it.

222
00:15:32,353 --> 00:15:36,913
What we want is status derived from
evidence, and we already have the

223
00:15:36,953 --> 00:15:39,993
evidence, because we
did the classification.

224
00:15:40,473 --> 00:15:44,113
Moved. It came out of a
source field, exactly.

225
00:15:44,313 --> 00:15:48,913
Nothing to check. Verified. It
matched the system of record.

226
00:15:49,113 --> 00:15:54,113
Nothing to check. Needs eyes.
It was interpreted from prose.

227
00:15:54,313 --> 00:15:57,913
Here is the sentence. Read
it, accept it or fix it.

228
00:15:58,553 --> 00:16:02,193
Contradicted. The document
and the system disagree.

229
00:16:02,393 --> 00:16:05,353
Resolve this before
anybody schedules anything.

230
00:16:06,073 --> 00:16:09,713
Not stated. Nobody answered
this question anywhere.

231
00:16:09,913 --> 00:16:12,233
Go and ask the
contractor, don't guess.

232
00:16:12,873 --> 00:16:18,153
And consequence. Implied, not stated,
with the sentence that implied it.

233
00:16:19,033 --> 00:16:23,513
Six statuses. If that is too many for
a reviewer at eight in the morning,

234
00:16:23,713 --> 00:16:27,673
collapse it to three.
Confirmed. Needs eyes.

235
00:16:27,993 --> 00:16:31,233
Missing. Here is what
it does to Nora's week.

236
00:16:31,513 --> 00:16:35,593
Forty fields. Six need
eyes. One contradicted.

237
00:16:35,993 --> 00:16:39,153
She reads six sentences
instead of ten pages.

238
00:16:39,353 --> 00:16:43,913
And when somebody asks how long the
review takes now, she has a number, and

239
00:16:43,993 --> 00:16:47,193
the number comes out of the work
instead of out of a feeling.

240
00:16:47,833 --> 00:16:51,273
Forty, six, one.
That is the slide.

241
00:16:51,673 --> 00:16:55,433
That is the whole business
case, and it fits on one line.

242
00:16:56,073 --> 00:16:59,673
One more thing about that report,
and it matters more than it sounds.

243
00:17:00,073 --> 00:17:04,393
It did not cost a second
pass. There is no review step.

244
00:17:04,793 --> 00:17:09,353
No critique call, no self-check, no
sending the document back through to

245
00:17:09,433 --> 00:17:13,913
grade its own homework. The status
falls out of the same single output as

246
00:17:13,993 --> 00:17:18,553
the values, because it is only a record
of which class the field was in and

247
00:17:18,673 --> 00:17:20,873
whether a quote and
a lookup existed.

248
00:17:21,113 --> 00:17:25,553
Which means the long document
gets sent once, not twice.

249
00:17:25,833 --> 00:17:30,073
That is worth sitting with, because one
design decision bought two different

250
00:17:30,233 --> 00:17:34,473
things. The structure that makes the
output trustworthy is the structure

251
00:17:34,553 --> 00:17:39,513
that makes it cheap. Classification
isn't a tax you pay for reliability and

252
00:17:39,593 --> 00:17:43,353
then pay again at the till. Which
is where we pick up next time.

253
00:17:43,753 --> 00:17:48,390
What a run actually costs, and which
parts of a design drive the bill.

254
00:17:48,430 --> 00:17:52,430
So. Back to Ben. What
does done mean here?

255
00:17:52,750 --> 00:17:55,070
Not, does it give
me the right answer.

256
00:17:55,310 --> 00:17:59,390
You can get the right answer by
accident, and Nora did, for about eight

257
00:17:59,470 --> 00:18:05,150
months. Done is this. Somebody
else runs it and gets my answer.

258
00:18:05,230 --> 00:18:10,270
That is the test. It is a brutal one,
and it is the only one that catches

259
00:18:10,350 --> 00:18:15,310
what went wrong in week one. Think
about why the governance file failed it.

260
00:18:15,430 --> 00:18:19,790
The file worked for Nora because Nora
remembered which rule went in for

261
00:18:19,910 --> 00:18:23,390
which problem, and that
memory was never in the file.

262
00:18:23,470 --> 00:18:26,590
It was in her head
and it did not travel.

263
00:18:26,710 --> 00:18:31,470
So Ben inherited sixty instructions
with no reasons attached, and the first

264
00:18:31,630 --> 00:18:35,150
time one of them misfired there
was nothing to reason about.

265
00:18:35,310 --> 00:18:37,470
Now hand over the
other thing instead.

266
00:18:37,590 --> 00:18:40,430
Forty fields, each
labelled with its class.

267
00:18:40,470 --> 00:18:42,830
The Move fields with
their source field.

268
00:18:42,870 --> 00:18:46,430
The Interpret fields with their
rule and their quote requirement.

269
00:18:46,470 --> 00:18:48,643
The Look up fields
with their table.

270
00:18:48,697 --> 00:18:52,163
The consequence list. Ben
doesn't need to know why.

271
00:18:52,217 --> 00:18:56,990
He can see why. Every rule names
the field it governs, so a rule that

272
00:18:57,110 --> 00:19:02,190
misbehaves points at exactly one place,
and fixing it can't reach across and

273
00:19:02,350 --> 00:19:06,270
damage field twenty-three. That is
the difference between a prompt and a

274
00:19:06,350 --> 00:19:10,243
specification, and it is the
only difference that matters.

275
00:19:10,377 --> 00:19:12,883
Developers worked this out
a while ago, by the way.

276
00:19:12,937 --> 00:19:16,830
They keep the specification in a file
that sits beside the work, called

277
00:19:16,910 --> 00:19:22,190
agents dot m d, or claude dot m d,
so whoever opens the project next

278
00:19:22,270 --> 00:19:25,510
inherits the reasons
and not just the rules.

279
00:19:25,550 --> 00:19:29,870
Different room, same conclusion. Which
brings me to the part I want you to

280
00:19:29,990 --> 00:19:34,723
take to work. A prompt that only
works when you run it is a skill.

281
00:19:34,777 --> 00:19:38,150
A prompt that anybody
can run is an asset.

282
00:19:38,190 --> 00:19:40,963
Your organisation cannot
see the first one.

283
00:19:41,017 --> 00:19:44,830
It shows up as one person being
slightly faster at something, which is

284
00:19:45,310 --> 00:19:49,070
invisible, and which earns you more of
the same work rather than credit for

285
00:19:49,150 --> 00:19:51,910
it. The second one
has a name on it.

286
00:19:51,950 --> 00:19:55,990
It gets handed to a team. It
gets pointed at in a review.

287
00:19:56,030 --> 00:19:59,257
Same forty fields. Same
afternoon of thinking.

288
00:19:59,363 --> 00:20:03,710
Completely different thing to own. One
last practical piece and then I will

289
00:20:03,790 --> 00:20:08,790
let you go. Before you change a single
rule, go and find three documents you

290
00:20:08,830 --> 00:20:12,750
have already done by hand, with the
answers you know are right sitting

291
00:20:12,910 --> 00:20:15,830
beside them. That
is a regression set.

292
00:20:15,870 --> 00:20:19,310
It takes twenty minutes to assemble
and it is the difference between

293
00:20:19,350 --> 00:20:23,870
tinkering and engineering, because
now when you change something you can

294
00:20:23,950 --> 00:20:26,750
re-run all three
and see what broke.

295
00:20:26,830 --> 00:20:30,510
Notice that the regression set has
no product name attached to it.

296
00:20:30,590 --> 00:20:34,510
Every other piece of this is called
something different depending on where

297
00:20:34,590 --> 00:20:40,270
you are sitting. That one is just a
folder, in every tool, forever, which

298
00:20:40,310 --> 00:20:43,430
tells you which part of
today was the durable part.

299
00:20:43,710 --> 00:20:46,510
Nora found hers in a
folder she already had.

300
00:20:46,990 --> 00:20:50,750
Everybody has that folder.
So here is the afternoon.

301
00:20:50,990 --> 00:20:54,710
Print your target. Sort the
fields into four columns.

302
00:20:54,910 --> 00:20:59,230
Count what is in the Interpret column,
then go through it once more and ask

303
00:20:59,390 --> 00:21:02,270
which of those your own
system already knows.

304
00:21:02,510 --> 00:21:07,470
That number, how many you moved out
of Interpret, is this whole session,

305
00:21:07,550 --> 00:21:10,230
measured. Your
prompt isn't broken.

306
00:21:10,430 --> 00:21:11,800
It was never
designed.