What AI Taught Me About Describing My Own Work

The line in the brief was: "Clean, modern, with a human touch." I had written it. The client had approved it. Six weeks later we were three rounds deep and arguing, politely, about a photograph, and I realised that neither of us could point at the sentence and say which of the two options in front of us it ruled out. It ruled out nothing. It had never ruled out anything. It had been a mood, agreed to by two people who each assumed the other meant what they meant.

What made me go back and look at that sentence properly was not a design book. It was spending a year writing prompts for generative models and discovering that the model does exactly what my brief did: it takes a vague description and fills the gap with its own defaults, confidently, and then you find out what it assumed.

The list that made it obvious

NASA publishes guidance on how to write a requirement. It includes a list of words you are not allowed to use, because they cannot be verified. Here it is, verbatim, and I would like you to read it as a designer rather than an engineer:

"flexible, easy, sufficient, safe, ad hoc, adequate, accommodate, user-friendly, usable, when required, if required, appropriate, fast, portable, light-weight, small, large, maximize, minimize, robust, quickly, easily, clearly"

The same guidance also bans "indefinite pronouns (this, these) and ambiguous terms (e.g., 'as appropriate,' 'etc.,' 'and/or,' 'but not limited to')". Its list of characteristics of a good requirement includes "clear and unambiguous", "concise and simple", "free of implementation specifics", "measurable, verifiable" and "stated positively".

Almost every design brief I have written or received breaks most of those rules in the first paragraph. "Clean" is the same species of word as "robust". "Modern" is "appropriate" wearing better clothes. "With a human touch" is "user-friendly" with the serial numbers filed off.

The formal version of this lives in ISO/IEC/IEEE 29148:2018, Systems and software engineering, Life cycle processes, Requirements engineering, second edition, published November 2018, which specifies the processes and required information items for requirements work. Design does not have an equivalent standard and I am not arguing that it should. But it is worth knowing that another discipline has spent decades on the problem of writing down what you want, has concluded that the hard part is precision rather than eloquence, and has published its findings.

A page of field notes. Observation and interpretation kept visibly apart. University of Edinburgh Heritage Collections, CC BY 3.0, via Wikimedia Commons.

Where the engineers admit defeat, which is also useful

The obvious objection is that design is not a missile. Frederick Brooks made that objection about software forty years ago and he was right. In No Silver Bullet: Essence and Accidents of Software Engineering (1986) he wrote the sentence that has been quoted at every project kick-off since:

"The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is so difficult as establishing the detailed technical requirements..."

And then, crucially, he refused to conclude that the answer was more paperwork:

"it is really impossible for a client, even working with a software engineer, to specify completely, precisely, and correctly the exact requirements of a modern software product before having built and tried some versions of the product he is specifying."

Brooks's own diagnosis of the client is blunter than anything I would say out loud: "the client does not know what he wants. He usually does not know what questions must be answered, and he almost never has thought of the problem in the detail that must be specified."

So the goal is not a perfect brief written in advance. Brooks's answer was to build something and look at it. The Design Council's Double Diamond, introduced in 2003, puts the same instinct into a diagram: its first half "helps people understand, rather than simply assume, what the problem is", and only the Define stage narrows to a challenge worth solving. Discover, Define, Develop, Deliver. Two of those four stages happen before anyone makes the thing.

The useful position is somewhere in between. You cannot specify everything up front. You can specify far more than most briefs do, and you can notice which parts you are unable to specify, because those are the parts you do not yet understand.

Why prompting exposed it

A prompt has a property a brief does not: it executes immediately, and you can see what it assumed. Write "clean and modern" to a generative model and it will produce the mean of everything captioned that way, instantly, at no cost, forty times. The ambiguity is not deferred to week six. It is in front of you in ten seconds.

Anthropic's own prompting documentation makes the brief analogy explicitly: "Think of Claude as a brilliant but new employee who lacks context on your norms and workflows. The more precisely you explain what you want, the better the result." Its stated golden rule is a test I have now stolen for client work: "Show your prompt to a colleague with minimal context on the task and ask them to follow it. If they'd be confused, Claude will be too."

Replace "prompt" with "brief" and "Claude" with "the studio". That is the test. Most briefs fail it, including mine, and the failure is invisible because everyone in the room shares enough context to paper over the gaps until the moment a decision depends on them.

Before and after

These are real lines from my own briefs, rewritten. The rewrite is not longer because longer is better. It is longer because a sentence that can be checked takes more words than a sentence that cannot.

The column that matters is the third one. A specification is not a description, it is a decision procedure. If a line in your brief cannot resolve any future argument, it is decoration.

Test for a brief line

1. Name two plausible executions of this project.

2. Does the line accept one and reject the other?

     No  -> the line is a mood. Keep it if it helps
            morale, but do not count it as a requirement.
     Yes -> the line is a specification. Write down the
            check you just performed. That check IS the
            line.

3. For every line that failed, ask: do I not know yet, or
   do I know and have not said it?

     Do not know -> that is a research task, name it as one
     Know        -> say it

Test for a brief line

1. Name two plausible executions of this project.

2. Does the line accept one and reject the other?

     No  -> the line is a mood. Keep it if it helps
            morale, but do not count it as a requirement.
     Yes -> the line is a specification. Write down the
            check you just performed. That check IS the
            line.

3. For every line that failed, ask: do I not know yet, or
   do I know and have not said it?

     Do not know -> that is a research task, name it as one
     Know        -> say it

Test for a brief line

1. Name two plausible executions of this project.

2. Does the line accept one and reject the other?

     No  -> the line is a mood. Keep it if it helps
            morale, but do not count it as a requirement.
     Yes -> the line is a specification. Write down the
            check you just performed. That check IS the
            line.

3. For every line that failed, ask: do I not know yet, or
   do I know and have not said it?

     Do not know -> that is a research task, name it as one
     Know        -> say it

What the evidence says about describing a thing

Here is where I want to be careful, because there is a genuine research literature here and it does not all point the same way.

Explaining to yourself helps you learn

Bisra, Liu, Nesbit, Salimi and Winne published Inducing Self-Explanation: a Meta-Analysis in Educational Psychology Review in 2018. They pooled 69 independent effect sizes from 64 research reports covering 5,917 participants, and found an overall effect of g =.55, with a 95 per cent confidence interval from.45 to.65. Self-explanation, in their definition, is "a process by which learners generate inferences about causal connections or conceptual relationships" directed at oneself. The intervention is simply being prompted to explain, rather than being given an explanation.

One sub-finding is pointed enough to change how I run reviews. Effects were larger when the control group got no additional explanation (g =.67) than when the control group was given an instructional explanation (g =.35). Being told is worth less than working it out. A brief written by the studio and handed over is a weaker artefact than a brief the client has been made to articulate.

Trying to explain reveals how little you knew

Rozenblit and Keil's The misunderstood limits of folk science: an illusion of explanatory depth, published in Cognitive Science in 2002, is the study I think about most. People rate how well they understand something, then write a detailed explanation of it, then re-rate. The ratings fall. They describe the illusion as believing you understand with "far greater precision, coherence, and depth than they really do".

The domain effect is the part that gets left out of the popular retelling and is the most useful part for a studio. The collapse in confidence showed up for devices and natural phenomena, dropping by something in the order of 0.7 to 1.0 points on a seven-point scale between the first rating and the rating taken immediately after writing the explanation. For procedures there was, in the authors' words, "no drop in the self-ratings across time". Facts dropped only modestly. Narratives behaved much like procedures.

So the illusion is not general. It attaches to things with hidden mechanisms. A design brief is a mechanism dressed as a narrative: it reads like a story about a brand, and it functions like a machine that is supposed to produce decisions. That is exactly the combination that lets you feel you understand it while you do not.

The counter-evidence, reported honestly

There is a well-known effect that runs the other way, and it would be dishonest to write this piece without it. Verbal overshadowing, reported by Schooler and Engstler-Schooler in 1990, is the finding that describing a face in words makes you worse at recognising it afterwards. In the original study, participants who verbally described a bank robber were about 25 per cent worse at picking him out of a lineup than controls.

If that generalised, it would be an argument against ever writing a brief about something visual.

It does not generalise cleanly, and the replication record is genuinely interesting. A Registered Replication Report published in Perspectives on Psychological Science in 2014, volume 9, issue 5, pages 556–578, coordinated 31 laboratories for its first study and 22 for its second. The two studies differed only in timing. In the first, participants described the face immediately after the video and then waited 20 minutes before the lineup: the pooled effect was a drop of 4.01 per cent, with a 95 per cent confidence interval running from a drop of 7.15 per cent to a drop of 0.87 per cent. In the second, participants waited 20 minutes and then described the face immediately before the lineup: the pooled effect was a drop of 16.31 per cent, confidence interval from 20.47 to 12.14 per cent, and it held consistently across all 22 labs.

So: the effect is real, it is smaller than the original 25 per cent, and its size depends heavily on when the describing happens relative to the judging. The authors' conclusion is that there is "a robust verbal overshadowing effect that is strongly influenced by the relative timing of the tasks".

I am not going to pretend this maps neatly onto design briefs. It is about eyewitness recognition of an unfamiliar face, which is a memory task, not a specification task. But the timing finding is suggestive enough to act on cheaply: I no longer write descriptive language immediately before a selection round. Describe early, then look. Not look, then describe, then choose.

What I actually changed

Three things, none of them clever.

The brief has two columns. One column for lines that can reject a plausible execution. One for lines that cannot. The second column is allowed to exist and is labelled "intent". It is not allowed to be cited in a review.

The client writes the first draft of the criteria. Not the whole brief, just the list of what would make them say no. This is the self-explanation finding applied directly: being handed our explanation is worth less to them than producing their own. It is also the fastest way to find out that two people at the same company have different criteria, which is information you want in week one and not week six.

Every adjective gets challenged once. Somebody in the room reads the brief aloud and stops at every evaluative word to ask what it rules out. It is a tedious meeting. It is shorter than round three.

What I got wrong, and what I still do not know

I spent a long time believing that vagueness in a brief was a kindness: room for the designer to work, trust rather than instruction. That was self-flattering. In practice the vagueness was mine, not a gift to anyone, and the room it left was filled by whoever had the strongest opinion in the last meeting before the deadline.

What I do not know is whether the specified brief produces better work or merely fewer arguments. I have no measurement. I know it produces fewer rounds, because I count rounds. I know clients sign off faster. Neither of those is the same as the finished thing being better, and I have seen at least one project where the tight brief closed off an option I now think we should have taken. The honest position is that specification reduces variance, and reducing variance cuts the bottom off as well as the top. That is a trade, and I would like to be able to say I have measured it. I have not.

Sources

• NASA, "Appendix C: How to Write a Good Requirement", NASA Systems Engineering Handbook reference material. nasa.gov/reference/appendix-c-how-to-write-a-good-requirement

• ISO/IEC/IEEE 29148:2018, Systems and software engineering, Life cycle processes, Requirements engineering, edition 2, published November 2018. iso.org/standard/72089.html

• Frederick P. Brooks, Jr., "No Silver Bullet: Essence and Accidents of Software Engineering", University of North Carolina at Chapel Hill technical report 86-020, 1986. www.cs.unc.edu/techreports/86-020.pdf

• Design Council, "The Double Diamond", introduced 2003. designcouncil.org.uk/our-resources/the-double-diamond

• Kiran Bisra, Qing Liu, John C. Nesbit, Farimah Salimi and Philip H. Winne, "Inducing Self-Explanation: a Meta-Analysis", Educational Psychology Review, volume 30, 2018, pages 703–725. link.springer.com/article/10.1007/s10648-018-9434-x

• Leonid Rozenblit and Frank Keil, "The misunderstood limits of folk science: an illusion of explanatory depth", Cognitive Science, 2002. Pre-publication proofs. time.com/wp-content/uploads/2015/02/ioed_proofs.pdf_1.pdf

• V. K. Alogna and 99 co-authors, "Registered Replication Report: Schooler and Engstler-Schooler (1990)", Perspectives on Psychological Science, 2014, volume 9, issue 5, pages 556–578. journals.sagepub.com/doi/10.1177/1745691614545653

• Anthropic, "Claude prompting best practices", Claude developer documentation. platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices

Related reading

• Provenance, Disclosure, and the Mechanical Turk

• Latent Space as a Material

• Notes From an Exhibition

• Arguing With a Talk

More from the Moc Mallen Studio journal