Agentic AI has collapsed a few layers in the traditional software development process. Particularly the claim is that you can develop a feature of a software just by writing the specification.

Already there are many frameworks written as skill bundles that direct agents how to do that. Read a spec and produce all the artefacts you could expect. A working prototype or complete software with tests and documentation.

But there is a difference between doing this with or without rigour. The looser the specifications the higher the chance of incomplete, buggy or totally irrelevant output. Otherwise SDD is still subject to the “GIGO” principle. “Garbage in Garbage out”.

But what makes a good spec over a bad one? I am going to take some well known popular applications and spec them out and let agents drive the implementations to see the results.

Agentic coding is a phenomenal experience. I’ve been working with AI coding agents for a year now, primarily using Claude Code after trying a variety of models. But there’s one well-known problem — they have a ‘context window’ - a limited memory area for what they’re holding in their ‘agentic minds’. Context is nearly everything because it guides them to approach the task from the proper perspective and captures the small details that wire projects together, keeping the process focused and relevant.

What constitutes context? It could be the code already written, high-level architectural ideas, pitfalls to avoid, or observations about the domain the problem is trying to solve. “Code” alone does not capture everything. My earlier approach was to write extensive specs and capture them as docs, which I had to constantly reference when working on a task. It wasn’t scaling, and it wasn’t enough. The agents frequently forgot key details I had decided were essential and wrote docs on, but they weren’t always bringing that knowledge to bear… creating weird spaghetti. Spaghetti that solved the problem at hand but was inconsistent with earlier pieces. I solved most of this by following rigid structures (like templates) for different parts of the codebase, which nearly solved the inconsistency issue, but not entirely.

Depending on the programming stack you use, there’s still information loss about the domain under consideration unless you meticulously document it in the project. The agent wasn’t really “learning”. This is not a meta discussion about what “learning” is and whether they really do. My insight was that they lack “knowledge” - the kind we bring to bear when we learn a codebase and the domain it’s operating in — the little trickles of insights gained that turn into a knowledge repository. What they needed was a way to build this “knowledge” themselves, through working on a project, and to bring it to bear on the problem at hand.

How did I solve this without agents before? The answer was actually so simple. I used structured notes - a scratchpad filled with entries about different aspects of the codebase or domain. For example, I documented data flow, persistence methods, and design choices—things that happen all the time in artisan coding. These notes acted as a personal memory aid, helping me recall critical details quickly and consistently during development.

But what do I mean by structured notes? You need to distinguish between free-form writing and structured writing. Free-form writing about a problem solved in a project would be like a couple of paragraphs explaining what was solved. But depending upon how good you are at keeping all relevant details, the free-form approach becomes lossy and inconsistent. So I follow a system where I design a template for each type of learning that I have. For instance, a “problem solved” template might capture the problem, the root cause, and what was learned. Or a “design principles” template might have fields for the principle itself (limited to 150 characters), the rationale behind it, and what anti-pattern it prevents. The template forces me to list the relevant details. And I design little templates to use for other things, like “patterns discovered” or “workarounds learned”. These are, by necessity, brief and distinct from project-level documentation. Having a structure enhances consistency and makes the gathered knowledge really valuable.

Now, how do I make the agents follow such a system? Define templates, and use them whenever the context dictates, preferably in a seamless way… like when I think and jot them down.

Enter Pensieve, a scratchpad for AI agents to use - a command-line interface (CLI) tool that agents can use to define templates and record notes in them and recall them as needed. Agents are very good at using CLI-based tools, and they automatically discovered how to use them with a little prompting. The agents designed templates in collaboration with me for each project I was working on and started filling them in as we tackled tasks together. And slowly, as we started new sessions, the agent started recollecting past learnings, informing the task at hand and making it more guided and precise than before. You no longer needed to yell the same information over and over.

But why exactly is this better than using CLAUDE.md or docs in the repo? It is because they are unstructured, and the agents follow those instructions in an inconsistent manner. The more you have in this memory, the less likely the agent is going to find it and use it. Also, without specifying limits, the agents generated verbose, pointless documentation which they themselves did not follow. By forcing the agents to create structured notes with well-defined fields and LIMITING the length of them as needed, more disciplined knowledge capture happened.

Besides templates, Pensieve also encourages agents to link entries with each other. Nothing is lost forever — a note can be “augmented,” “superseded,” or “deprecated” by another note. In most cases, this is sufficient, and the agent is encouraged to follow these links when inspecting a note. This allows the agent to crawl through memory AS NEEDED for the task at hand, digging deeper and deeper into memory, building a “graph of sorts” in its agentic mind.

I no longer have the repo for the same open. Instead, I plan to share a spec of the tool, which should be sufficient for any agent to design one.

Large Language Models (LLMs) have unleashed a Cambrian explosion of software. Over the decades, coding was partly considered elitist, despite efforts to make it accessible to more people. Many people became coders without calling themselves that. Writers and script writers were never afforded the title, despite, for all intents and purposes, they were indeed “coding”.

Still, there was a class of software considered inaccessible to them. You needed an elaborate ritual of Software Engineering professions—from Product designers, analysts, managers, developers, and QA—to build these classes of software. Cloud apps, mobile apps, SaaS products.

The gate that prevented them had nothing to do with the complexity of the problem that they were trying to solve in a domain. The complexity of the business, be it payment processing workflows, scheduling apps, delivery apps with their own niche problems, was something product builders were able to dissect and find elegant solutions. But the translation of their solutions into software was blocked by something much more insipid.

It was the sheer amount of accidental complexity Software Architects and engineers had created over the decades. These abstractions were just endless indirections that tackled particular problems sometimes meaningful, many times not so. It still forced those looking to translate their solutions to software to deal with them.

What are React Hooks and why do you need them? What is pydantic for? What is a REST API and why do I need it?

These are worthwhile questions, but the answers and the rabbit holes of knowledge required to master them bore both deep insights and rote knowledge that needs translation and execution. The latter of which LLMs offer to solve. They can never really solve the former except to rehash pre-discovered wisdom.

How do we deal with this new world? Is it an all-or-nothing approach to go for? Is a new development approach on offer? Has a new type of coder emerged? We will set out to find out here.