When I'm building a Task, it surprises me how little I'm thinking about the AI step.
I ran an experiment last week. Built a competitive intelligence automation. Six phases, seven prompts, scraping, web search, social search, three analysis passes, a summary, a memory write at the end. Complex bit of kit.
The point was to push our AI Task Builder. Feed it a spec (requirements, outcomes, context), let it compile the script, write the prompts, wire the whole thing together. Hands off.
I recorded the session if you want to watch it play out:
The builder is really good at scaffolding complex operations
I've built enough Tasks by hand to know how long the scaffolding usually takes. It's not the interesting part. It's the plumbing. Setting up the scrape, deciding which pages to hit, writing the fail-fast condition when pages come back empty, shaping the intermediate data so the downstream analysis can actually use it, placing each prompt in the pipeline, writing the memory write so the output is findable later.
That scaffolding is the majority of my time, and it can eat up hours if I'm being really careful.
The builder produced a working version in a few minutes. Six phases with clean hand-offs. Seven inline prompts with system and user messages that fit their spots in the pipeline. Fail-fast where it belonged. A summary before the memory write. Checkpoints and retries on request.
Not every decision matched mine. A couple of the phase boundaries felt off and I'll probably restructure them. But the shape was right. And the shape used to be most of the work.
Watching it produce a competent pipeline in the time it takes to make coffee is when it clicked: most of building a Task is structural, not creative. If a builder can produce that structure from a spec, the structure isn't where the value is.
The value's somewhere else.
The AI step is the smallest part of the work
Seven prompts. Out of roughly forty distinct operations in the script.
I sat with that ratio for a while, because it shows up in every Task I've built.
Maybe 10% of my time goes to the prompts. Another 10% to picking models and tuning. The other 80% is everything around the model.
Formatting the input. Deciding what context to inject where. Shaping the output so the next step can consume it without another parsing pass. Choosing where to fail fast, where to retry, where to checkpoint. Working out which parts deserve their own prompt and which should just be a Python function.
I don't think that's a limitation of the tools. I think it's the whole thing.
A model is a function
Text in, text out. The quality is bounded by the shape of what you hand it. Feed it a raw HTML dump, you get raw-HTML-dump answers. Feed it three clean variables and a well-labeled context object, you get clean structured answers doing the reasoning you actually wanted.
Every good step comes down to two questions, over and over:
What does this model need to see to do its job well? What shape does its output need to be in for the next step to work?
Those aren't prompt questions. They're structural. They shape the pipeline underneath the prompts, and that's where most of the reliability lives.
The 80% compounds
An input transformation you got right once gets reused across six Tasks. A memory doc you populated last month gets read by three prompts this month. A retry policy you tuned in one place becomes the default everywhere.
Prompts don't compound. Prompts are the part you re-tune when the model changes.
That's the inverse of most AI writing out there. Prompts get all the attention, and the infrastructure gets treated as plumbing. In real systems the ratio flips. The plumbing is where the reliability comes from. Prompts are the last-mile tuning on top.
Why they feel different
This answers a question worth asking: why does a well-built automation feel different from a chatbot doing the same job?
The chatbot is 90% model, 10% wrapping. The automation is 10% model, 90% everything else.
The AI is the smallest, most impressive-looking part. Everything around it is what makes it worth having.