Let me show you a task instead of an architecture. Small one. The kind that lands on a data team at a fund a dozen times a week and usually eats an afternoon. Watch where the human shows up, and where they don’t.
The task: the desk just got API access to a new prime broker, and someone needs margin data out of it. Brand-new feed, nobody on the team has touched it, and the records are dirty in the way every PB file is dirty. The question attached is simple. Which accounts are over their margin limit this month? In the old world that’s a Symphony message, a ticket to whoever owns the broker integrations, an afternoon reading API docs to work out what the fields even mean, a throwaway script, and a number pasted back into the thread. Here it’s a prompt.
”Here’s the prime broker feed. Tell me which accounts are over their margin limit.”
That’s the whole instruction. A pointer to the endpoint and a question. No schema, no field spec, nothing to configure. The agent does the obvious first thing a decent analyst would. It goes and looks at the data instead of guessing about it.
It writes a small connector for the feed, the bit of code that authenticates, pages through the endpoint, and pulls records, then runs it against a sample and reports what it found. Field names, types, row counts, the problems. The as-of dates came across as YYYYMMDD integers, not dates. The sub-account field is blank on a couple thousand rows, the pending-allocation give-ups anyone who’s worked that flow will recognize on sight. (Every PB file I’ve touched in twenty years arrives broken in some quiet way. This one managed two.) It doesn’t paper over any of that. It tells me straight, the way you’d want a teammate to.
This is already a different thing from a copilot. Nobody filled out a “describe your source” form. Nobody mapped a field or wrote a line of that connector by hand. The agent wrote it, ran it, read what came back, and built its own picture of the feed.
Catching the bad rows, reshaping the rest
Next it proposes the cleanup in plain language, before it does anything irreversible. Hold back the rows with a blank sub-account rather than drop them, since an unallocated position can still be the breach I’m hunting for, and surface the unattributed total so I can see what I’m setting aside. Parse the YYYYMMDD integers into real dates.
Those plain-English rules are the actual input. The platform turns them straight into the validation and transform code that runs on every row, generated rather than hand-written. The description isn’t a summary of some script hiding underneath; it is the specification. I don’t have to read the generated code, and the agent doesn’t push it at me, but if I want to see it I ask and it shows me. And if the hold-back rule is catching rows it shouldn’t, or the date parse trips on a malformed field, I say so and it adjusts. I’m testing a concrete proposal against my real data, not arguing with a chatbot in the abstract.
It doesn’t stop and wait for a formal sign-off. It tells me the plan, and unless I cut in, it builds. That’s the right default here, because nothing’s been written to the destination yet and the plain-English rules I just read are exactly the ones that run. There’s no gap between the description and the code where the meaning drifts.
Standing it up and watching it run
Now the agent builds the pipeline, and this is the step where demos and real tools part ways, so I want to be precise about it.
It creates the whole thing in a single create_pipeline call. Source, validation, transform, destination, committed together. There’s no window where a half-built pipeline sits around waiting for someone to remember to finish it. Either the pipeline exists and is correct, or it doesn’t exist.
Then it runs the job and waits. It doesn’t kick off the run and declare victory at submit time, which is the most common way an agent lies to you without meaning to. It polls the job to completion, watches the state, and reports back only when the work has really finished or really failed. If it fails, I hear why. If it succeeds, I get the numbers: a little over 84,000 position rows in, 2,310 held back as unattributed. That held-back count is the one I actually care about. An unallocated position is exactly where a breach hides, and if it looks wrong for the month I want to know before I trust the answer, not after.
The answer, over the same surface
The data is clean now, structured, sitting in the destination. The original question hasn’t moved. Which accounts are over their margin limit this month?
The agent queries the output it just produced and answers, over the same surface it used to build everything else. The pipeline that created the data and the query that reads it back aren’t two separate tools with a human carrying results between them. It’s one loop, and the agent ran all of it.
Count the human moments
Look back at what I did. I gave it a goal. I read the cleanup plan it proposed, and could have rewritten any of it in a sentence; this time I let it run. I checked the held-back count. I read the list of accounts.
What I didn’t do: open a form, read the PB’s API docs, write the connector, name fields, hand-write a date parser, configure a destination, paste a query into a console, or babysit a job. That list used to be the afternoon. It’s the part an agent-native platform is built to absorb.
This is the line between an agent that narrates a workflow and one that runs it. A narrator hands you a plan and leaves you holding the work. This one did the work and left me the judgment calls: whether to trust a feed I’d never seen, whether holding back the unallocated rows was the right call, whether the final list of breaches actually looked like our book. I’ll take that trade. It only works when every step of the loop is something the agent can reach out and call for itself: the connector, the pipeline, the run, the query.
Todd Fearn is the founder and CEO of Datris, an open-source, agent-native data platform. He has spent three decades building data infrastructure for financial services, with stops at Goldman Sachs, Bridgewater, Deutsche Bank, Salomon Brothers, and Freddie Mac.