The Model Is No Longer the System
That relationship is beginning to loosen.
The recent DeepSeek V4 comparison is interesting not simply because one model is cheaper than another. The more important observation is that a much smaller, cheaper model can now sit surprisingly close to a substantially larger one on capability, while post-training improvements can move performance far more than another enormous increase in model scale.
The exact leaderboard positions will change. They always do. What matters is the direction.
We are approaching a point where capable inference is cheap enough that the model itself stops being the main economic constraint.
Once that happens, the question changes from:
What is the smartest model we can afford?
to:
What is the cheapest system that can reliably complete the job?
That sounds like a pricing question. It is really an architecture question.
Because a system that can afford ten attempts, multiple models, external tools and independent verification behaves very differently from one that gets a single expensive shot.
The unit of AI economics is moving from inference to verified completion.
And that changes how useful AI systems should be built.
Cheap intelligence changes the shape of the problem
Imagine two models.
One costs $1 to perform a task and succeeds 95% of the time. Another costs $0.03 and succeeds 75% of the time.
If the task is a single prompt with no verification, the first model may be the obvious choice.
But once inference becomes cheap, the second model can be embedded inside a larger process.
It can make an initial attempt, retrieve more information when uncertain, call a deterministic tool, ask another model to challenge the answer, or escalate to the expensive model only when ambiguity remains.
The relevant metric is no longer simply cost per token, or even intelligence per dollar.
It begins to look more like:
That distinction matters because repeated inference is not automatically useful.
Running the same model three times with nearly identical prompts may produce three versions of the same mistake. Cheap compute becomes valuable when it buys different ways of testing the problem.
One model may extract information. Another may critique the interpretation. A retrieval system can return the primary source. A calculator can check the arithmetic. A specialist model can resolve a coding task. A stronger reasoning model can be reserved for the small percentage of cases that remain genuinely difficult.
The model begins to look less like the product and more like one component in a computational workflow.
That is a substantial change.
From one answer to a workflow
The first wave of LLM applications was structurally simple:
user → prompt → model → answerA large part of the current shift toward agents is the replacement of that pattern with something closer to:
event
↓
retrieve
↓
construct context
↓
reason
↓
use tools
↓
verify
↓
update state
↓
decide
↓
actThere may be many model calls inside that sequence.
Some tasks are sequential. Others can run in parallel. One model might classify the incoming event, another extract structured information, another reason about its consequences, while deterministic software checks the result.
This is why comparing AI systems only by the intelligence of their underlying model increasingly misses the point.
Once the model is embedded inside a workflow, system quality depends on several additional questions.
Was the right information retrieved?
Was the context assembled correctly?
Was the task routed to the right model?
Did the model have access to appropriate tools?
Was the result checked?
Did the system preserve relevant state?
Could it recover from failure?
And, critically, can anyone reconstruct what happened afterward?
A more capable model can improve every stage. But it does not remove the need for those stages.
Models become interchangeable resources
This also weakens the case for building an application around a single model provider.
Different forms of work already benefit from different model characteristics.
A cheap model may be sufficient to classify documents or extract structured data. A frontier reasoning model may be worth the cost when the evidence is contradictory. A coding model can be better suited to changing an analytical pipeline. A model from a different family may be useful specifically because its errors are less correlated with the first one.
The architecture starts to resemble routing:
task
↓
router
├── fast / cheap model
├── reasoning model
├── coding model
├── specialist agent
└── independent criticThe interesting engineering problem is therefore no longer just model selection.
It is model allocation.
Which type of intelligence should be used at which stage, and under what conditions should the system spend more?
A good architecture should be able to change those answers without rebuilding the application around them.
This is particularly important in a market where model rankings can change in weeks.
The durable layer is increasingly the orchestration around the models, not the model itself.
Retrieval is not memory
As agents become persistent, another distinction becomes important: retrieval and memory are not the same thing.
Retrieval answers:
What information should be brought into this task now?
Memory answers:
What state should survive from previous tasks?
The difference is easy to blur because both ultimately place information into a model's context window. But architecturally they solve different problems.
Suppose an agent is analysing a company's latest earnings release.
Retrieval may need to find the new filing, the prior quarter's guidance, a transcript, consensus estimates and perhaps relevant competitor results.
Persistent state may contain the current thesis, unresolved questions, assumptions used in a valuation model, earlier decisions and the reasons behind them.
Those should not simply be dumped together into a giant prompt.
A large context window is not a substitute for information architecture.
The useful question is not how much information a model can technically consume. It is how much relevant information the system can reliably select.
Good context might therefore be constructed from something like:
current event
+ relevant primary sources
+ current state
+ assumptions affected by the event
+ previous decisions
+ task-specific tools
= working contextThe context window is temporary working memory.
The system around it determines what belongs there.
Context engineering is part of reasoning
This becomes more consequential as AI systems move into domains where decisions accumulate over time.
Two identical models can produce dramatically different results depending on what they are shown.
Give one model an earnings release and ask what changed, and it may produce a competent summary.
Give it the same release together with the company's previous guidance, the assumptions in a valuation model, the investment thesis, management's earlier claims and the relevant industry data, and the task changes completely.
The model is no longer summarizing a document.
It is reasoning inside a deliberately constructed decision environment.
That makes context orchestration an important part of system intelligence.
The orchestration layer has to decide what is current, what is stale, what should be fetched again from a primary source, what can safely be summarized, what must remain exact, and what previous state is relevant to the present task.
It also has to preserve provenance.
A fact from a filing should not gradually turn into an assumption simply because several agents repeat it.
A useful system should be able to distinguish:
source fact
↓
derived observation
↓
analytical assumption
↓
conclusion
↓
actionThose objects are related, but they are not equivalent.
Once an application starts making decisions rather than merely generating text, that distinction becomes essential.
Tools give models a way to test reality
Language models are particularly useful where information is ambiguous and interpretation matters.
They are much less attractive for tasks where deterministic software already knows how to produce the right answer.
That suggests a natural division of labour:
Models interpret. Tools measure.
A model should not estimate arithmetic in prose if a calculator can perform it exactly.
It should not invent a current market price when an API can retrieve one.
It should not reason abstractly about how a changed assumption affects a valuation if it can run the actual valuation model.
Tool use therefore turns an LLM from an isolated reasoner into part of a larger computational system.
A workflow may look like:
new information
↓
retrieve primary sources
↓
extract changed facts
↓
run deterministic checks
↓
update analytical assumptions
↓
execute model / simulation
↓
interpret results
↓
challenge interpretation
↓
decide whether anything changedThis is also where permissions begin to matter.
Reading a database, modifying a model, drafting an order and executing an order are not the same capability.
An agent architecture can expose them separately.
That is a much more useful notion of control than simply asking the model to "be careful."
Verification is the scarce layer
Once inference becomes cheap, verification becomes comparatively expensive.
Not necessarily in dollars. In design.
Some outputs are easy to verify.
A JSON object can be checked against a schema. A calculation can be recomputed. A quoted number can be reconciled with a filing. A portfolio constraint can be tested programmatically.
Other outputs are fundamentally harder.
Was management's explanation credible?
Does a weak quarter actually challenge the long-term thesis?
Is an apparent pricing problem temporary or structural?
There may be no unit test that immediately returns red or green.
This is where agent systems need more than model confidence.
A model saying that it is 95% certain does not constitute independent evidence.
Verification may instead involve deterministic checks, comparison with primary sources, adversarial critique, alternative models, explicit assumption tracking, historical cases or human review.
And because many of these systems are changed continuously, verification has to exist at another level too: evaluation of the system itself.
Offline evals can answer whether a new model, prompt, retrieval strategy or workflow is actually better on representative tasks.
Did it find the right information?
Did it miss contradictory evidence?
Did it preserve citations correctly?
Did it change an assumption without justification?
Could another agent detect the error?
Online evaluation can then observe production behaviour: tool failures, retrieval misses, escalation rates, latency, cost, contradictory outputs and decisions that later had to be reversed.
This is how a benchmark improvement turns into a system improvement.
Without that layer, it may simply produce a more capable model operating inside the same flawed process.
Guardrails belong around capabilities
The same architectural shift changes how guardrails should be thought about.
In many discussions, a guardrail means restricting what a model is allowed to say.
For an agentic system, a more consequential question is what it is allowed to do.
A system might freely allow a model to reach an unusual conclusion while placing strict limits around operational capabilities.
For example:
read source material allowed
query external data allowed
run analysis allowed
propose state change allowed
modify canonical state requires verification
recommend consequential action requires additional checks
execute external action restricted
override hard limit prohibitedThis distinction becomes increasingly important as AI moves from answering questions to controlling software.
The system can permit intellectual flexibility while maintaining operational discipline.
That is a much more robust model than trying to encode every acceptable outcome into a prompt.
The uncomfortable problem: agents can fail convincingly
There is another reason the system around the model matters.
Agent failures are often difficult to see.
A programming agent that breaks a function may eventually encounter a compiler error or a failing test.
Many real-world tasks do not offer that luxury.
An agent can retrieve the wrong source, omit an important piece of context, make a plausible inference, store it as persistent state and continue operating from that false premise.
Nothing necessarily crashes.
The system may even become increasingly internally consistent.
That is why observability matters.
For any consequential decision, it should be possible to reconstruct something close to:
trigger
→ retrieved evidence
→ constructed context
→ model calls
→ tool calls
→ verification
→ state changes
→ resulting actionThe goal is not to preserve every token forever.
It is to make important decisions inspectable.
What caused the system to change its view?
Which source introduced the new information?
Which model interpreted it?
Which assumptions changed?
What checked the conclusion?
Which downstream state was modified?
This is roughly the equivalent of tracing in distributed software systems. Once an AI application contains multiple models, tools, retrieval layers and persistent state, it begins to need the same kind of operational discipline.
And in domains where mistakes can remain plausible for a long time, that discipline matters even more.
Finance is an unusually difficult test case
Financial analysis makes all of these problems visible at once.
The input is heterogeneous: filings, transcripts, market data, models, news, prior assumptions and portfolio state.
The workflow is persistent: yesterday's conclusion affects today's analysis.
The answers are partly objective and partly interpretative.
And the final output may eventually become an external action.
There is rarely one prompt that captures the entire problem.
A new earnings report may trigger retrieval, reconciliation with earlier guidance, model updates, competing interpretations, risk checks and perhaps no action at all.
The difficult part is not generating another summary of the quarter.
It is maintaining continuity between evidence, assumptions, conclusions and decisions.
This suggests a useful architectural separation.
The research agents, reasoning models and analytical tools can remain relatively interchangeable.
The durable component is the layer that owns the process around them: events, state, context, provenance, verification, permissions and action.
That is the layer we have been calling Tickoni.
Not another financial model, and not an attempt to predict which AI provider wins.
More like a financial harness around whatever intelligence happens to be useful.
Its architecture can therefore stay relatively stable while the models underneath it change quickly:
┌── research agents
├── reasoning models
market + company ├── analytical tools
events ───────────────→ harness ───────────────→ portfolio / broker
├── retrieval
├── evaluation
├── verification
└── persistent stateThat pattern is not unique to finance. Finance simply makes its value unusually easy to see.
The model gets cheaper. The system gets more important.
The DeepSeek comparison is interesting because it shows what happens when intelligence moves down the cost curve.
A system can afford more attempts.
More specialised models.
More independent interpretations.
More retrieval.
More continuous monitoring.
More verification.
But this creates a paradox.
The cheaper intelligence becomes, the less useful it is to think of the model as the application.
More intelligence means more outputs to coordinate, more context to manage, more tools to govern and more state that can become inconsistent.
The engineering challenge moves outward.
The important questions become:
What triggered the work?
What evidence did the model see?
Why was this particular model used?
What tools were available?
What was independently checked?
What changed in persistent state?
What actions were permitted?
Can the entire path be reconstructed later?
Models will continue getting smarter. Some will become much smaller. Some will specialise. Prices will continue moving.
The precise frontier will change again.
But if capable intelligence really is becoming abundant, then intelligence itself becomes a weaker source of differentiation.
The scarce layer becomes everything required to turn it into a reliable outcome.
Not merely an answer.
A verified, observable and controlled completion.
Comments
Post a Comment