Why coding agents moved first—and financial research agents are harder

The judgment

Coding agents advanced quickly not because software engineering is easy, but because software gives agents a unusually complete environment for action and feedback.

Code work often has:

• machine-readable state;
• executable actions;
• fast tests and error messages;
• relatively inexpensive isolation and rollback;
• inspectable diffs and version history.

Financial research rarely receives such clean feedback. Company value, industry structure, and investment outcomes may take months or years to reveal themselves. The result is also shaped by external events. A flawed decision cannot be reverted like a code change.

Moving a chat interface and tool calls into finance is not enough to create a financial research agent.

Action requires more than knowledge

An agent that can act reliably needs to understand at least six things:

1. the current state;
2. the actions available;
3. the permissions attached to those actions;
4. the signal that determines whether an action succeeded;
5. the recovery path after failure;
6. the point at which work must stop or return to a person.

Software repositories encode much of their state in files, tests, logs, and version control. The state of an investment case may be distributed across a data room, financial model, management interview, policy document, market data, and an analyst's tacit experience. Those sources can disagree.

If state is invisible, an agent cannot act consistently. If state is not preserved, a long research project cannot resume reliably.

Why tests matter so much

SWE-bench asks systems to resolve real software issues in real repositories. The result is judged through repository state and executable tests, not merely whether the generated code looks plausible.

Tests are not perfect truth. They can miss security, performance, maintainability, and edge cases. Yet they provide software agents with unusually dense, low-latency, repeatable feedback.

Financial research has to construct its own validation environment rather than wait for a natural unit test to appear.

Why the financial verifier is expensive

“Correct” research includes several different layers:

• factual accuracy;
• source qualification;
• consistent numerical definitions;
• reasonable inference;
• adequate treatment of risk;
• fit with the institution, mandate, and decision date.

Tools and deterministic rules can verify parts of the first three. The later layers require domain rubrics, institutional context, and accountable judgment.

Investment return is also an unreliable short-term reward signal. Good process can be followed by a bad outcome after an external shock. Weak analysis can benefit from luck. A system trained to optimize short-term outcomes can learn the wrong lesson.

Six pieces of infrastructure finance must build

Observable research state

Projects, materials, sources, claims, counterarguments, revisions, and the current conclusion must persist outside a single conversation.

Source qualification

A search result, a company statement, third-party data, and independently verified fact should not share one evidence status.

Executable checks

Calculations, citation locations, date alignment, entity matching, and disclosure rules should be delegated to deterministic tools where possible.

Checkpoints and versions

Long tasks need intermediate state, visible failure, and report history so one error does not overwrite the entire project.

Permission boundaries

Internal files, customer data, and public information need explicit access rules. Tool availability is not authorization.

Human escalation

Source conflict, material risk, contextual value judgment, and final commitments require a defined review path.

The report is not the whole product

If a system produces only a final PDF, most of the work remains invisible:

• why particular sources were selected;
• which figures were revised;
• which counterevidence changed the interpretation;
• which questions were left open;
• what the next assignment should inherit.

The value of a coding agent includes the inspectable change process, not just the final code. Financial research needs the same principle: how a judgment formed and changed should remain part of the record.

Boundary

This comparison does not support copying software benchmarks directly into finance. It supports a narrower conclusion: feedback density, reversibility, and observable state are central to agent reliability, and financial research must deliberately build the equivalents it lacks.