Yes, Claude can build it. Your edge is knowing what good looks like.
Across 398,198 interactive Claude Code sessions, people made most planning decisions while Claude made most execution decisions. Sessions that showed clearer task knowledge also met Anthropic's transcript-based success measures more often. The finding is not that expertise guarantees good output. It is that when more execution is delegated, the result, constraints, proof, and stopping condition still have to be defined.
What the study saw
398,198 interactive Claude Code sessions, classified for expertise, decisions, and transcript-based success.
What it means
If an agent can execute more of the work, where does human judgment move?
The boundary
The study saw transcripts and classifier judgments, not deployed quality or economic value.
An accountant asks an AI coding agent to automate month-end reconciliation. The hard part is not writing Python. It is knowing which transactions may be matched, which exceptions require a human, what audit trail must remain, and what should stop the run.1
The competence step
Most of the measured gain arrived before mastery.
398,198
interactive sessions sampled
Adjusted verified success rose from 14.5 percent in novice-rated sessions to 28.3 percent at intermediate, then to 29.5 percent at advanced and 32.9 percent at expert. Most of the observed difference appeared between novice and intermediate.12
Adjusted verified success
0 to 40%- 14.5%Novice5,911 sessions
- 20.9%Beginner73,858 sessions
- 28.3%Intermediate96,516 sessions
- 29.5%Advanced146,879 sessions
- 32.9%Expert36,422 sessions
Most of the observed difference appeared between novice and intermediate.
A separate occupational comparison
34% software and math versus 29% other identified professions
Verified success in code-producing sessions; occupation was inferred for about 70% of sessions.
Who made the decisions?
People led planning. Claude led execution.
Approximate average shares attributed by Anthropic’s transcript classifier. Planning and execution are separate decision categories, not measures of time or total work.
Anthropic uses a version of this example to explain task-specific expertise. The accountant may be new to Python and still be the expert in the job. A senior engineer asking a first question in an unfamiliar language may be the beginner. Here, expertise means being able to state the task precisely, ask for the right checks, and notice when the result is wrong.12
In Anthropic's sample, people outside software occupations were using Claude Code for technical work. The useful question is no longer only whether the agent can produce code. It is whether the person directing it can define a result worth producing.1
What expertise meant
A working grasp captured most of the observed advantage
The researchers rated the apparent expertise shown in each session. They looked for precise directions, targeted verification, and whether the person corrected Claude or Claude corrected the person. The rating was task-specific rather than a judgment about the user's job title.12
Sessions rated novice reached Anthropic's strictest verified-success measure at an adjusted rate of 14.5 percent. Beginner-rated sessions reached 20.9 percent, intermediate sessions reached 28.3 percent, and advanced and expert sessions reached 29.5 percent and 32.9 percent. Most of the difference appeared between novice and intermediate.12
Occupation mattered less than the coding label suggests. In sessions that produced code, software-related occupations reached verified success in about 34 percent of sessions, compared with about 29 percent for other identified professions. Every one of the ten largest identified occupation groups landed within seven percentage points of software and math users.1
This does not show that anyone can build reliable software without technical review. It supports a narrower claim: domain knowledge can help someone direct technical execution even when writing code is not their profession.1
The division of labor
People chose what. Claude chose how.
Anthropic analyzed 398,198 interactive Claude Code sessions from 234,751 users between October 2025 and April 2026. The sessions came from Claude Code's command-line interface, Claude.ai, and its desktop app. The analysis excluded headless use, software development kits, and third-party IDE integrations.12
The researchers used classifiers to identify meaningful decisions in each transcript. They separated planning decisions, such as what to build, which approach to take, and what counted as done, from execution decisions, such as which files to change, what code to write, and which commands to run.12
In a typical session, people made about 70 percent of the planning decisions. Claude made about 80 percent of the execution decisions.12
The person still set much of the direction while the tool handled much of the implementation. The division of labor changed, but responsibility for deciding what counted as good work did not disappear.1
A better definition of done
Write the result, constraints, proof, and stop condition
When execution becomes easier, judgment moves to the boundaries of the task. Before the agent starts, someone must define the result and the constraints. After it acts, someone must decide whether the evidence is good enough. The study did not test the following workflow as a recipe. It is our practical inference from the behavior Anthropic measured.12
- 01
Required result
A reconciliation report that flags every unmatched transaction and explains why it was flagged.
- 02
Legitimate exception
Canceled orders should be excluded from missing-payment alerts, but retained in the audit trail.
- 03
Proof before acceptance
The total must reconcile to the known finance report, and every unmatched record must retain its source ID.
- 04
Stop condition
Do not update the ledger. Stop and show the differences if the totals fail to match.
The strongest limit
The study saw transcripts, not consequences
Anthropic could not observe whether most outputs were deployed, retained, discarded, or economically useful. Expertise, occupation, decisions, and success were inferred largely from the same transcripts. Precise instructions and explicit confirmation can make a session look both more expert and more successful even when external quality is uncertain.12
The appendix includes a partial check on that concern. In a separate sample of Anthropic employee sessions linked to whether commits reached the main branch, the expertise gradient appeared again, including comparisons within the same engineer. The difference was smaller, around three to five percentage points per expertise level.2
That check makes a pure scoring artifact less likely. It still does not show that expertise caused success, or that a commit reaching the main branch produced a good deployed outcome.2
An outside dataset, SWE-chat, adds a different caution. It analyzed nearly 6,000 public coding-agent sessions and found that 44.3 percent of total agent coding effort ended up in associated commits among the 48.6 percent of sessions it could map unambiguously.3
SWE-chat did not test Anthropic's expertise gradient. It reinforces a simpler distinction: generated work and accepted work are not the same thing.3
The evidence does not show that domain experts will always win, or that agents cannot learn more of the judgment themselves. For now, if an agent can build more of the answer, the person still owns the question, the evidence, and the definition of done.1
Aligned
Sources and limits
One company study, its method appendix, and one outside caution
The story keeps the reported association, its measurement limits, and the strongest outside counterpoint together.
- 01
Primary public report
Anthropic: Agentic coding and persistent returns to expertise
Sample, study period, reported rates, classifier summaries, occupation comparison, division of labor, and stated limitations.
Limit: Vendor-authored analysis of Anthropic's own product. The report cannot observe real-world outcomes and relies on transcript classifiers.
Published · checked - 02
Methodological appendix dated June 2026
Anthropic: Agentic coding and persistent returns to expertise, appendix
Exact expertise-level counts and adjusted rates, regression design, classifier validation, controls, model sample sizes, and the partial employee commit check.
Limit: Produced by the same research team and organization as the public report. It is methodological depth, not independent corroboration.
Published · checked - 03
Independent outside context
Stanford University researchers: SWE-chat: Coding Agent Interactions From Real Users in the Wild
Outside context showing that 44.3 percent of total agent coding effort ended up in associated commits among the sessions the study could map unambiguously.
Limit: Opt-in public-repository sample of nearly 6,000 sessions, roughly 85 percent Claude Code. The commit-mapping analysis covers only 48.6 percent of sessions, misses many wholly abandoned sessions, and does not test Anthropic's expertise gradient.
Published · checked
What this analysis does not establish
- This is an analysis of Claude Code sessions, not all AI work.
- The study measured transcript evidence of success, not deployed quality or economic value.
- Expertise, occupation, decisions, and outcomes were inferred largely from the same transcripts.
- Anthropic's employee commit check is partial validation, not independent evidence of causation or deployed quality.
- The associations do not establish that expertise caused the measured success difference.
- The independent counterpoint studies durable code in a smaller, opt-in public-repository sample. It does not replicate Anthropic's expertise result.