Please stop using stupid models

Theo argues that stronger coding models earn their cost by completing longer workflows with fewer interruptions. His useful proposal is to compare complete, tested changes rather than isolated edits. His model rankings, productivity anecdotes, and explanation of exponential improvement are less secure than that proposal.

Actionable Insights

The following experiments are recommendations from this analysis, not commands or measured results from the video.

• Compare narrow edits with complete workflows on the same repository. Create agent-evals.md with a small set of past bugs, their starting commits, reproduction steps, and acceptance tests. For each bug, compare a file-specific edit request with an outcome-based request to investigate, fix, test, and prepare a reviewable PR. Keep the starting code, tool permissions, and budget comparable across models. Record accepted fixes, human minutes, total cost, and regressions rather than rewarding longer sessions. This tests Theo’s 8:38–10:10 argument without assuming his model ranking transfers to your codebase. METR’s evaluation code provides a research reference, not a ready-made evaluation suite for your repository.

• Add execution evidence to one low-risk PR before expanding autonomy. Require the agent to reproduce the original failure, add a regression test, run the relevant existing tests, and report the exact commands and results. For a UI bug, also request a recording of the affected interaction. Keep human merge approval and use an isolated environment without production credentials. Greptile TREX, the sponsored product shown in the video, advertises sandboxed PR testing. Evaluate any such service on confirmed bugs found, false positives, and review time. A recording shows one path through the application, not that the whole change is correct.

• Test model routing against an all-frontier baseline. Start with one bounded task split into planning, implementation, and review. Give a cheaper worker an explicit acceptance test and let the stronger model handle ambiguous planning or failed work. Log retries and integration repairs as part of the total cost. The comments from @CrimsonTheOriginal and @poteb propose this approach; Anthropic’s agent-pattern guide describes routing and orchestrator-workers. Keep the split only if accepted output costs less without increasing defects or human repair time. Parallel workers can create conflicting changes and duplicate context costs.

• Turn repeated failures into repository fixes. After a failed run, inspect the actual tool calls, files read, test output, and first incorrect assumption. Add a reproducible setup instruction, focused regression test, or clearer tool description, then rerun the same task from its original state. Put repository guidance in the instruction file your coding tool actually loads. Compare recurrence and intervention counts before and after the change. At 11:43–12:44 Theo explicitly warns that a model’s explanation of its own mistake may be inaccurate, so do not substitute that explanation for the trace.

• Set a budget and stop condition before trying a longer run. Specify the maximum spend or iterations, when the agent must stop for missing credentials or ambiguous requirements, and which external actions need approval. Ask for a checkpoint with completed tests and unresolved failures, not a declaration of success. Track quota-related downtime separately from API-equivalent cost. Use the calculation net value = value of human time saved - added model cost - review and repair cost as a local decision aid, not a promise of savings. This addresses the video’s labor-cost argument and the commenters’ practical objection that a subscription can run out before the working week ends.

Core thesis

Theo calls the scope of delegated work prompt width. A narrow request names the file and the edit; a wide request starts with a bug report and includes investigation, implementation, verification, PR preparation, and follow-up on review. At 9:39 he describes handing over a screenshot and asking for a tested fix with a video and PR link. The work spans more steps even when each step is ordinary.

His central distinction is between peak capability and consistency. A model that occasionally solves a difficult problem may still interrupt routine work too often. Conversely, fewer routine failures can make broader delegation useful. However, the video’s response-quality curves are illustrations of his impressions, not measured distributions. The evidence does not establish a universal ordering of the named models.

There is also an important concession. At 9:08 he says David Cramer is right for well-specified ticket-to-code tasks. At 3:32–4:03 he accepts that moving away from Astra can feel better because of inconsistent behavior. Those concessions narrow the provocative opening claim that users who do not notice an upgrade are prompting badly.

Key moments and interpretation

• 1:31–3:02: The sponsor segment argues for running proposed changes rather than only reading code. The useful practice is reproducible runtime evidence; claims of product superiority remain advertising.
• 4:03–5:33: Theo distinguishes unusually good results from the frequency of bad results. Evaluate repeated failures and repair effort, not one impressive demo.
• 8:38–10:10: He retries tasks that older models failed, then expands the delegated workflow. This is a sensible experiment if task difficulty and evaluation criteria stay comparable.
• 11:43–12:44: Failed long runs can expose missing setup instructions or confusing repository structure. The stronger claim that every agent problem would also block a new developer is not established.
• 14:49–16:22: He proposes having the agent find relevant files, run the app, and prepare evidence before involving a person. His six-hour success story is anecdotal, not a completion-rate estimate.
• 18:24–19:25: He describes time spent refining skills and acknowledges affordability limits for students, people outside Western countries, and others. That qualification limits how broadly the affordability argument applies.
• 21:27–21:57: He reports a median prompt duration increase from 53 seconds to 2 minutes and 20 seconds, and a P95 increase from a bit under 7 minutes to over 16 minutes and 20 seconds. The underlying logs are not available here. Duration alone does not measure accepted work or productivity.
• 22:27–23:59: He demonstrates compounding failure probability. The basic mechanism is valid under restrictive assumptions, but it does not establish his broader claim about exponential increases in useful runtime.

Comment insights

The extracted comments contain a substantive alternative to the video’s recommendation. @CrimsonTheOriginal, with 318 likes in this snapshot, describes using frontier models for planning and more bounded workers for implementation. In follow-up replies, the commenter clarifies that the model performs the decomposition; this is not necessarily manual micromanagement. @poteb describes a similar split, with cheaper implementation and stronger review. These are practitioner reports, not controlled comparisons, but they show why wide delegation and mixed-model execution are compatible.

Cost objections concern continuity as well as price. @RacoonProds, with 153 likes, says the suggested workflow can exhaust a quota in 24 hours. @Orc1nusOrca reports higher subscription usage for Astra than Sol despite expectations about performance per cost. Neither report verifies a provider’s accounting or current limits. Together they identify a missing operational metric: how much accepted work a team completes before access becomes constrained.

@RCCurtright argues that clear scope and verification loops can make a less capable model sufficient. @shawnnosaurus points to the surrounding workflow and permission to change it. These comments challenge the video’s tendency to attribute outcomes primarily to model choice or prompting skill. @huhwhatguy provides the opposing experience of finding broad codebase review and planning useful with Astra.

The joking revert/reverb comment is not evidence of a real model output. Insults and speculative accusations about compensation do not resolve the technical question. The creator’s own comment explicitly identifies Greptile as the sponsor. Like counts describe this extraction snapshot, and the top-comment sample is not representative of all viewers or developers.

Deep research and verdicts

External pages below were fetched on 2026-10-05. Web search was unavailable, so this is a targeted primary-source review, not an exhaustive survey of current model performance. Older studies provide methodological evidence and counterexamples, not October 2026 rankings.

Longer workflows expose reliability differences

METR’s March 2025 time-horizon research supports the general concern. It reports that success declines with task length and estimates an approximately seven-month doubling trend in the human task duration associated with 50% model success over its historical sample. METR also discusses task selection, measurement uncertainty, and limits on extrapolation.

This is different from measuring how long an agent stays active. METR’s duration is how long an appropriately skilled human takes to complete the task. A model can run longer because it is doing more work, encountering more errors, or simply operating more slowly.

Verdict: Agree with the general reliability argument, with high confidence. Mixed on the video’s model-specific conclusion, with low confidence in any ranking from the evidence available here. METR supports testing longer tasks but does not verify Theo’s Fable, Astra, Opus, or Soul comparisons. The practical takeaway is to measure task completion and interventions at several workflow scopes.

Fewer failures can have large cumulative benefits, but the math needs qualification

At 22:27–23:59 the example assumes a 5% failure probability per 10-minute window, then lowers it to 3%. Recomputing the example with Python gives the following values under independent windows with a constant failure probability and no recovery:

The 30-minute failure probability at 95% per-window success is 14.26%, rather than exactly 15%. After four hours, the success improvement is about 18.94 percentage points. Reducing failure from 5% to 3% is a 2-percentage-point absolute reduction and a 40% relative reduction, not merely a 2% relative improvement.

For this simplified model, success after n windows is p**n, and the 50% success horizon in minutes is 10 * log(0.5) / log(p). The horizon rises from approximately 135.13 to 227.57 minutes, about 1.68 times. Success decays exponentially with the number of windows. That fact alone does not imply that the horizon grows exponentially with a linear improvement in per-window reliability. Real agent errors can also be correlated, detected, retried, or repaired.

Verdict: Mixed, with high confidence. The compounding effect is real within the stated model, and most of the video’s rounded examples are directionally reasonable. The leap to exponential runtime improvement is overclaimed. METR’s separately measured historical exponential trend does not validate that mathematical leap. Use measured success curves for your tasks rather than treating this illustration as a forecast.

Expensive models can save labor, but savings are conditional

At 10:10–10:41 Theo argues that additional tokens can be worth paying for if they save valuable engineering time. That is a valid comparison, but the hypothetical threefold productivity increase is not a measured result.

METR’s July 2025 randomized study supplies a useful counterexample to assuming time savings. It studied 16 experienced open-source developers on 246 issues, primarily using Cursor Pro with Claude 3.5/3.7 Sonnet when AI was allowed. Developers took 19% longer with AI, despite believing afterward that it had made them faster. METR explicitly says the finding does not establish that AI slows most developers, that newer systems cannot help, or that better workflows could not improve outcomes.

Verdict: Mixed, with high confidence that the economics depend on measured outcomes and limited confidence about the named current models. The study challenges intuition as a productivity metric, not the possibility that Theo benefits from his own setup. Count review, preparation, repair, and subscription downtime before deciding whether a higher tier pays for itself. Do not assume the video’s API-equivalent subscription figures describe your plan.

Broad delegation does not require the strongest model for every step

Anthropic’s Building effective agents recommends the simplest adequate solution, describes routing easier work to cheaper models, and distinguishes fixed workflows from agents that choose their own steps. Its orchestrator-workers pattern resembles the commenters’ planning-and-delegation approach. It also warns about latency, cost, compounding errors, and the need for sandbox testing and limits.

This is vendor engineering guidance rather than an independent comparison. It supports an alternative to choosing one model for everything. A strong planner can delegate a wide task while using cheaper workers selectively.

Verdict: Disagree with the blanket inference that not noticing an upgrade proves poor prompting, with high confidence. Mixed on whether routing saves money in any given repository, with medium confidence. Decomposition and integration have costs; the right decision comes from comparing accepted output at a fixed quality bar. Supplying a filename is useful context when known, not evidence of weak engineering.

Runtime testing improves the evidence available for review

The Greptile homepage identifies TREX as an agent that writes and runs PR tests in a sandbox. That supports the product identity and advertised function shown at 2:01. It does not independently establish better bug detection than competing reviewers. Anthropic’s guide independently recommends environmental feedback, including code execution, during agent work.

Verdict: Agree with combining code review and runtime checks, with high confidence. Uncertain about Greptile’s comparative effectiveness because no independent comparison was checked. Tests and recordings can demonstrate a reproduced case, but they cannot certify security, coverage, or correctness of untested paths. The sponsor relationship makes the distinction between advertised capability and measured benefit important.

Screen-level insights

Three extracted frames were opened and inspected directly. Other timestamps above rely on the transcript, not an assumption that their screens show a successful execution.

• 2:01, frames/002_000121.jpg: A Greptile page reads “AI code review that runs your code.” The top banner names TREX, and a sandbox illustration contains test-running boxes. This is a product page, not a live test run. Alongside the nearby sponsor narration, it confirms that the transcript’s “Trex by Grapple” refers to Greptile TREX. It does not demonstrate a bug being caught.
• 4:33, frames/005_000273.jpg: An X post displays a hand-drawn “Response Quality” chart labeled “GPT-6 Astra” and “Fable 5.1”. The blue curve has deep troughs; the orange curve is steadier. No numerical axis scale, sample size, or experiment is shown. The screen supports the interpretation that Theo is explaining his perceived consistency differences, not presenting benchmark results.
• 12:44, frames/015_000764.jpg: Twitch Stream Manager shows an ended poll asking “Do you watch the agent while it works?” The visible answers are Occasionally at 47%, Almost never at 26%, Usually at 17%, and Lol people do that? at 9%. Chat and an activity feed occupy the rest of the interface. This supplies audience context for the nearby discussion of autonomous runs and learning from failures. It is not a representative survey or evidence that less supervision improves outcomes.

The chart verifies labels on that screen only. No attempt was made to map every inconsistent caption spelling to a verified model release. Raw captions remain unchanged.

Recommended workflow and why it matters

Start with a reproducible bug and an explicit definition of done. Let the agent investigate and implement inside a restricted environment, require tests and inspectable evidence, then review the change before merging. Afterward, compare the result against a narrower request or mixed-model workflow. Expand the delegated steps only when completed work, defect rates, cost, and human attention justify it.

The strongest lesson is to reconsider which parts of a workflow the agent can handle, including investigation and verification. The weakest lesson is to infer someone else’s competence from their model choice or prompt style. Different repositories, budgets, and quality requirements can make different approaches rational.

Verification notes

After the unslop editorial pass, the synthesizing assistant completed four separate reviewer-role passes for source evidence, transcript/comment/frame fidelity, hallucinations and overclaims, and Actionable Insights. These were same-assistant checks, not independent agent reviews. The fidelity and overclaim passes confirmed that the edits preserved quotations, technical content, facts, numbers, confidence, and uncertainty.

The Actionable Insights check confirmed that all five experiments have concrete first steps, evaluation criteria, relevant links, and cautions. Evidence includes the complete extracted transcript, the available comment snapshot, three directly inspected frames, four named external pages, and a Python recomputation of the probability example. Corrections distinguish percentage points from relative percentages, clarify the exponential-growth argument, and identify the sponsor from the screen and creator comment. Raw captions were not changed.

The underlying creator logs, current model rankings, subscription-value claims, and sponsor performance have not been independently verified. Web search was unavailable; direct page fetches supplied the external research.