In my test, Karpathy’s CLAUDE.md changed how Claude Code handled a vague request, and little else. On “Add discount support to the cart,” it asked first in 3 of 6 runs. Without the file, 0 of 6 asked. Bug fixes looked the same either way, and no fix run wrote a reproducing test.
Karpathy’s CLAUDE.md is a 65-line instruction file by Forrest Chang, derived from Andrej Karpathy’s observations on LLM coding mistakes. It has four principles: Think Before Coding, Simplicity First, Surgical Changes, and Goal-Driven Execution.
I had Claude Code run this test on my machine: 60 scored runs on October 20, 2026, with Claude Code CLI 2.1.296 and Sonnet 5.5. Then I kept one rewritten line from it in my work AGENTS.md for twelve days, and that follow-up is further down.
On this page
- Does Karpathy’s CLAUDE.md change what Claude Code does?
- The vague task: fewer guesses, less code
- Bug fixes and tests: Surgical Changes and Goal-Driven Execution
- Cost, speed, and the backfire check
- Who wrote it, and where the 41%-to-3% numbers come from
- How to install it in Claude Code, Cursor, or Codex
- Should you add Karpathy’s CLAUDE.md to your project?
- How this was tested, and its limits
- FAQ
Does Karpathy’s CLAUDE.md change what Claude Code does?
Partly. It changed the code on one task out of five: on a vague request, Claude Code stopped guessing silently. On the four concrete tasks, the file changed what the model said, not the code it wrote.
Each task ran 6 times with a 3-line project description as CLAUDE.md, and 6 times with the Karpathy file added unedited.
| Principle | What I checked | Without file | With file |
|---|---|---|---|
| Think Before Coding | Asked before editing (vague task) | 0/6 | 3/6 |
| Think Before Coding | Picked a reading silently | 6/6 | 0/6 |
| Simplicity First | Added throwing validation (vague task) | 6/6 | 0/6 |
| Simplicity First | Extra code on a small add | 0/6 | 0/6 |
| Surgical Changes | Bug fix stayed in the function | 6/6 | 6/6 |
| Surgical Changes | Mentioned unused code (small add) | 0/6 | 4/6 |
| Goal-Driven Execution | Wrote a reproducing test | 0/12 | 0/12 |
| Backfire check | Asked about a one-word typo | 0/6 | 0/6 |
I fixed the reading rule before the runs: a gap of 3 or more out of 6 counts as “changed,” 0 or 1 as “no change,” and 2 as “unclear.” I ran the four principles as one file, so a change on a task is the file’s effect, not one principle’s.
For comparison, concrete single rules held in 45 of 45 runs in my earlier CLAUDE.md rules test, which used the same harness.
The vague task: fewer guesses, less code
The file’s first principle reads: “Don’t assume. Don’t hide confusion. Surface tradeoffs.” On the discount request, 3 of 6 file runs asked before touching a file, and the other 3 stated their reading first. Without the file, all 6 runs built a full percent-plus-fixed-amount scheme, and none said the request was ambiguous.
One file run never used the word “assume,” so the scoring script called it a silent pick. A hand check of all 12 replies found it saying: “You didn’t say what kind of discount you wanted, so I supported percent and a fixed amount.” It counts as a stated reading.
This is the failure I care about. Earlier in 2026, I asked an agent in one line to add an “exclude canceled” option to a settlement lookup API at work. It treated only CANCELED status as canceled, so partial cancellations stayed in the totals, and its own tests passed. I caught it by comparing sample records before review. Reverting and redoing it with a written definition cost about 40 minutes.
Code size moved with the questions. Without the file, runs added a median of 23 lines of code (21 to 26), and all 6 added validation that throws. The 3 file runs that implemented added 5, 5 and 11 lines, with no throwing validation. Half the file runs wrote nothing because they asked, so I can’t separate Simplicity First from Think Before Coding here.
On the small add, a function that returns total quantity, both conditions wrote one 8-line function plus the export, 6 of 6.
Bug fixes and tests: Surgical Changes and Goal-Driven Execution
Fixing the loop in cartTotal stayed inside the function in 6 of 6 runs with or without the file. Nobody touched the comment typo or deleted the unused oldShippingFee.
The file’s dead-code line (“mention it – don’t delete it”) did show up in replies. With the file, the unused function got mentioned in 4 of 6 quantity runs (0 of 6 without) and 5 of 6 label-fix runs (1 of 6 without).
It didn’t stop scope creep on the vague task. All 3 file runs that implemented discounts also fixed the unrelated loop bug unasked, as all 6 runs without the file did. I hold human PRs to the same boundary: when a PR at work mixed a business-logic change with surrounding refactoring, I asked to keep only the needed change and move the refactor to a separate task.
Goal-Driven Execution changed nothing. The file says “Define success criteria. Loop until verified.” Across the two fix tasks, 0 of 12 file runs wrote a reproducing test, the same as 0 of 12 without it, and both conditions ran the existing tests. On legacy code, my bar for trusting an agent’s change is whether I can verify it, not how confident the model sounds, so a principle that never produced a test doesn’t help me.
Cost, speed, and the backfire check
The file calls its own approach a “bias toward caution over speed,” so I added a one-word comment typo as a backfire check. Both conditions fixed only the comment, in a median of 3 turns, and 0 of 6 file runs asked a question.
On the four concrete tasks, the file added about $0.002–0.004 per run at median API prices. The vague task got cheaper and faster, because half the runs stopped to ask: the median file run took 6.5 turns and 16.5 seconds, against 11.5 turns and 27.6 seconds. All 30 runs without the file cost $2.23, and all 30 with it cost $2.10, both CLI-reported API-price estimates.
That doesn’t match my own August. In a side repo, Think Before Coding stopped even small fixes with a question often enough that I removed it in the second week. My test runs were single-turn claude -p calls, not a working session.
On small edits, a question up front is mostly a nuisance to me. My line: one or two files, about ten changed lines, revertable from the diff in under a minute. Then I want three lines of assumptions and the change, no question. If a word like “amount” or “status” can mean two things, or the change reaches the DB, config, or several files, I want it to ask first.
Who wrote it, and where the 41%-to-3% numbers come from
Karpathy didn’t write the file. Forrest Chang did, and the README calls it “derived from Andrej Karpathy’s observations on LLM coding pitfalls,” linking Karpathy’s post on X as the source. The repo has since moved from forrestchang/andrej-karpathy-skills to multica-ai/andrej-karpathy-skills; the old URL redirects. As of November 3, 2026, it had about 228k GitHub stars, and its CLAUDE.md was still at commit 2c60614, the version I tested.
Korean posts repeat the mix-up, with many titles presenting it as Karpathy’s own file. In Naver results I went through and counted on October 20, 2026, about 13 of the top 25 blog posts for “카파시 claude.md” covered this file, and none measured it.
The “12 rules” version is a third party’s. Write-ups credit rules 5 to 12 to an X post by @mnilax, which claims mistakes drop from about 41% with no rules to about 11% with four and about 3% with twelve. It describes 30 codebases, 50 tasks, and 6 weeks. What I couldn’t find published: how tasks were picked, who judged a mistake, how many runs each condition got, and the data. I didn’t test the 12-rule version.
How to install it in Claude Code, Cursor, or Codex
The README (checked November 3, 2026) recommends the plugin. From inside Claude Code, run /plugin marketplace add forrestchang/andrej-karpathy-skills, then /plugin install andrej-karpathy-skills@karpathy-skills. That makes the guidelines available as a skill across all your projects.
The per-project route is a plain file. In a new project, curl -o CLAUDE.md https://raw.githubusercontent.com/forrestchang/andrej-karpathy-skills/main/CLAUDE.md downloads it. In a project that already has a CLAUDE.md, the README appends it with curl ... >> CLAUDE.md after a blank line. I tested this file route, appended below a 3-line project description, so my numbers describe that setup, not the plugin.
If you append it, watch the length. Claude Code’s memory docs (checked November 3, 2026) still say to “target under 200 lines per CLAUDE.md file,” and that “longer files consume more context and reduce adherence.” The same page says Claude treats CLAUDE.md “as context, not enforced configuration.”
For Cursor, the repo commits its own project rule, .cursor/rules/karpathy-guidelines.mdc, with setup notes in CURSOR.md. If a Cursor rule seems to do nothing, scope is the first thing to check, as I found when my Cursor rules weren’t applying.
For Codex, there’s no Codex file in the repo. Codex reads AGENTS.md, walking from the project root down to the current directory, and stops adding files at 32 KiB by default (project_doc_max_bytes), per OpenAI’s AGENTS.md guide checked November 3, 2026. So the text would go into AGENTS.md. I haven’t measured it with Codex or Cursor.
Claude Code can also read AGENTS.md (v2.1.277 or later), but by default only when there’s no CLAUDE.md in the working directory or above it. My work setup keeps the rules in AGENTS.md and has CLAUDE.md import it with @AGENTS.md.
Should you add Karpathy’s CLAUDE.md to your project?
I wouldn’t add the whole file. I’d take Surgical Changes, rewritten as something a command can check, and leave Think Before Coding and Goal-Driven Execution out. That comes from this test, two weeks with the file in a side repo, and twelve days with one line of it at work.
In late August 2026, I copied the whole file into a personal side repo as its CLAUDE.md, not as the plugin. It never went into a work repo. Surgical Changes made a big difference I could feel: unrequested cleanups and formatting changes clearly dropped. Later I removed the file and moved two lines about change scope into my existing CLAUDE.md. A grep across two personal repos now finds the same intent, but none of the original wording.
At work, an AGENTS.md of 80–110 lines holds the rules, under the 200-line target. My filter for it: if one command can’t tell whether a sentence was followed, it doesn’t go in.
Think Before Coding fails that filter. It asked in 3 of 6 runs and stated a reading in the other 3, and no single command tells those apart. Goal-Driven’s test-first line went 0 for 12, so it’s out. Surgical Changes passes once I rewrite it:
- Limit changed files to the files named in the request and their tests. If `git diff --name-only` shows any other path, revert it.
To be fair to it, the bug fix stayed surgical 6 of 6 with or without the file. I kept it because of August, not because of this test.
What the line caught in twelve days at work
I added that line to the work AGENTS.md around October 22, 2026, and planned to judge it by PR diff size on November 3. In those twelve days, the agent opened a dozen or so PRs, and three or four had out-of-scope paths in git diff --name-only. Most were lock files, build files, or import cleanups in a shared utility.
I reverted two by the rule. The rest were my fault: the request had left out a file the change really needed, so I fixed the request and reran it. The one I remember best was a one-field addition where a shared DTO got reordered top to bottom, and the diff grew to about 80 lines. After the revert it was under 10.
Diff size turned out to be the wrong measure. PR sizes varied too much for twelve days to say whether the rule or the task made the difference. What I did notice was fewer reviewer comments about files outside the change.
So I’m keeping the line, and I now count PRs where --name-only shows an out-of-scope path instead of counting lines. I also added an exception: lock and build files are allowed when the request is a dependency change. Before that, the rule reverted a legitimate dependency update once, and the agent did the same fix twice.
For asking first, I use a mode instead of a rule. In Plan Mode (Shift+Tab), I describe the requirements by voice and have the agent ask back about what’s missing. Outside Plan Mode, I define any slippery term in one line up front, and the agent writes its assumptions in three lines before implementing.
How this was tested, and its limits
It’s a small test: 60 single-turn runs, one model, and one six-file Node project, on October 20, 2026. Six runs per condition can show a large gap, but 6 of 6 only means the true rate is probably above about 61%.
The harness matches my earlier CLAUDE.md test. What changed:
src/cart.jsgot a comment typo (“Retruns”) and an unusedoldShippingFeefunction in a different style.- Prompts dropped the earlier “keep it small” hints, and checks ran per task.
- The Karpathy file was fetched at 04:11 UTC from repo commit
2c60614: 65 lines, 358 words.
The README says MIT, but the repo has no LICENSE file at its root, so I quote short lines and don’t reprint the file.
In -p mode, a question ends the run. Interactive sessions may ask more or less.
17 runs, all on the discount and label tasks, hit a permission denial for python3 or sed through Bash. They switched to the Edit tool, and a few couldn’t run a final npm test. Pre-allowing those commands is the fix, and I compared how each CLI mode handles that in stopping coding agents’ permission prompts.
FAQ
Is the plugin the same as the CLAUDE.md file?
The guidelines are the same, but the plugin installs them as a skill across all projects, while the file sits in one project’s CLAUDE.md. I only measured the file.
Can I use it with my existing CLAUDE.md?
Yes. The README’s append command adds it below your current file. Check the total stays under Claude Code’s 200-line target, and expect it to change replies more than code on concrete tasks.
Sources
- multica-ai/andrej-karpathy-skills, GitHub, checked October 20 and November 3, 2026: author, “derived from” description, about 228k stars on November 3, README link to Karpathy’s X post, install commands, Cursor rule, redirect from forrestchang.
- CLAUDE.md at commit 2c60614, GitHub, fetched October 20, 2026 (still the latest commit on November 3): the file I tested and the lines I quote.
- How Claude remembers your project, Claude Code docs, checked November 3, 2026: the 200-line target, “context, not enforced configuration,” and when Claude Code reads AGENTS.md.
- AGENTS.md guide, OpenAI Codex docs, checked November 3, 2026: AGENTS.md discovery and the 32 KiB default limit.
- Karpathy CLAUDE.md Rules + 8 More, Agentpedia Codes, May 11, 2026: the @mnilax 12-rule version and its 41%, 11% and 3% figures, as retold there.
Show 2 more sources
- My earlier CLAUDE.md rules test, loopnotes.dev: the 45-of-45 result and the shared test harness.
- Naver searches (October 20, 2026, query “카파시 claude.md”, top 25 blog results): the Korean post tally.
Related posts
- AI Agent Calling the Wrong Tool? I Cut It from 6/20 to 1/20
- RAG Retrieval Failure vs Generation Failure: Top-k First
- Claude Outage Fallback: Request Failed, Files Still Changed
- HTTP 402 Payment Required: x402 Agent Spending Limits
The test script, inputs and raw logs for this article are public: loopnotes-lab on GitHub.