CLAUDE.md rule frequency audit: every rule fires 100%. What to measure instead.

The thesis: "frequency" is a category error here.

The phrase "rule frequency audit" sounds reasonable. It maps onto how engineers usually think about config: rules in a hot path get hit a lot, rules in a cold path get hit rarely, and you want to know which is which before you prune. The intuition works for log lines, feature flags, A/B variants, sometimes even for prompt skills. It does not work for CLAUDE.md.

The reason is in the analyzer:

That hardcode (line 264, sitting one comment under // Token math: assume CLAUDE.md fires every turn) is not a simplification. It is the API behavior. CLAUDE.md gets concatenated into the system prompt at session start and re-sent on every API call until the session ends. There is no per-rule gating, no line-level dispatch, no "skip section if irrelevant". Every non-blank line you wrote is paying its token cost on every turn.

So the literal "how often does each rule fire" question has a boring answer: 100%. Sorting your file by firing count gives you the same number 214 times. Cutting everything below the 10th percentile gives you an empty file.

What an honest audit looks like.

The metric people actually want when they ask for a frequency audit is effective frequency: how often does the agent actually act on this rule when it fires. That number varies wildly across rules. A specific named-tool preference like prefer pnpm over npm gets followed close to 100% of the time it's relevant. A vague aspirational line like always write clean code gets followed somewhere around 0% of the time it's relevant, because nothing about it tells the model when it's satisfied.

You cannot read effective frequency directly. Anthropic does not expose a "this rule was applied" signal in the response. What you can do is read the structural shape of each line and predict which quadrant it lands in:

The 7 shapes the analyzer flags, as effective-frequency predictors.

The Finding.kind union at analyzer.ts line 7 names them. Each one predicts a specific failure mode for effective frequency, not a quality opinion. The static frequency column reads "100%" for every row because that's the literal firing rate. What changes is the shape of the failure when the rule fires.

The audit, rephrased for the file you already wrote.

Two ways to read the request. One is the way the question is usually posed (count firings). The other is the way it can actually be answered (score shapes). Toggle:

One counterargument, with a real answer.

Pushback we get on this framing: "Fine, every line fires every turn at the prompt level. But the model still attends to some parts of the prompt more than others. Surely that's a kind of per-rule frequency." The answer is: yes, attention is non-uniform, but you still cannot measure it per-rule without instrumenting the model internals. What you can measure deterministically, from outside the model, is whether the line you wrote has any chance of getting attended to in the first place.

A 35-word line with three subclauses won't. A line with the word appropriatelyin it can't, because the model has no test for "appropriate" in the current task context. A line that duplicates a previous line costs tokens twice but adds no signal. These are the things the analyzer reads. The remaining variance (attention weight inside the rules that pass the shape check) is the part you cannot fix in CLAUDE.md; that lives in hooks, skills, and tool descriptions.

Where the analyzer lives, line by line.

• analyzer.ts:5 - Finding type, the 7 kinds.
• analyzer.ts:124 - VAGUE_TERMS list (appropriate, properly, carefully, well, as needed, where applicable, ...).
• analyzer.ts:130 - ASPIRATIONAL list (always, never, must, should always, in all cases, every time).
• analyzer.ts:150 - bloat threshold: more than 28 words on a single line.
• analyzer.ts:194 - cache_bust scan: only fires for lines with idx less than 20.
• analyzer.ts:211 - duplicate scan: same trimmed lowercased line, minimum 10 chars.
• analyzer.ts:230 - missing_why scan: looks for because / why / reason / past / got burned / incident / happened / caused in the next 4 lines.
• analyzer.ts:244- conflict scan: contradicting absolutes (the shipping check looks for both "never use comments" and "add comments").
• analyzer.ts:264 - the hardcode this entire page is about.

The whole analyzer is 322 lines, no network calls, no upload. The input goes nowhere; the Finding[] array lands in the same browser tab.

Related guides on this site.

• CLAUDE.md line firing audit - the same 7 checks, with the exact thresholds and the file:line where each implementation lives.
• Do CLAUDE.md rules fire on every turn? - the firing surfaces breakdown: CLAUDE.md (unconditional) vs skills / hooks / MCP descriptions (conditional).
• Prune dead CLAUDE.md rules - the action page once you have the audit output.
• Karpathy 12 rule scorecard - the file-level pass that runs alongside the per-line audit.