Maximum Diff Size Release Analysis Guide

A maximum diff size release analysis gate stops automated analysis when a source change exceeds its configured review budget. It does not classify a large change as defective. It says the tool cannot prove complete risk mapping, test selection, or changed-line coverage at that size, then asks for a split or a human review.

The AI Release Guardian skill supplies the initial max_diff_lines configuration and keeps its result as a recommendation. Pair it with the release readiness scorecard, use the risk-based testing guide to classify code risk, and browse QASkills for test-impact and coverage guidance.

What Is a Release Review Diff Budget?

A review budget states how much changed source a tool can check with its promised method. The skill uses 2,000 source lines as a starting value, not a rule for all teams. Each team should commit its limit, state what counts, and change that rule through normal review.

The gate protects claim quality. Without it, an agent may inspect the first files, miss a deleted assertion, and still return a polished GO. A clear stop is more accurate than presenting partial inspection as complete release evidence.

Diff size is not risk by itself. A twelve-line auth change can pose more harm than a built client with several thousand lines. The guardian must still rank money, auth, data shape, public contracts, business rules, config, screens, tests, and docs by what changed.

Decision question	Diff-size gate answers it?	Correct evidence source
Can the declared review method inspect the whole change?	Yes	Counted source diff and configured limit
Is the change safe to release?	No	Gate results, risk map, coverage, and human decision
Is a small change low risk?	No	Changed behavior and blast radius
Are generated files harmless?	No	Generator source, regeneration check, and output review
Should a large change be rejected permanently?	No	Split plan or explicit review process

A useful maximum diff size release analysis rule keeps review scope apart from code risk. Crossing the line yields NO-GO because a full review is missing, not because line count predicts bugs. The next step is clear: split the change, narrow the review unit, or use a written exception owned by a person.

The limit also makes the gate easy to test. Run it on fixed diffs just below and above the line, with renames, deleted files, binary files, and built output. A rule that cannot explain its count will cause disputes late in a release.

How Should Git Diff Numstat Count Changes?

Count from fixed base and head commits, never from branch tips read at different times. Use the same endpoints for the risk map and the line budget. Store both full SHAs with the totals so another reviewer can repeat the count.

The official Git diff documentation says that A...B compares B with the merge base of A and B. It defines --numstat as a tool-friendly list of added lines, deleted lines, and paths. Choose two-dot or three-dot use on purpose, then keep it tied to the pull request under review.

For a normal pull request, the repo skill uses origin/main...HEAD to show what the branch changed since it split. Release branches or direct commit ranges may need other endpoints. Do not swap in a worktree diff when the CI verdict covers committed code.

Follow this counting procedure:

1. Resolve the configured base reference and judged head to full commit SHAs.
2. Run git diff --numstat with explicit endpoints and stable rename behavior.
3. Classify every path as source, generated, vendored, lockfile, test, documentation, or binary.
4. Sum additions and deletions for source paths, while reporting every separate category.
5. Compare the source total with gates.process.max_diff_lines and store calculation evidence.

The Node script below uses NUL-separated output and turns off rename checks so odd file names stay safe to parse. It counts a rename as deleted plus added lines. That choice is strict, simple, and easy to explain.

import { execFileSync } from 'node:child_process';

const [baseSha, headSha] = process.argv.slice(2);
if (!baseSha || !headSha) {
  throw new Error('Usage: count-diff-lines <base-sha> <head-sha>');
}

const output = execFileSync(
  'git',
  ['diff', '--numstat', '-z', '--no-renames', baseSha + '...' + headSha, '--'],
  { encoding: 'utf8' },
);

const records = output.split('\0').filter(Boolean);
let changedLines = 0;
const binaryPaths: string[] = [];

for (const record of records) {
  const [added, deleted, ...pathParts] = record.split('\t');
  const path = pathParts.join('\t');
  if (!path) throw new Error('Unexpected numstat record');
  if (added === '-' || deleted === '-') {
    binaryPaths.push(path);
    continue;
  }
  changedLines += Number(added) + Number(deleted);
}

console.log(JSON.stringify({ baseSha, headSha, changedLines, binaryPaths }));

This basic script counts each text path. The next step adds repo-owned path rules instead of hiding guesses in the command. Keep those rules in source control because one new exclusion can move the same diff across the limit.

Do not use patch byte size as a hidden stand-in for changed source lines. Binary patches, long built lines, and format work can skew bytes in different ways. If your team picks another measure, name it in the config so readers know what was counted.

When Are Generated File Exclusions Safe?

Generated files, lockfiles, and vendored code can dominate the diff without needing the same semantic review as hand-written source. The skill says to list them separately and explain each exclusion. An artifact left out of the source budget must still appear in the release report.

A built client still brings risk. Review its source input, build tool version, command, and clean rerun check. A lockfile can change packages or hash data. Vendored code may change license, security, or build traits even when local staff did not write each line.

Use repository-specific path rules such as these:

Category	Example path	Count toward source budget?	Required evidence
Handwritten source	src/orders/service.ts	Yes	Hunk review, risk map, selected tests
Test code	tests/orders/refund.test.ts	Yes, unless policy has a separate test-code limit	Assertions added, removed, or weakened
Generated output	src/api/generated/client.ts	Separate	Source schema, generator pin, clean regeneration
Lockfile	pnpm-lock.yaml	Separate	Manifest delta and dependency review
Vendored code	vendor/parser.c	Separate	Upstream revision, integrity, applicable scans
Documentation	docs/refunds.md	Separate	Executable examples and policy effects checked
Binary	assets/model.bin	Separate	Digest, producer, scan, and intended use

Path classification should follow deterministic rules. Prefer exact directories, generated-file headers, and repository configuration over an agent guess based on filename style. When one artifact matches several categories, use the more demanding category or report the conflict.

Never exclude migrations from the source budget. The skill treats data-shape changes as high risk and asks how rollback works. A short migration can affect each write path, while a nearby ORM snapshot may be built output, so keep those two groups clear.

Never exclude deleted tests. The guardian treats removed or weak checks as real findings because a deletion can remove coverage for code that did not change. Count those lines and state which contract the old test proved.

The guide on schema authority for test data helps when one schema change touches migrations, ORM models, OpenAPI, and TypeScript. Its constraint field map procedure can split those layers into smaller review units with clear clashes.

How Does Quality Gate Configuration Store the Limit?

Place the limit in release-gates.yaml so the rule gets reviewed with code. The repo guide defines gates.process.max_diff_lines and fails the gate when source lines exceed the honest review budget. Store the config commit or digest in the final report.

gates:
  tests:
    required_suites: [unit, integration, e2e-smoke]
    max_new_skips: 0
  coverage:
    changed_line_blockers: 0
  static:
    lint_errors: 0
    type_errors: 0
    new_security_findings: 0
  process:
    risk_map_reviewed: true
    max_diff_lines: 2000

Do not let the agent raise this number to pass its own review. A limit change alters team rules and needs human approval before the release at hand. If not, a large change can edit both the app and the rule meant to stop it.

The gate result should show measured source lines, the configured maximum, both SHAs, category totals, classification revision, and excluded paths. A bare "diff too large" message gives too little evidence for reproducible counting or exception review.

{
  "gate": "process.max_diff_lines",
  "status": "fail",
  "base_sha": "1a7b6d9e51f4d0f0f1967591ebad993a44ee1234",
  "head_sha": "8c22a70e90e83d8fe098a58fc362f8e816dea987",
  "source_changed_lines": 2471,
  "maximum": 2000,
  "separate_totals": {
    "generated": 6310,
    "lockfiles": 184,
    "documentation": 96,
    "binary_files": 1
  },
  "evidence": "artifacts/diff-budget.json"
}

The maximum diff size release analysis row belongs beside test, coverage, lint, data, and review rows. It must not erase those results. A split does not make a failed test or an unchecked auth branch go away.

For workflow steps, use the GitHub Actions testing pipeline guide. Run the budget job before slow broad suites when quick feedback helps. Still publish its row with the exact judged SHA, as shown in binding release evidence to HEAD.

How Does a Large Pull Request Review Stay Honest?

A split helps only when each new change is clear and testable on its own. Dividing files at random can leave broken states or spread one behavior across many reviews. Split by code flow, contract edge, feature flag, migration phase, or built-file life cycle.

A safe data change often separates an additive migration, a backward-compatible read path, new writes, backfill, and cleanup. Each part should remain compatible with the application versions deployed alongside it. The release report must still explain the full order and rollback needs.

A built API client update can split the OpenAPI change, tool config, built output, and client use. The output commit may stay large, but its proof becomes a clean rerun match plus focused client tests. That is more honest than claiming a person read the meaning of each built line.

Use this sequence when the gate fails:

1. Group changed files by user-facing behavior and dependency order.
2. Identify commits that can merge without leaving main broken or weakening required checks.
3. Move mechanical formatting and generated output away from behavioral source changes.
4. Preserve tests with the behavior they prove, including negative and rollback cases.
5. Run a fresh guardian report for each judged head, then review cumulative rollout risk.

Avoid Git tricks that make the pull request look small while the deploy still brings the full unchecked change. The gate covers what reaches the release base, not how neat the commits look. Compare the real base and head set by team rules.

The test impact analysis guide helps plan splits because each unit should map to tests for its changed code. If a planned split has no focused proof, it may not be a sound review unit on its own.

How Should an AI Release Guardian Handle Exceptions?

Some changes cannot fit the normal budget, such as framework moves, built snapshots, or a one-time repo cleanup. That does not permit a quiet gate bypass. It needs a named exception path with a different review plan.

The exception should state scope, owner, reason, other proof, end point, and affected release. A person accepts it before the verdict can become GO WITH WAIVERS. The repo report needs a real owner and an accepted waiver, so an agent cannot approve its own draft.

Other proof may include code-owner review, migration drills, a separate security review, clean build checks, staged rollout, or several domain reviewers. The right checks depend on the change. More full-suite tests do not replace an incomplete code review, since each check answers a different question.

OWASP's CI/CD Security Cheat Sheet treats pipelines, repos, build tools, and deploy steps as parts of security. This supports keeping exception rights outside the guardian. A tool that can raise its limit, accept its waiver, and deploy puts too much control in one actor.

The human control boundary guide defines that split in detail. The guardian may count lines, group risk, suggest a split, and draft exception notes. It must never approve the exception, merge the pull request, tag a release, or start a live deploy.

Keep the first failed gate row for each exception. Replacing it with "pass by exception" hides that the normal review limit was crossed. A clear report shows both the failed count and the accepted, time-bound waiver as separate facts.

Which Failure Modes Should the Gate Test?

The budget code needs tests because line counts have edge cases. Fixed diffs should cover spaces and tabs in paths, binary files, full deletes, missing final newlines, renames, submodules, and path-rule clashes. Pin the expected totals for known commit pairs.

Test team rules apart from parser code. A source total at the limit passes when the rule says "exceeds," while an additional line fails. Missing config should follow a stated default, though the skill prefers a proposed starting value over a hidden limit.

Required checks add another rule. GitHub says required checks must pass against the latest commit SHA. Set up the diff check so a path filter does not leave it pending on the large changes that need it most.

Test these outcomes in CI:

Scenario	Expected gate result	Required report detail
1,999 source lines under a 2,000 limit	Pass	Endpoints and category totals
Exactly 2,000 source lines	Pass	Boundary rule stated
2,001 source lines	Fail	Split recommendation
300 source plus 8,000 generated lines	Pass budget, generated review still required	Generator evidence and separate total
Binary-only model update	Source count alone cannot pass release	Digest, producer, and applicable scan
Missing base commit	Error and NO-GO	Fetch or resolution failure
Classification conflict	Error or demanding category	Conflicting rules named

A test must also prove that the report cannot claim GO when this required gate fails. The schema says gate rows and waivers drive the verdict, so CI should recompute the verdict from those rows instead of trusting free text.

How Does Risk-Based Release Review Treat Large Diffs?

A maximum diff size release analysis rule needs one clear release unit. In a monorepo, that unit may be the full pull request because shared packages can affect many apps. Per-service totals can help owners, but they must not hide one large cross-cutting diff.

Count the full base-to-head diff first, then group paths by package or service. If a shared library changes, trace its users before giving each folder its own budget. The risk lies in runtime behavior affected by the change, not only in the edited folder.

Monorepo pattern	Useful measurement	Unsafe shortcut
Independent services	Total plus per-service source lines	Passing each service while ignoring coordination files
Shared package change	Shared lines plus affected consumers	Counting only the package directory
Root build configuration	Whole-repository review unit	Assigning it to an arbitrary service
Generated clients in many packages	Inputs plus generated totals by consumer	Treating every generated directory as invisible
Database schema shared by services	Migration and all contract adaptations	Counting each adapter without rollout sequence

Stacked pull requests also need fixed endpoints. A child diff may be small against its parent but large against the release base. Decide whether the guardian checks the child for review, the full stack for release, or both, then store each result.

For final release work, compare the real release base with the head that will merge or deploy. If not, several small stacks can join into one large diff that no gate reviewed. The maximum diff size release analysis gate should fail that full unit when it crosses the committed limit.

Code ownership can guide a sound split. Put a shared contract change in one reviewed unit, update each user in order, and preserve compatibility tests for both old and new contracts. Do not divide one unsafe migration just to place each folder below a number.

When several repos form one release, each repo can run its own gate. A release lead should also record all repo versions and shared risks. A local line count cannot prove that the full rollout is clear, so use one report that names each repo SHA and gate result.

How Do You Separate Mechanical Churn From Code Risk?

Format work, file moves, built snapshots, and package data can make huge patches. A maximum diff size release analysis report should name that churn without calling it safe. The reviewer needs proof that the same command can reproduce it and that code edits are not hidden inside.

For format-only changes, pin the tool version and command, run it on the base tree in a clean setup, and compare the output. Review any remaining diff for behavioral changes. If a clean rerun does not match, return those paths to normal line review.

File moves need a clear rename rule. Git can find similar files, but the result depends on command flags and limits. The official Git docs list rename and copy flags, so record the exact command. A move with edits should show its hunks instead of getting one broad mechanical label.

Built output needs a similar proof chain. Use these steps so reviewers can repeat the result:

1. Identify the authoritative schema, template, or source file changed by humans.
2. Record the generator package, version, configuration, and exact command.
3. Regenerate from a clean checkout with network and dependency inputs controlled by repository policy.
4. Compare regenerated output byte-for-byte with the proposed generated paths.
5. Review consumer behavior, compatibility, security findings, and tests separately.

Churn category	Evidence that supports separate counting	Remaining review obligation
Formatter output	Pinned clean rerun matches	Confirm command scope and source inputs
Pure file move	Recorded rename method and unchanged blob	Review import, ownership, and build effects
Generated client	Pinned generator reproduces output	Review API input and consumer compatibility
Lockfile update	Manifest intent and package-manager resolution	Review dependency and scanner changes
Snapshot update	Test command reproduces snapshot	Review whether new expected behavior is correct

Snapshot churn needs extra care. A passing update proves the checked-in snapshot matches current output, not that the output is right. Ask a person to review changed behavior, above all for access, money, public contracts, and error paths.

Package lockfiles also stay in the release proof. Count them on their own when team rules say so, but review each added, removed, or changed dependency, and scan the resulting installed dependency set with the repository's normal tool. OWASP's CI/CD guide treats packages and pipeline parts as delivery risk, not inert text.

The maximum diff size release analysis gate should show both the raw text total and the rule-based source total. Raw size shows all churn, while the grouped total explains the review method. Keeping both stops exclusions from fading out of view over time.

Review path rules from time to time with real failed and waived reports. If one group often needs exceptions, improve its proof rules instead of raising the main limit without review. A human must approve each rule change, and it should not grade the release that proposes it.

How Do You Apply the Gate Each Day?

Start with a measured trial if the team needs to learn its usual diff sizes. Keep the gate visible during the trial, then commit a limit with owners and an exception path. Do not tune it just to make each recent pull request pass.

Review the rule when repo shape or build tool use changes. A monorepo may need service budgets plus one total check, while a schema repo may group built clients in another way. Keep one clear result for the release unit under review.

The maximum diff size release analysis gate works best as an early truth check. It asks whether the guardian can back its later claims. Tests, coverage, migrations, scans, and human review then show what those claims mean, so one line limit never becomes a false quality score.

Apply the AI Release Guardian skill to one normal pull request, record each group total, and commit gates.process.max_diff_lines with its owner. If the next change crosses the limit, split it or record a named human exception before asking for a release choice.

Frequently Asked Questions

Is 2,000 changed lines the correct limit for every repository?

No. It is the starting value in the gate guide, not a proven constant for each team. Pick a limit that fits your review method, repo shape, and human process. Keep it in source control, test both sides of the line, and require human review before any change.

Should generated files count toward the same maximum?

The skill says to report them on their own because review should focus on source inputs and a clean rerun. They are never hidden. Record built totals, source schema, tool version, rerun match, and client tests so one budget exclusion does not remove them from release review.

Does a large diff automatically mean NO-GO?

It means NO-GO under a required review gate because the promised proof is not complete. It does not mean the code has a bug. A human-owned exception with other review proof may support GO WITH WAIVERS, or the team can split the release into testable units.

Should test files count as source lines?

Count them unless committed rules set a separate test budget, and always inspect deleted or weak checks. Tests shape trust and can hide risk when removed. Show app and test totals on their own if useful, but never drop test lines from proof just to pass the limit.

Can full-suite success compensate for an oversized diff?

No. A passing suite proves that known tests passed for one commit. It does not prove that the guardian mapped each changed behavior, found missing cases, or saw a changed contract. Test results and review scope are separate gates, so one cannot replace the other.

How do renames affect the line count?

Choose one fixed rule and write it down. The sample turns off rename checks, so a rename counts as deleted plus added lines, which is strict. Another rule may use Git's rename score. Pin your choice in tests and use the same endpoints and flags in local and CI runs.

Who may approve an exception to the limit?

A named person allowed by repository rules, never the guardian that calculated the count. Keep the failed gate, waiver owner, acceptance, other evidence, scope, and endpoint in the report. The release owner then makes the final choice with that full record in view.