This workspace has grown a second copy of something it already had more than once: two address guards where the newer one was missing thirteen CIDR blocks, two bounded replay caches each declaring REPLAY_CAPACITY = 100_000, four ways to read a record from another PDS. Each was found by a person reading code, weeks later. This finds the structural ones the day they are written.

cargo run --release --manifest-path tools/xray-clones/Cargo.toml -- --root .

It exits 1 when it reports a group, 0 when it reports none, and 2 on an error.

What a fingerprint is #

Every function, method and impl block above the size floor is parsed with syn and walked with its visit API, which emits a token per node. The tokens keep the shape and drop the names:

Kept	Dropped
Control flow: `if`, `match`, `for`, `while`, `loop`, `?`, `.await`	Every identifier — function, variable, field, type and method names alike
Call arity, method-call arity, tuple and array length	Literal values, so `10.0.0.0/8` and `192.168.0.0/16` are one token
Binary and unary operators, including which one	Lifetimes
Match arms, patterns and their nesting	Attributes, which is also how doc comments and `#[cfg]` leave
The shape of types — reference, slice, tuple, `dyn`, `impl Trait`	Path segments and their number, so an import and a full spelling agree
Nesting, through an explicit close token after every node	Parentheses and visibility

A macro body is not Rust until something expands it. Where the body parses as a comma-separated expression list — vec!, format!, assert_eq! and most others — those expressions are normalised like any other. Where it does not, only the fact of a macro call survives.

The fingerprint is the first 16 hex digits of a SHA-256 over that token stream. It is what allow.toml names, so it is built from fixed specifications — SHA-256 and xxHash64 — rather than from the standard library's hasher, which is free to change between compiler releases.

What "similar" means #

The measure is Jaccard similarity over the set of 5-grams of the token stream: shared k-grams divided by all k-grams either one has. Identical shapes score 1.0. --shingle sets k; --threshold sets the bar, default 0.80.

Candidate pairs come from banded MinHash over 128 permutations, so that one item is compared against a few likely neighbours rather than against all 1,273. The banding is chosen to catch a pair sitting exactly on the threshold 98% of the time, and every candidate it proposes is then scored exactly. Pairs are joined into groups transitively, so a group can hold two members that are not themselves above the threshold; the reported minimum similarity says when that happened.

What this will catch #

The same function copied and renamed, wholesale or with edits.
Two implementations of one idea written to the same skeleton — a second bounded cache, a second poll loop, a second retry wrapper.
A family of route handlers that repeat the same ceremony around one differing call.

What this will not catch #

Two implementations of one policy written differently. A chain of is_loopback() || is_private() || … and a table of CIDR strings with a lookup encode the same rule and share almost no shape. On this workspace's own history the two address guards scored 0.14. This is the largest gap, and it is inherent to a structural measure.
Code that duplicates a dependency. A hand-rolled JWT parser beside the jacquard_common::service_auth that does the same job scored 0.25. Pass --scan <dir> to include a dependency's sources; it will still only find the ones that look alike.
Duplication inside a macro body that does not parse as expressions.
Duplication that a generic or a trait already spans, since the copies then do not exist in source.
A pair below the size floor. The floor exists because short functions all look alike, but it is also why two 16-line replay caches went unreported.

Suppression #

Repetition that the language requires is not a finding, so these never reach the report:

#[test] and #[bench] functions, which repeat a setup shape on purpose. --include-tests brings them back, and it is worth running: the two address guards were invisible at 0.14, but their tests scored 0.82, which is a pointer to the same bug.
Bodies in an impl Display, Debug, From or TryFrom block. The trait fixes the shape.
Anything under #[automatically_derived], and any file whose first lines say @generated or DO NOT EDIT.
Getters and one-line delegations.
An impl block reported beside a method inside it — they overlap in source, so the enclosing one is dropped.

The allowlist #

A reviewed duplicate is silenced by fingerprint in tools/xray-clones/allow.toml, never by lowering the threshold: the threshold is global, and dropping it to hide one pair hides every pair like it.

[[allow]]
fingerprint = "a71039fe0c16115a"
reason = "Two poll loops that must not share a task, by design."

reason is required. An entry without one is a parse error rather than a silent allow. A group is silenced when any of its members' fingerprints is listed, and every member's fingerprint is printed beside it for copying.

Flags #

Flag
`--root <DIR>`	Repository root; `crates//src` and `crates//tests` under it are scanned
`--scan <DIR>`	An extra directory, repeatable — a dependency's sources, say
`--min-lines`, `--min-stmts`	The size floor, default 25 lines or 12 statements. An item clearing either is considered
`--threshold`, `--shingle`	The bar and the k-gram length
`--same-crate-only`, `--cross-crate-only`	Where a group's members live
`--include-tests`	Report `#[test]` functions too
`--no-impl-blocks`	Fingerprint only the functions, not whole `impl` blocks
`--allow <FILE>`, `--no-allow`	The reviewed-duplicate list
`--focus <SUBSTRING>`	Print the nearest neighbours of matching items with their similarity, whatever the threshold. Answers "is this function like anything else?" rather than surveying the tree
`--top <N>`, `--json`	How much to print, and in what form. `--top 0` prints every group

Measured against four duplications this workspace really grew #

Each of these was found by a person reading code, weeks after it landed. Run over the commit before each was fixed, the tool finds one of the four.

	Best score	Found?
Two SSRF address guards, the newer one missing thirteen CIDR blocks	0.14 between the guards, 0.82 between their tests	Yes, with `--include-tests`, as the top group
Two bounded replay caches, each declaring `REPLAY_CAPACITY = 100_000`	0.51	No. `--focus` ranks the true counterpart first of 71, but the score is below any threshold that keeps a report readable, and both sit under the size floor
Four ways to read a record from another PDS, one skipping the bounded body	0.42	No. The four are written against four different clients
A hand-rolled service-auth JWT parser beside `jacquard_common::service_auth`	0.25	No. The library's version is a different shape, and lives outside the workspace

The pattern is consistent: this finds a shape that was copied, and does not find a rule that was reimplemented. Two of the three misses are visible to --focus as the item's nearest neighbour, which is the way to use it on a function you suspect.

Of the ten largest groups in the tree today, six were judged real duplication worth factoring and four were judged legitimately similar, so expect roughly two in five reports to be something you decide to keep.