How similar are these pages, really
Templated architectures are where thin content hides. This fetches a set of real pages, removes the header, footer, and navigation that are identical by design, and measures what is left with the same fingerprinting technique used for deduplication at scale.
Near Duplicate Detector
Press compare to fetch a set of pages and measure how similar their content actually is.
Two measures run side by side. SimHash produces a 64 bit fingerprint per page from weighted token hashes, and similarity is the proportion of matching bits, which is what large scale deduplication actually uses because comparing fingerprints is cheap. Jaccard similarity over three word shingles is slower and more precise, and is used here to confirm the SimHash result rather than replace it. Boilerplate is stripped first: the header, footer, and navigation are identical on every page by design, and leaving them in would make everything look like a duplicate of everything.
Programmatic pages are where thin content hides
A service crossed with a location, a product crossed with a size, a category crossed with a filter. These are legitimate architectures that become a liability at exactly the point where the only thing changing between pages is a noun.
The threshold is qualitative
There is no published cutoff at which duplication becomes a penalty. What happens instead is quieter: one page gets picked as canonical and the others stop being served, and nobody notices because nothing broke.
Strip the template first
Any similarity measure that includes the shared header and footer will report every page on a site as a near duplicate of every other. The interesting number is main content against main content.
Fix it with specifics, not synonyms
Rewriting the same sentence three ways does not help. What separates two location pages is genuinely different content: different projects, different constraints, different local detail.