Tutorial · Content

How to find duplicate and near-duplicate content

Find pages with the same or almost the same text, duplicate titles and H1s, and decide what gets a canonical and what gets merged.

2 min readUpdated 21 October 2026

When several pages have the same or almost the same text, Google decides for itself which one to show, and it often doesn't pick the one you would. The most common causes: filters and parameters in a shop, versions with and without a trailing slash, copied product descriptions, city pages where only the city name changes.

Exact duplicates

  1. Crawl the site in Whole site mode.
  2. Open the Content tab, Exact duplicates filter. The crawler compares the text of pages without the menu and footer, so two pages with the same menu don't count as duplicates just because of it.
  3. Check titles and H1s: the Page titles tab, Duplicate filter, then the same in the Meta descriptions and H1 tabs.

Near duplicates

  1. Turn on the near-duplicate check in the crawl settings and set the similarity threshold as a percentage.
  2. Set the part of the page that counts if the site has an unusual layout: enter the element, class or id of the main content, e.g. main or .content.
  3. After the crawl, look at the columns with the number of near duplicates and the similarity percentage. Clicking a URL shows which pages it is similar to.
  4. With a licence and meaning turned on, the crawler also finds pages that are similar in meaning, even when they are written in different words.

What to do with a duplicate

CaseSolution
The same page at two addresses (parameters, slash)A 301 to one address, or a canonical if both have to exist.
Shop filtersCanonical to the main category; filters with no search value kept out of the index.
Two pages on the same topicMerge them into one better page and redirect the weaker one.
City pagesGive each one real local content, or remove them.

After the fix, check the Canonicals tab: the Canonical points to non-indexable URL and Canonical URL not linked filters catch the most common mistakes.

Next