Duplicate content is any content that appears on more than one URL, either within a single site (internal duplication) or across multiple sites (external duplication via syndication or scraping). Google’s documentation states that duplicate content itself isn’t penalized — but the practical effect of duplication is that Google must choose which version to index and rank, often splitting or diluting signals that would be stronger consolidated on one URL.
For site owners, the concern is signal dilution: links and authority that could strengthen a single canonical URL are spread across multiple duplicates instead.
Types of Internal Duplicate Content
URL parameter variations: The same content served at multiple URLs due to tracking parameters, session IDs, sort orders, or filter parameters. /products/?sort=price and /products/?sort=name may serve identical content with a different URL each time.
Protocol and subdomain variations: http://example.com vs https://example.com, or www.example.com vs example.com. These should serve the same content via canonical or redirect but sometimes don’t.
Trailing slash variations: /about/ and /about — the same page accessible with or without a trailing slash. If both return 200 responses without a canonical, these are duplicates.
Pagination without canonical: Paginated series (/blog/page/1/, /blog/page/2/) where the introductory content on each page creates partial duplicate.
Print-friendly and mobile versions: Legacy sites with separate /print/ or /m/ URLs serving similar content to the main URL.
Category and tag archive pages: Many CMS platforms generate archive pages for every category and tag. A post in 3 categories and 5 tags may be accessible from 8 different archive URLs.
Syndicated content: Your own content that you’ve republished on other platforms, or that other platforms have syndicated without canonical attribution.
The Canonical Tag Solution
The <link rel="canonical"> tag is the primary tool for consolidating duplicate content. It declares that among multiple URLs serving similar content, one is the authoritative “canonical” version.
<link rel="canonical" href="https://example.com/products/" />
On all parameter variants, sorted views, and filter combinations that duplicate the canonical URL, point canonical to the clean URL. Google will typically respect the canonical by:
- Indexing the canonical URL rather than the variants
- Consolidating any links pointing to variant URLs into the canonical
Important: canonical is a suggestion, not a directive. Google may ignore it if it believes the canonical hint is incorrect (e.g., if you canonical a mobile-variant page to a desktop URL when the content is actually different). For cases where you need a hard rule, use redirect instead.
When to Use Redirect vs. Canonical
301 redirect: Use when a URL should never be accessed directly — the old URL serves no purpose other than to reach the new URL. The best fix for http:// → https://, www → non-www consolidation, and print/mobile variants that no longer need to exist.
Canonical: Use when the URL has a user-facing purpose (e.g., a sorted view of a product list is useful for users) but you want Google to index only the canonical version. The alternative URL continues to function and serve users.
The distinction: redirects eliminate the duplicate URL entirely. Canonical tags keep both URLs functional while consolidating search signals.
Noindex for Thin and Near-Duplicate Pages
For pages that are generated by CMS but serve no meaningful unique content — tag archive pages with just a list of post titles, author pages with no description, pagination deep pages — <meta name="robots" content="noindex, follow"> removes them from the index without redirecting.
Noindex is appropriate when the page has user-facing value (users arrive there via navigation) but no search value. The page continues to function; it’s just invisible to search.
Consolidation: Merging Duplicate URLs
When two URLs have accumulated external links and rankings independently, a redirect from the weaker URL to the stronger one is the cleanest consolidation. This combines the link equity of both into one URL rather than trying to manage two competing pages.
Common consolidation scenarios:
- Two blog posts on the same topic published at different times
- A main guide and a “what is X” article that overlaps substantially
- A migrated URL where the old URL still receives some links
Before consolidating, check that the content is substantively similar enough that a redirect would make sense to users following it.
Identifying Duplicate Content
Screaming Frog or similar crawlers: Will identify duplicate page titles, meta descriptions, and near-identical body content. Configure to check for duplicate <h1> tags and meta descriptions as a proxy for duplicate content.
GSC Coverage report: Duplicate without user-selected canonical and Duplicate, Google chose different canonical than user are both GSC signals indicating duplication problems.
Site: operator searches: A quick check: site:example.com "exact phrase from article" will surface multiple indexed pages containing the same text if duplication exists.
Canonical audit: Crawl your site and check that every page either has no canonical (self-canonical implied) or has a canonical pointing to a live, non-redirected URL.