Why Google isn't indexing your page

You published the page. It is in the sitemap. Search Console still says it is not indexed.

The sitemap is not a ticket. It is a hint.

Indexing problems get easier when you stop treating “Google” as one step. A crawler has to discover the URL, fetch it, understand the rendered page, choose a canonical, decide whether the content belongs in the index, and then decide when to serve it for a query.

Debug the first stage that failed.

Start with the exact URL

Copy the URL from Search Console instead of typing what you think it is. Small differences matter:

• http versus https
• www versus the apex domain
• Trailing slash versus no slash
• Uppercase versus lowercase path
• Query parameters
• An old route that now redirects

Fetch that exact URL outside your browser:

curl -sS -D headers.txt -o page.html https://example.com/problem-page

Inspect the final status, redirects, response headers, and saved HTML. Your browser may hide a redirect, reuse a service-worker response, or show an authenticated version Google cannot access.

The final public URL should normally return 200. A 301 or 308 is fine for an old URL when it reaches the intended canonical in a short, stable chain. A 404, 410, 5xx, redirect loop, or login wall explains the problem before any meta tag does.

Also watch for soft 404s: a 200 page whose content says “not found,” contains almost nothing, or redirects users with client-side JavaScript. The status and the page need to tell the same story.

Check crawling and indexing separately

robots.txt and noindex do different jobs.

• A robots.txt disallow asks a compliant crawler not to fetch the URL.
• A noindex directive asks a crawler that fetched the URL not to place it in search results.

If you block the URL in robots.txt, Google may be unable to see a noindex instruction on the page. Google’s robots meta documentation calls out that interaction directly.

Check all possible sources of an indexing rule:

<meta name="robots" content="noindex" /> <meta name="googlebot" content="noindex" />

And the response header:

X-Robots-Tag: noindex

The header matters for HTML too, not only PDFs and images. CDN rules, staging middleware, and framework route configuration can add it after the template looks clean.

Fix the rule at its source. Do not add a second index tag and hope it outvotes noindex.

Follow the canonical trail

Open the raw HTML and find every canonical declaration:

<link rel="canonical" href="https://example.com/preferred-page" />

The canonical should be absolute, public, indexable, and appropriate for the content. Common failures include:

• Every page points to the homepage.
• Staging URLs survive the production build.
• Paginated or filtered pages point somewhere unrelated.
• HTTP headers and HTML specify different canonicals.
• The sitemap lists one URL while the page declares another.
• Internal links consistently use a noncanonical form.

A canonical is a signal, not an order. Google combines redirects, canonical annotations, sitemap inclusion, links, and content similarity when choosing a representative URL. Its canonicalization guide describes redirects and rel="canonical" as stronger signals than sitemap inclusion.

If Search Console says “Duplicate, Google chose different canonical,” compare the two pages. They may be more alike than your route names suggest.

Inspect what the crawler can render

View source, not only the Elements panel.

If the title, main copy, links, canonical, or structured data appears only after a large client-side application boots, rendering becomes another failure point. Google can render JavaScript, but a failed API call, blocked bundle, infinite loading state, or environment-specific exception can leave it with an empty shell.

Use Search Console’s URL Inspection rendered view. Compare it with:

1. The original HTML response
2. A logged-out browser session
3. A browser with JavaScript disabled when the route is intended to be server-rendered

Check the console and network for public visitors. A page that works only because your browser has an admin cookie is not publicly indexable.

For content routes, I want the primary text and links in the initial HTML whenever the architecture allows it. That is useful to people on slow devices too, not just crawlers.

Prove the page can be discovered internally

A sitemap can introduce a URL. Internal links tell search engines where the page belongs and how important it is relative to the rest of the site.

Find links to the page from rendered, indexable pages:

rg 'href="/problem-page"' src

Then check the deployed HTML. A source component does not help if a feature flag, pagination limit, or client-only menu keeps the link out of the public page.

Use descriptive anchor text. “Read more” repeated 20 times gives less context than a link naming the guide. Keep important content within a sensible path from navigation, a hub page, or a closely related article.

Orphan pages are easy to create when a CMS publishes a record but no collection page includes it.

Treat the sitemap as inventory

The sitemap should contain the canonical URLs you want indexed, return valid XML, and stay within the protocol limits. Each listed URL should resolve without a redirect when practical.

Do not include:

• noindex pages
• Redirecting legacy URLs
• Staging or preview hosts
• Search results and arbitrary parameter combinations
• Duplicate URL forms

Submitting a sitemap does not guarantee crawling or indexing. Google describes it as a hint in its sitemap documentation.

Use the sitemap report to find fetch and parsing failures. Use URL Inspection for the individual page. They answer different questions.

Ask whether the page earns a separate URL

Once the technical signals agree, look at the content without searching for another tag to change.

Is the page substantially useful on its own? Does it answer the promised question? Is it a near-copy of a category, location, or template variant? Would a visitor need to return to search immediately?

Thin does not mean a specific word count. A short reference can be the best result. The problem is a page whose unique value is a swapped city name, one generated paragraph, or a product filter already represented by a stronger canonical page.

Google’s people-first content guidance explicitly warns against writing to a preferred word count and mass-producing pages without additional value.

Combine duplicate pages when one stronger resource serves the reader better. Improve a thin page when it represents a real intent but does not finish the job.

Use URL Inspection as the final debugger

For the exact canonical URL:

1. Inspect the indexed version.
2. Note the crawl status, declared canonical, and Google-selected canonical.
3. Run a live test after deploying the fix.
4. Review the screenshot, rendered HTML, and loaded resources.
5. Request indexing only after the live test shows the intended result.

A request is not a guarantee or an instant refresh. Keep the page linked internally and in the sitemap, then monitor the result.

If many URLs fail in the same way, stop inspecting them one by one. Find the shared template, middleware rule, deployment, or CMS field and fix the class of problem.

The order I use

When a page is missing from search, I check in this order:

• Exact URL and final HTTP status
• robots.txt, meta robots, and X-Robots-Tag
• Redirect and canonical agreement
• Public rendered content and runtime errors
• Internal discovery and anchor context
• Sitemap consistency
• Duplicate or insufficient page value
• URL Inspection live result

SiteCMD’s live-site checks automate the repeatable status, redirect, canonical, robots, sitemap, metadata, and link checks. Connecting Search Console adds the field evidence about impressions and affected pages. The decision about whether a page deserves to exist still belongs to a person.

For a release-wide review, the pre-launch website checklist covers indexing alongside performance, accessibility, security, monitoring, and rollback.

Find the first broken stage, fix it, and inspect the same URL again. Indexing is much less mysterious when every signal tells one story.