Lesson 2 of 12 · 20 minutes

Understand discovery, crawling and indexing

Explain why a published page may still be missing from search.

Publishing a page makes it available on your website. Discovery, crawling, indexing, and serving it in results are separate steps. A search engine may learn a URL through a link or sitemap, fetch it, interpret its contents, and decide whether and when to show it.

These stages help you diagnose a problem. “Our page does not rank” could mean nobody linked to it, the server refused access, an indexing instruction excluded it, another URL was selected as canonical, or the page does not compete well for the query. Each cause needs a different response.

Follow one important URL

Choose a public service page. Open it while signed out. Find the internal link that leads to it. If you have access to Google Search Console, inspect the exact URL and compare the indexed result with a live inspection. A successful live test does not mean the page has already been indexed.

A sitemap is a discovery aid, not a promise of indexing. Likewise, a search using site:example.com can provide clues but is not a complete inventory or a substitute for URL Inspection.

Apply it

  1. Record the exact preferred URL, including HTTPS.
  2. Check that an ordinary visitor can open it without signing in.
  3. Find at least one relevant internal link to the page.
  4. Inspect the URL in Search Console if you control the property.
  5. Record the observed status and the next diagnostic step, rather than repeatedly requesting indexing.

Check your reasoning

If a page is excluded by an intentional indexing rule, rewriting its heading will not resolve that exclusion. Diagnose the stage first; choose the fix second.

Record your work in the SEO workbook. Your notes and progress save only in this browser; keep a copy of important work.

Check your understanding

A live inspection succeeds. What does that prove?

Further reading: Official guidance ↗