Discovered or crawled, currently not indexed: how to fix both

"Discovered – currently not indexed" means Google knows the URL exists but has not crawled it yet. "Crawled – currently not indexed" means Google fetched the page and decided not to index it, at least for now. The first is usually a discovery or crawl priority problem. The second is usually a value problem: the page looks thin, duplicate or unimportant. Both are fixed the same way, in the same order: make the page eligible, make it easy to find, then make it worth indexing.

#What each status means

Both statuses appear in Google Search Console's Page indexing report, under "Why pages aren't indexed". Here is what Google's own documentation says:

Neither status is an error. Google never promises to index every URL. But if a page you care about sits in either bucket for weeks, something about the page or the site is telling Google it can wait.

#The difference, in practice

Discovered: Google hasn't looked

On a small site, the "overload" reason rarely means your server is actually struggling. More often Google found the URL through a sitemap only, sees few or no internal links to it, and gives it low crawl priority. New sites with many URLs at once (for example 200 templated pages an agent generated in one afternoon) hit this a lot. Slow or erroring servers make it worse.

Crawled: Google looked and passed

Google fetched the page. Common reasons it then holds back: the main content is thin or missing from the HTML (a client-rendered shell), it duplicates another page on your site or elsewhere, it is one of many near-identical template pages, or the site as a whole has not earned enough trust yet. Resubmitting does not help here; Google says so directly. The page has to change.

#Find the pattern before fixing pages

Don't start with one URL. Start with the list, because the cause is usually shared by a whole group of URLs.

1. Filter to the URLs you submitted

   In the Page indexing report, the dropdown above the chart defaults to All known pages. Switch it to All submitted pages (or one sitemap). That hides URLs Google found on its own, such as old slugs, ?sort= variants and tag archives, and leaves the pages you actually asked Google to index.

2. Export and group by path

   Open the status, then export the examples table. It holds at most 1,000 rows, so on bigger sites treat it as a sample. Group the URLs by their first path segment to see which template is affected:

   # Adjust the column if your export puts the URL elsewhere
   tail -n +2 export.csv | cut -d, -f1 | tr -d '"' \
     | awk -F/ '{print "/" $4}' | sort | uniq -c | sort -rn
   #  142 /integrations
   #   31 /blog
   #    4 /docs

   One template dominating the list (142 programmatic integration pages above) points to a site-level cause: too many similar pages, or a section with no internal links. A handful of scattered URLs points to page-level causes.

3. Decide which URLs you care about

   Parameter variants, paginated archives and old URLs that redirect can sit in this status forever without harm. Leave them, or stop linking to them. Spend your time on the pages you want to rank.

#A triage order that works

Work through these in order. There is no point improving content on a page Google is not allowed to index.

1. Eligibility: can this page be indexed at all?

   Check four things on the live URL:

   URL=https://your-site.com/guides/widgets
   
   # 1. Returns 200 directly (no redirect, no 404)
   curl -s -o /dev/null -w "%{http_code} %{redirect_url}\n" "$URL"
   
   # 2. No noindex in the header or the page
   curl -sI "$URL" | grep -i x-robots-tag
   curl -s "$URL" | grep -io '<meta name="robots"[^>]*>'
   
   # 3. Canonical points to itself
   curl -s "$URL" | grep -o '<link rel="canonical"[^>]*>'
   
   # 4. Not blocked in robots.txt
   curl -s https://your-site.com/robots.txt

   A canonical that points to another URL, or a stray noindex left over from staging, explains many "not indexed" pages on its own. Also check that the main text is in the raw HTML, not added by JavaScript later.

2. Discovery: can Google find it easily?

   Link the page from at least one or two related, already indexed pages with descriptive anchor text, and make sure it is reachable within a few clicks of the homepage. Google's guidance is plain: every page you care about should have a link from at least one other page on your site. Then confirm the URL is in your sitemap, the sitemap lists only 200, canonical, indexable URLs, and it is submitted in Search Console. Our internal linking guide covers how to do this on a small site. The links must be real <a href> elements without nofollow; a card with a click handler is not a link to Google.

   If you have server or CDN logs, check whether Googlebot has fetched the URL at all:

   grep "/guides/widgets" access.log | grep -i googlebot | tail -5
   
   # Anyone can fake the user agent. Confirm an IP really is Google:
   host 66.249.66.1              # should end in googlebot.com, google.com or googleusercontent.com
   host crawl-66-249-66-1.googlebot.com   # should resolve back to the same IP

3. Value: is the page worth indexing?

   Read the page as a searcher would. Does it answer something better than what already ranks? Is it a near copy of another page on your site? If you have many template pages that only swap a keyword, pick the best ones and make them genuinely different, merge the rest, or mark them noindex. A smaller set of strong pages tends to get indexed faster than a large set of weak ones.

#Using URL Inspection and "Request indexing"

Paste the URL into the search bar at the top of Search Console. The URL Inspection tool shows Google's indexed version: whether the URL is on Google, how it was discovered (sitemaps and referring pages), the last crawl, whether crawling and indexing are allowed, and the user-declared versus Google-selected canonical.

Click Test live URL to fetch the current version. Use View tested page to see the HTML Google received and, for a successful test, a screenshot. If your content is missing there, the problem is rendering, not quality.

When "Request indexing" helps:

• You fixed an eligibility problem (removed noindex, fixed the canonical, unblocked robots.txt).
• You published or substantially changed an important page and want it recrawled sooner.

When it does not help:

• The page is "Crawled – currently not indexed" and you have not changed it.
• You already requested it. Google says requesting a recrawl multiple times for the same URL won't get it crawled any faster.
• You have many URLs. There is a quota for individual requests; submit a sitemap instead.

#Rule out crawl capacity

Google's own description of "Discovered" mentions overloading the site, so check that before blaming content. Open Settings > Crawl stats (available for root-level properties only). It shows host status for robots.txt fetching, DNS resolution and server connectivity, the average response time, and crawl responses by type. Rising server errors or response times mean Google will crawl less. Google names 5xx and 429 responses and a slowing site as reasons it lowers crawl capacity. If URL Inspection reports "Hostload exceeded", Google is holding back because of your server.

"Crawl budget" in the strict sense is rarely your problem on a small site. Google's crawl budget guide is written for sites with over a million pages, or over 10,000 pages that change daily. Below that, the status almost always means low priority (weak links, many similar pages) rather than a hard limit.

#How long to wait

Google says crawling can take anywhere from a few days to a few weeks. For a new site or a newly linked page, give it two to four weeks after your fixes before you change course. Check progress in URL Inspection rather than with site: searches, which Google notes are not always exhaustive.

If a page is still not indexed after a month, and it is eligible, linked and in the sitemap, the answer is almost always value. Rework it or fold it into a stronger page.

#Decision table

#Brief your coding agent

Most of the triage is mechanical, so an agent can do it if you give it the list and the order. Paste the exported URLs and something like:

These URLs are "Discovered/Crawled – currently not indexed" in Search Console.
For each one, on production:
1. curl it: report status code, x-robots-tag, meta robots and the canonical.
2. Check the main text is in the server HTML (not added by client JS).
3. Find which route renders it and list every internal <a href> to it in the code.
4. Check it is in sitemap.xml.
Fix eligibility problems first. For pages with fewer than two internal links,
add links from related pages with descriptive anchors. Don't create new pages.
Group pages that look thin or near-duplicate and list them for me to decide on.

Keep the value decision (rewrite, merge or noindex) for yourself. An agent will happily "fix" thin pages by padding them, which doesn't make them more useful.

#What not to do

• Don't click "Request indexing" every day. It won't speed things up.
• Don't use third-party "instant indexing" services or the Indexing API for normal pages. Google limits that API to pages with job posting or livestream structured data.
• Don't block the page in robots.txt to "reset" it. Google then can't crawl it at all.
• Don't generate more pages to compensate. More thin pages make the problem worse.
• Don't rewrite URLs or slugs hoping for a fresh start. You lose the discovery you already had.

Serpkin can help with the first two steps. When Search Console is connected, it reads URL inspection for a few sitemap pages per scan, turns pages reported as not indexed into a mission with Google's reason attached, and its site scan flags the usual causes: sitemap URLs that redirect or are noindex, canonicals that point at redirects, errors or noindex pages, content that needs JavaScript, and orphan pages. Each mission names the source files and the checks that prove the fix. For the bigger picture on agent-built sites, read SEO for AI-built websites.