Google’s Search Console offers tools to identify crawling and indexing problems.
One of the most helpful but often overlooked is the “URL inspection” tool. To use, paste your URL into the “Inspect any URL” field on the top of every Search Console page. You can also access the “URL inspection” tab in the left desktop menu or from the “Pages” section by clicking any URL in the report.
Search Console’s URL inspection tool is at the top of every page and via the left-side menu item.
The URL inspection tool shows obvious non-indexation issues (such as robots.txt blocking) and not-so-obvious.
One of the not-so-obvious in the Pages report is called “Duplicate without user-selected canonical,” wherein Google found two pages with the same content and chose to index one of them. The URL Inspection tool identifies the duplicate pages. Click one of the URLs in the report and then “Inspect URL.”
“Duplicate without user-selected canonical” is a not-so-obvious indexing glitch. Click image to enlarge.
You may need to fix via 301 redirects or new canonical tags, although it’s unnecessary if Google indexes the right page.
Another indexation glitch to investigate is “Duplicate, Google chose different canonical than user,” where Google indexed a page despite a conflicting canonical tag. This often occurs when the non-http page has the proper canonical tag, but Google indexed the https version. The fix could require a developer to point all URLs to https.
“Duplicate, Google chose different canonical than user” is another indexing glitch. Click image to enlarge.
Google discovers web pages via internal or external links and sitemaps. The URL inspection tool shows all links pointing to each indexed page. If a link points to a private page — one you didn’t intend to be crawled or indexed — identify how Google found it. Then correct (internal links) or redirect (external).
The URL inspection tool shows all links pointing to each indexed page. Click image to enlarge.
Search Console can confirm whether Google can see, access, and crawl all elements on a web page.
Paste a URL into the inspection tool and click “View crawled text” to access the HTML rendered by Google.
Paste a URL into the inspection tool and click “View crawled text” to access the HTML. Click image to enlarge.
Then copy that code and paste into any text or HTML editor. Use Ctrl+F (CMD+F) to search for the element you want Google to render. For example, you can verify that Google JavaScript links and renders the anchor texts as intended.
After updating a page, ask Google to re-index it by clicking “Request indexation.” That will prioritize indexing the updated page over others on your site. Avoid using this option too often or for too many pages. A properly structured site is easily crawled by Google without having to request it.
Ask Google to re-index a page by clicking “Request indexation.” Click image to enlarge.
Copyright © 2005 – 2023.
Practical Ecommerce® is a
registered trademark of
Confluence Distribution, Inc.

source