Page Inventory is here. If you're ever wondering about the number of pages in your search results or find the need to check up on any missing pages in your search, look no further. Page Inventory will provide you with an overview of indexed content for all your domains to ensure transparency and give you more control.
Gain more insight to crawled content through Page Inventory
Page Inventory can be accessed from MyCludo ➜ Settings.
You can access Page Inventory to see a table of all the sites you're crawling. The table is split up so that you'll have an overview for each crawler, showing the data for each domain assigned to it. This table will provide data from the last finished crawl:
Page Inventory explained
Total page requests in last crawl
The number of pages found during the last crawl.
Total pages indexed in last crawl
The number of pages that were indexed during the last crawl.
Total pages in search results
The number of pages that appear in your search results.
This parameter may vary from Total pages indexed in last crawl from time to time, as it can take up to three crawls for deleted pages to be definitively dropped from the search results, or if you're using smart crawling *.
The domain(s) that the crawler is set up to index from.
The number of pages crawled during the last crawl. Crawled content doesn't equal to indexed content and account for all the pages that the crawler came across - including pages with canonical/noindex tags or missing fields.
The number of pages indexed during the last crawl. These are the pages that will appear in your search results. This number should stay below the page limit included in your subscription.
The number of pages that weren't indexed due to a missing title**.
The number of pages that weren't indexed due to a missing description**.
The number of pages that weren't indexed due to a required field not being found.
Typically, only title and description are required fields, but you might have a custom setup with additional required fields.
The number of crawling errors during the last crawl. Crawling errors most often happen due to server timeouts. If you have a crawling error and would like to have it investigated, you're always welcome to contact support.
* This security restriction is set up to ensure that no pages are dropped from the index in case of crawling errors, server downtime, etc. If you're using smart crawling, this number will almost always be different, as only pages that were updated since the last crawl (reading the <lastmod> tag) will be crawled.
** If you come across pages that weren't indexed due to a missing title or description and you think these should be included, contact support to resolve.
If you have any findings within Page Inventory that you'd like to investigate, go to Crawler Logs to review all crawled urls.