With Page Inventory, a more detailed crawler log view has been introduced. You should view the crawler log if you ever find the need to investigate the details around the last crawl.
Access Crawler logs by going to Settings ➜ Crawlers and click the date stamp for the crawler you want to view the log for.
With the new crawler logs, we introduce two new fields to filter from: Domain and Log type.
Crawler log fields
A dropdown of crawled domains.
Enables you to filter the crawler logs based on the type of message.
As always, you can search within crawler logs to find more specific information. This is especially useful if you want to view the logs for a specific url.