Google has launched a brand new robots.txt report inside Google Search Console. Google additionally made related data round robots.txt obtainable from inside the Web page indexing report in Search Console. Lastly, Google has determined to sundown the robots.txt tester.
The brand new robots.txt report. Google’s new robots.txt report exhibits offers you some data Google has in your robots.txt file together with:
- Which robots.txt information Google discovered for the highest 20 hosts in your website
- The final time Google crawled these information,
- Plus any warnings or errors encountered.
Google additionally added to this report the power request a recrawl of a robots.txt file for emergency conditions, Google mentioned.
What it appears to be like like. You’ll be able to entry this report inside Google Search Console beneath settings, here’s what it appears to be like like:
Extra. Google additionally made related data round robots.txt obtainable from inside the Web page indexing report in Search Console.
With this new robots.txt report, Google has determined to sundown the robots.txt tester.
Extra assist. Google has a extra detailed assist doc on this robots.txt report over here.
Why we care. In case you are having indexing and crawling points, this report might provide you with extra perception into if the difficulty is expounded to your robots.txt file.
You must in all probability overview this report for the websites you handle inside Google Search Console, simply to make sure there aren’t any points with Google having the ability to entry your website because of robots.txt directives.
