Google May Expand Unsupported Robots.txt Rules List via @sejournal, @MattGSouthern

Google may expand its unsupported robots.txt rules list using HTTP Archive data and could broaden how it handles common misspellings of disallow.

The post Google May Expand Unsupported Robots.txt Rules List appeared first on Search Engine Journal.