Google has announced an update to Googlebot, set to be released in November. This new update will now allow Googlebot to crawl selected sites over HTTP/2.
https://twitter.com/googlewmc/status/1306563662740705282
HTTP/2 is the successor of both the HTTP/1 and SPDY protocols. Due to the limitations of HTTP/1.1, SPDY was created by Google engineers in 2009 with the aims of reducing web page latency speeds and improving web security. HTTP/2 was based on Google’s SPDY protocol but was able to provide a large improvement on performance. After its release in 2015, HTTP/2 was embraced by all browsers, with Microsoft dropping any support for SPDY quickly after HTTP/2’s release and Google Chrome dropping SPDY support in 2016.
The benefits of this update are centred around making crawling more efficient, in terms of server resource usage. Google are now looking to start with crawling a smaller number of sites over HTTP/2 and expanding this out over time.
There is no discernible reason as to why a site would need to opt out of this update, however Google have noted that should a site wish to opt out, this can be done by instructing the server to respond with a 421 HTTP status code when Googlebot attempts to crawl your site over HTTP/2. Failing this, sites can also contact the Googlebot team (however, this solution is only temporary).
by Zack Cornick
