Beginning on the 25th of Letshost made a few changes and, according to an email from them “upgraded the firewall as it was not working correctly” – naturally and on reflection, I’d prefer if they had simply left the firewall alone.
From that date, the servers are now blocking access to Googlebot crawlers (or more specifically their IP addresses). This means that when the bot goes to look at a web page it has stored in its index, the new Letshost firewall configuration throws the bot a complete curve ball (see below) and Google then marks this page as a 404 – or not found. The graph above shows the suddent rise in the 404 errors.
Below shows a selcesion of the IP adresses that they have blocked (although they may have whitelisted a few of thses now.
What’s worse is that once the page is returned as a 404, Google can remove that page from its own index and thus insuring that the page won’t be returned in search results any further.
Constant emails to the support team result in nothing bar an attempt to pacify – but little to correct the situation.
he real point is that if the server is blocking access to these IP addresses on one of my sites, then there is a good chance that it’s blocking access to other sites as well. Its only when the individuals in control of those other sites realise that this is happening can begin to take action.
Anyhow, if they are not aware of this error on the Letshost firewall, I think it’s incumbent on me to educate them..
Tuesday: I see today that I have 75 soft 404’s, however many of these are returning a response to ‘fetch as Google’
