On Saturday, I was having some technical trouble with one of my Linux web servers. After scouring the web for an answer with no solution in sight, I decided to call in reinforcements and ask a question over at Web Hosting Talk. Being ever persistent, I went back and started toying with my server some more. Then my computer-nerd-ADD set in, and I decide to go back to Google and type in the specific error code I was getting, just in case I missed it the last 20 times I did a search. As my eyes scanned the SERPs, my heart leapt when I saw my exact error code nice and bold. But wait, this link points to Web Hosting Talk forums. I read the blurb a little further and realize that it’s my post! In less than 10 minutes Google had spidered WHT and indexed it.
So what exactly is the significance? When it comes to established web sites that have a certain amount of authority, the big G won’t hesitate to snap up new pages and pop them into the index. I know what you’re thinking, “Yeah, that’s great Glenn, but we already know about the Google Sandbox and trusted authority sites.” My point is I think the speed at which new pages can be spidered and indexed has far been underestimated. Instead of waiting a day or two to see pages in the SERPs, it’s actually possible for them to appear in a matter of minutes. That’s one Speedy Spider!