Tag Archives: google

The Zopim chat widget and Google processing Javascript when crawling

A client contacted me with an odd problem recently. When searching for their own company name in Google, this is the snippet beneath the link to their site:

“We’re sorry! Seems no one can serve you now. If you leave your email address, we’ll get back to you soon.”

Ah, I thought, they’ve got some odd text in their page somewhere, and Google has picked up on it. So I look at the source of their home page and… no sign of the text. No use of “sorry” at all.

So, maybe this is an old cache of the page and the text has changed. I check the source of the cached page in Google, nope. Now, maybe the cache is a different version from what’s being used to build the snippet, but that’s unlikely and the client says they haven’t ever had the sorry text on the page.

So I checked whether anyone else is having this problem by searching for the exact start of the phase in Google.

Lots of results, all with the same snippet text:

Google Search results for "We're sorry. It seems no one can serve"

Opening up a few of the pages I can see they’re all using the same chat widget from Zopim.

Sensibly, if you try to use the chat widget when no one is available, Zopim will show a friendly message saying no one can be contacted. However, it’s not the same message as I’m seeing in the snippet.

So, I right-click on one of the pages in Chrome and use ‘Inspect Element’, this is using Chrome’s developer tools to see what’s on the page when it’s finished being made, including any changes Javascript code may have made to it. However, searching still doesn’t show the word ‘sorry’.

I’m running out of ideas now, so use a small script to grab the source of the client’s page as if it was a search engine crawler. Definitely no phrase in there, or use of the word ‘sorry’.

I remember Firebug in Firefox runs a little differently to Chrome’s developer tools, so use that to check a page. Hey presto, there’s the “We’re sorry…” text. Setting up the screenshot, I click on the little ‘f’ you can see below. That’s the Flashblock extension at work, in Firefox I have to click on any area where Flash wants to run. I installed it to stop obnoxious adverts running. However, when I allowed Flash to run, the message I was looking for disappeared.

So, when Google is crawling the web, it is running Javascript. That’s the only way that it could have seen this text that it’s grabbed as the snippet. It does not run Flash content, otherwise it would not have seen this message.

We're sorry text in source of the page
(Note, not my client. I’m under NDA with them so I’m not saying who they are.)

I’ve heard rumours of Google running Javascript within its crawler for years. I’ve seen it able to get at pages that some Javascript navigation had hidden away, although only when that navigation was using very common scripts like the ones that come in Dreamweaver, or sites using the #! URL schemes like Twitter did for a while. This is the first time I’ve seen it definitely run Javascript, and also pluck out a message only revealed by Javascript in to the search results snippet.

Does this mean we can be sure Google will crawl all of the content on a site using Javascript to load all of it’s content? No, I would say this current crawling is experimental – choosing the “We’re sorry…” text for the snippet on the brand search that kicked off my investigation was a very poor choice given the other text available on the page. I can only think they did this because the text was very high up on the page, within thesection. Does it point to them doing more and more to crawl the web ‘naturally’, as most web users do? Yes.

As more and more content gets hidden away behind AJAX and snazzy, Javascript-run interfaces, Google will have to put more and more effort in to being able to crawl that content effectively. This is proof they are doing that, if imperfectly.

If you use the Zopim chat widget, you may want to move the block of Javascript you put in thedown to the footer of the page and check if the chat still works. You don’t want a useless snippet in a brand search for your company just because of the chat service you’re using.

A Google A/B test for Adwords adverts

Yesterday I noticed when I was searching in Google using a Chrome ‘incognito’ window I was seeing a different version of Google’s search results than normal. Here’s what I was seeing in the incognito window:

New 'Ads' next to Adwords adverts

And for reference, here’s what I’d normally see at the moment:

current-google-adwords-block

The difference is the Adwords block of adverts above the search results had a white background rather than the yellow they’ve shown for many years, and each advert at the top is noted with an ‘Ad’ label on a dark yellow background. The adverts on the side also have this more noticeable label above them, where the subtle, grey on white label currently sits.

This is what’s known as an ‘A/B’ test – Google are showing me (or the one version of me represented by that web browser) a different version of their results page and seeing how I react to it. They’ll do this to thousands, potentially hundreds of thousands of people, then statistically judge which of the layouts is best, under whatever criteria they have set – understanding for the person seeing them, amount of people who click on the adverts, or anything you can record.

Does this mean we’re going to see a change in the search results soon? Well… maybe. Google are well known for testing potential changes before they go live, but most are subtle enough that you don’t notice them.

They have received criticism in the past because apparently a large percentage of searchers do not notice the yellow background to the adverts above the main results, and therefore don’t realise they are paid adverts and not part of the ‘natural’, unpaid for results. Perhaps this label next to each advert is a way for Google to ensure people do realise what each of the adverts above the main results are – paid placements.

I think this could be a good change if it is indeed true that people do not realise they are adverts. I also prefer the way the ‘sitelinks’ under the the adverts work – they’re the extra links under the top advert in the upper screenshot of the new layout, under the second advert in the lower screenshot.

Only time will tell if there is enough response to this for Google to decide to roll it out to everyone. Today, I don’t see it in any of my browsers. Such is the constant change that goes on behind most of the big sites we visit every day, most of which we never even notice.