The Zopim chat widget and Google processing Javascript when crawling

A client contacted me with an odd problem recently. When searching for their own company name in Google, this is the snippet beneath the link to their site:

“We’re sorry! Seems no one can serve you now. If you leave your email address, we’ll get back to you soon.”

Ah, I thought, they’ve got some odd text in their page somewhere, and Google has picked up on it. So I look at the source of their home page and… no sign of the text. No use of “sorry” at all.

So, maybe this is an old cache of the page and the text has changed. I check the source of the cached page in Google, nope. Now, maybe the cache is a different version from what’s being used to build the snippet, but that’s unlikely and the client says they haven’t ever had the sorry text on the page.

So I checked whether anyone else is having this problem by searching for the exact start of the phase in Google.

Lots of results, all with the same snippet text:

Google Search results for "We're sorry. It seems no one can serve"

Opening up a few of the pages I can see they’re all using the same chat widget from Zopim.

Sensibly, if you try to use the chat widget when no one is available, Zopim will show a friendly message saying no one can be contacted. However, it’s not the same message as I’m seeing in the snippet.

So, I right-click on one of the pages in Chrome and use ‘Inspect Element’, this is using Chrome’s developer tools to see what’s on the page when it’s finished being made, including any changes Javascript code may have made to it. However, searching still doesn’t show the word ‘sorry’.

I’m running out of ideas now, so use a small script to grab the source of the client’s page as if it was a search engine crawler. Definitely no phrase in there, or use of the word ‘sorry’.

I remember Firebug in Firefox runs a little differently to Chrome’s developer tools, so use that to check a page. Hey presto, there’s the “We’re sorry…” text. Setting up the screenshot, I click on the little ‘f’ you can see below. That’s the Flashblock extension at work, in Firefox I have to click on any area where Flash wants to run. I installed it to stop obnoxious adverts running. However, when I allowed Flash to run, the message I was looking for disappeared.

So, when Google is crawling the web, it is running Javascript. That’s the only way that it could have seen this text that it’s grabbed as the snippet. It does not run Flash content, otherwise it would not have seen this message.

We're sorry text in source of the page
(Note, not my client. I’m under NDA with them so I’m not saying who they are.)

I’ve heard rumours of Google running Javascript within its crawler for years. I’ve seen it able to get at pages that some Javascript navigation had hidden away, although only when that navigation was using very common scripts like the ones that come in Dreamweaver, or sites using the #! URL schemes like Twitter did for a while. This is the first time I’ve seen it definitely run Javascript, and also pluck out a message only revealed by Javascript in to the search results snippet.

Does this mean we can be sure Google will crawl all of the content on a site using Javascript to load all of it’s content? No, I would say this current crawling is experimental – choosing the “We’re sorry…” text for the snippet on the brand search that kicked off my investigation was a very poor choice given the other text available on the page. I can only think they did this because the text was very high up on the page, within thesection. Does it point to them doing more and more to crawl the web ‘naturally’, as most web users do? Yes.

As more and more content gets hidden away behind AJAX and snazzy, Javascript-run interfaces, Google will have to put more and more effort in to being able to crawl that content effectively. This is proof they are doing that, if imperfectly.

If you use the Zopim chat widget, you may want to move the block of Javascript you put in thedown to the footer of the page and check if the chat still works. You don’t want a useless snippet in a brand search for your company just because of the chat service you’re using.

A Google A/B test for Adwords adverts

Yesterday I noticed when I was searching in Google using a Chrome ‘incognito’ window I was seeing a different version of Google’s search results than normal. Here’s what I was seeing in the incognito window:

New 'Ads' next to Adwords adverts

And for reference, here’s what I’d normally see at the moment:

current-google-adwords-block

The difference is the Adwords block of adverts above the search results had a white background rather than the yellow they’ve shown for many years, and each advert at the top is noted with an ‘Ad’ label on a dark yellow background. The adverts on the side also have this more noticeable label above them, where the subtle, grey on white label currently sits.

This is what’s known as an ‘A/B’ test – Google are showing me (or the one version of me represented by that web browser) a different version of their results page and seeing how I react to it. They’ll do this to thousands, potentially hundreds of thousands of people, then statistically judge which of the layouts is best, under whatever criteria they have set – understanding for the person seeing them, amount of people who click on the adverts, or anything you can record.

Does this mean we’re going to see a change in the search results soon? Well… maybe. Google are well known for testing potential changes before they go live, but most are subtle enough that you don’t notice them.

They have received criticism in the past because apparently a large percentage of searchers do not notice the yellow background to the adverts above the main results, and therefore don’t realise they are paid adverts and not part of the ‘natural’, unpaid for results. Perhaps this label next to each advert is a way for Google to ensure people do realise what each of the adverts above the main results are – paid placements.

I think this could be a good change if it is indeed true that people do not realise they are adverts. I also prefer the way the ‘sitelinks’ under the the adverts work – they’re the extra links under the top advert in the upper screenshot of the new layout, under the second advert in the lower screenshot.

Only time will tell if there is enough response to this for Google to decide to roll it out to everyone. Today, I don’t see it in any of my browsers. Such is the constant change that goes on behind most of the big sites we visit every day, most of which we never even notice.

My first Stripe integration, using PHP

Recently I carried out my first integration to the new (to the UK) payment processing provider Stripe. My client Jasper Goodall was looking at moving to Paypal for taking payments and having integrated with Paypal in the past, I suggested Stripe as a more developer and client friendly alternative. He liked the look of Stripe and signed up.

I was moving the site from using Sagepay over to Stripe and as all the basket and post-payment logic was written, dropping in Stripe was very straightforward. Their documentation is very clear, testing is simple, and taking a payment is a doddle. I hit a few minor issues, which were:

PHP needs ‘mbstring’ turned on

My local PHP 5.2.4 install didn’t have mbstring on by default, so I had to install it. This was on my Windows PC so I had to move the mbstring DLL in to the main ‘php’ directory and edit the httpd.conf file to include the DLL as one that needed to be loaded. I then restarted Apache and it was working fine.

My host did have mbstring turned on, so I didn’t need to change anything there.

Secure certificate required

My client didn’t have a certificate to allow his to use SSL. However his host, Claranet, had a shared secure area that all of their customers could use without requiring their own certificate. This was good, but I had to re-code parts of his site so things like the stylesheets would load properly when using this area.

If you or your client are on shared hosting with one of the larger providers, it’s worth checking to see if they have a secure area you can use before investing in your own certificate. This can save you some money, but has the disadvantage that the URL of your secure pages won’t show your domain in that part of the website address, which may make some customers suspicious. It will be worth adding some text to your pages explaining how they are secured if you think that is going to be a problem.

Coding up receipts

Previously, we were using Sagepay which takes various information about the products in the customer’s basket and builds an e-mail notification of the sale for you. Stripe just take the amount of money you are charging, so I had to update the website to create a notification for the customer, and one to tell the client a sale had been made. These were very simple additions to the post-sale process.

Overall, Stripe was a delight to integrate with. I’ve set up shops using several payment gateways – Sagepay (was Protx), Worldpay, Secure Trading, Paypal, and Paypoint. Stripe was by far the easiest, taking only a few hours to integrate with including the bug fixing of my setup and the re-coding of the existing shop to send the right information through and use the extra secure area of their hosting. If you were starting from a cleaner base, you’d probably be looking at an hour or two including reading the documentation. Really nice.

I am giving (and listening to) a Freelancing talk on Tues 24th

On Tuesday 24th September I’ll be talking about Freelancing at the ‘Achieving Freelancing Awesomeness‘ evening hosted by Freelance Advisor in Hove, along with Kati Byrne and Bex White.

I’ll be talking about making friends with other freelancers to help you get more work in, Bex is talking about managing your time effectively so you can grow your business, and Kati about using social media as a freelancer. There will also be lots of time for questions and answers about the talk and anything to do with freelancing.

I’m very much looking forward to the event and hearing the other talks. If you’re interested you can find out more on Freelance Advisor, or go straight to sign up here.

The event is from 6pm-8pm at Freelance Advisor / Crunch’s office, which is in the ‘Perfumery’ building next to Hove station so it’s a doodle to get to – either train to Hove, or if you’re in Brighton and don’t want to walk it, the number 7 and 7A buses go to Hove station, you’ll need to cross over the tracks to get to the right building.

There are quite a few people registered already, so if you’re interested please get a ticket now.

Latest client project: Grafton Banks Finance re-build

Over the last few weeks I’ve been re-building the Grafton Banks Finance recruitment website. I’ve been working with them for many years, maintaining and extending the previous version of the site, which was frankly long in the tooth from their beginnings, being based on another site from within the same group.

Screenshot of Grafton Banks FinanceThe graphical side was adroitly handled by Nick Carter, and as ever he came up with the goods – a great looking site the client is happy represents them properly. The team at GBF were nervous about being able to work with Nick as this is outside their field of expertise, but he did a great job in guiding them through what they needed to think about, and then interpreting their brief.

I handled the front-end build and integration with their existing back-end PHP system, with some updates to help their business grow, including the addition of a much needed jobs by e-mail facility. Some of the front-end Javascript work saw me struggle with finding jQuery plugins and getting scripts to inter-operate. I’m very glad I signed up for a Javascript course earlier this year, and will be happy when I’ve got my head around the language in more depth so I hit less problems in this area in the future.

Screenshot of Grafton Banks Finance on an iPhoneThe site has a mobile-friendly version via Media Queries within the CSS controlling the layout of the site. It is not a fully ‘responsive design’ which would cope with all sizes of screen due to time constraints, the mobile version targets iPhone’s specifically, but also looks fine on my lower-end Android handset after an extra screen tap. This is another area I need to improve on, the conversion process only had some small hiccups and some friends in the Farm were happy to point me in the right direction to fix things, but I could have avoided the problems I was having with more experience. Some conversions of my own sites will give me some much needed practice in these areas.

Although I my own process with the work on the site could have been smoother, I think the end results for their business are good. The team at Grafton Banks Finance are lovely people, and I’m happy to have been part of building them a website that reflects their company and values properly. Check out the site here.