Internet Marketing Tips, Suggestions, & Ramblings

Making AJAX Application Crawlable for Search Engine Optimization

Making an Ajax application crawlable by the Google search engine is not incredibly complicated if you take the time to ensure the necessary technical requirements have been met. In this post, we will cover some of the key concepts in AJAX application search engine optimization.

Overview of AJAX SEO

Google and other search engines have a very difficult time crawling AJAX applications. Because of this, Google allows HTML snapshots to be delivered from the AJAX URL. The HTML page should be optimized for the terms that related to the AJAX content. In order to make this happen correctly we must follow the guidelines below. This information has been taken from documentation supplied by Google to ensure accuracy.

Indicate to Google that you have an AJAX site Ready for Crawling

First we need to create a URL that Google sees as being crawlable. In order to do this, we need to add a special character after the hash fragment.

Current URL

Correct URL!key=value

By adding this relationship to your URLs it will now become SEO friendly and Google will pay attention to the information following the !.

Enable your Server to Handle _escaped_fragment

If you would like to get Google to index an AJAX web property you must supply the correct information. Google request that you offer a HTML optimized piece of content. The Google crawlier will modify each URL you provide as follows.

The crawler will modify this URL!key=value

And Cause it to become this URL

According to Google, this is the reasoning behind this. Hashtags are never sent to a server as part of a HTTP request. But your server needs to know that it should deliver the HTML to the crawler.

After you have your URLs in place simply create an HTML snapshot. There are few different ways to do this so make sure to read the documentation. Google recommends one of the three methods.

  • If a lot of your content is produced with JavaScript, you may want to use a headless browser such as HtmlUnit to obtain the HTML snapshot. Alternatively, you can use a different tool such as crawljax or
  • If much of your content is produced with a server-side technology such as PHP or ASP.NET, you can use your existing code and only replace the JavaScript portions of your web page with static or server-side created HTML.
  • You can create a static version of your pages offline, as is the current practice. For example, many applications draw content from a database that is then rendered by the browser. Instead, you may create a separate HTML page for each AJAX URL.

Once you have enabled your URL correctly, created an HTML snapshot and map everything correctly. Your final step is to test the page. Make sure to use the Fetch as Googlebot tool to make sure everything is being picked up correctly.