Internet Marketing Tips, Suggestions, & Ramblings

Making AJAX Application Crawlable for Search Engine Optimization

Making an Ajax application crawlable by the Google search engine is not incredibly complicated if you take the time to ensure the necessary technical requirements have been met. In this post, we will cover some of the key concepts in AJAX application search engine optimization.

Overview of AJAX SEO

Google and other search engines have a very difficult time crawling AJAX applications. Because of this, Google allows HTML snapshots to be delivered from the AJAX URL. The HTML page should be optimized for the terms that related to the AJAX content. In order to make this happen correctly we must follow the guidelines below. This information has been taken from documentation supplied by Google to ensure accuracy.

Indicate to Google that you have an AJAX site Ready for Crawling

First we need to create a URL that Google sees as being crawlable. In order to do this, we need to add a special character after the hash fragment.

Current URL

www.example.com/ajax.html#key=value

Correct URL

www.example.com/ajax.html#!key=value

By adding this relationship to your URLs it will now become SEO friendly and Google will pay attention to the information following the !.

Enable your Server to Handle _escaped_fragment

If you would like to get Google to index an AJAX web property you must supply the correct information. Google request that you offer a HTML optimized piece of content. The Google crawlier will modify each URL you provide as follows.

The crawler will modify this URL

www.example.com/ajax.html#!key=value

And Cause it to become this URL

www.example.com/ajax.html?_escaped_fragment_=key=value

According to Google, this is the reasoning behind this. Hashtags are never sent to a server as part of a HTTP request. But your server needs to know that it should deliver the HTML to the crawler.

After you have your URLs in place simply create an HTML snapshot. There are few different ways to do this so make sure to read the documentation. Google recommends one of the three methods.

  • If a lot of your content is produced with JavaScript, you may want to use a headless browser such as HtmlUnit to obtain the HTML snapshot. Alternatively, you can use a different tool such as crawljax or watij.com.
  • If much of your content is produced with a server-side technology such as PHP or ASP.NET, you can use your existing code and only replace the JavaScript portions of your web page with static or server-side created HTML.
  • You can create a static version of your pages offline, as is the current practice. For example, many applications draw content from a database that is then rendered by the browser. Instead, you may create a separate HTML page for each AJAX URL.

Once you have enabled your URL correctly, created an HTML snapshot and map everything correctly. Your final step is to test the page. Make sure to use the Fetch as Googlebot tool to make sure everything is being picked up correctly.

About Garry Grant

Garry Grant is a veteran expert in search engine optimization and the digital marketing industry. With nearly 20 years of experience, Garry has successfully built a multi-service operation at SEO, Inc., developing proprietary technologies through complex strategic solutions. He has extensive experience in key initiatives and operational responsibilities grounded in information technology and performance management.

Garry’s expertise and esteemed reputation, coupled with SEO Inc.’s impressive client success record has earned him such accolades as Entrepreneur Magazine's 2005 Hot List for the Hottest Internet Property, Inc. 500 2007 Honorary award for Fastest Growing Private Company in America, an Inc. 500 top 50 Company in San Diego, and interviews with The New York Times, The Wall Street Journal, WIRED, Entrepreneur and The Huffington Post.

Garry Grant began his online career in 1993 creating strategic Web and e-business solutions for Homepage.com, The Rush Limbaugh Show, Premiere Radio Networks, Clear Channel Communications, EarthLink and Artisan Motion Pictures. Today, Garry and SEO Inc.’s highly skilled digital strategists develop proprietary technology and strategic digital marketing direction for Fortune 500 companies including, SC Johnson, McAfee, Entrepreneur.com., Inc Magazine, IGN, Tacorri, LPL Financial, National Kidney Foundation, G4 TV, Fuel TV and Sony, just to name a few.