Guide To Search Engine Optimization

Go Back To The Search Engine Optimization Guide

Rocky Roads And Dead Ends For Search Engine Crawlers

Sometimes certain types of information can actually hurt your website because it will either make your website very hard for the search engines spiders to crawl and sometimes there are a lot of things that keep the spiders from crawling your site. 

The term ”rocky roads” is an example of when the search engine gets to your site and will have a hard time navigating your site due to many different issues. These spiders only have a certain amount of time to crawl your whole website and then move on to the next.  If it can’t crawl your website then it will post an error message stating it can’t crawl your page because the page may not be reachable The term “dead ends” refers to web pages or links that are not searchable by the search engines.

Examples of Rocky Roads for the Search Engine Crawlers

Sometimes having more than 100 links to other pages may cause the spiders not to crawl each link.Some pages that require a “cookie” or better known as a “Session Identification” for the navigation to start on the website.

Examples Of “Dead Ends” for the search engine crawlers

Pages that require the user to create a login to sign in to access a page
Pages that are only available by submitting your name and email address before you can view the page.The best way to avoid these problems is to make sure all of the links that link from your home page are working properly. 

The search engines will land on your home page first before all the other pages and will start their crawl from the home page.  If the search engine spiders can’t make it off your home page then your other pages will not get indexed properly.  We will speak about another valuable technique to help the search engine spiders crawl your site more effectively later on in this guide.

Go Back To the Guide To Search Engine Optimization