Use Robots.txt, Save the World
Robots.txt Help the Search Engines Learn All About Your Website
There is a growing interest in the little known file that every website should have in the root directory: robots.txt
It’s a very simple text file you can find all about at the robotstxt.org website.
Why should you use it? Here are some good reasons for you to consider.
Controlled Access to Your Content
With a robots.txt file you can “ask” the search engines to “keep out” of certain areas of your website. A typical area you might like to exclude is your images folder: If you aren’t a photographer, painter and your images are for your website use only, there are good chances you don’t want them to be indexed and showing up on image search engines, for people to download, or hotlink.
Unfortunately grabbers and similar software (such as Email harvesting applications) will not read your robots.txt file disregarding any indication you may provide in this respect. But that’s life isn’t it, always someone being disrespectful to say the least …
Read more
Sitemaps Improve Site Value
Getting your pages indexed. It is your most important SEO goal and perhaps the one most vital in determining the success of your SEO campaign. However, many search engines have trouble finding links buried deep within the structure of your site. So how do you make sure your pages are easy for the search engines to find? With a sitemap. Creating a sitemap provides the search engines with a one-stop-shop for all of the pages on your site. And if designed correctly, your sitemap can also be a valuable resource to lost visitors looking to understand your site structure.
What is a Sitemap?
A sitemap displays the inner framework and organization of your site’s content to the search engines. Your sitemap should reflect the way visitors would intuitively work through your site. Years ago sitemaps existed only as a boring series of links in líst förm. Today, they are thought of as an extension of your site. You should use your sitemap as a tool to provide your visitor and the search engines with more content. Create details for each section and sub-section through descriptive text placed under the sitemap link. This will help your visitors understand and navigate through your site, and will also give you more food for the search engines. You can even go crazy and add Flash to your sitemap like we did with the interactive Bruce Clay sitemap! Of course, if you do include a Flash sitemap for your visitor, you will also need to include a text map so that the robots can read it.
Read more
