The importance of robots.txt in Search Engine Optimization

The Robots.txt file guides search engine bots through the pages that should be indexed. The Robots Exclusion Protocol – the rules for controlling how web pages on your site are indexed can be defined in robots.txt file. The file should lie at: http://www.yourdomain.com/robots.txt

The robost.txt file plays a vital role in the SEO process. It is important due to:
If you want to exclude pages that should be accessed ONLY by regesitered users.
If you want to exclude the duplicate content pages.

You may use the google webmaster tool to genearte and analyze the robots.txt file.
You can find out more information about robots.txt at http://www.robotstxt.org

Tips to preclude old web site pages which are appearing in the natural search results

It is very important to have a pre planned mapping strategy before new website scheduled launching date. A mapping strategy is a plan that maps existing (old website) pages (urls) to the new pages. This is particularly critical when page names (urls) will change from old site to new site.

If the pages are still listed in the search engine indexes and they have inbound links pointing to them then it is very important to redirect them. By redirecting these links to pages on the new site, we transfer the link value from the old links(pages) to the new ones. We must do by such method that will cause the server to issue a header response of 301 (Permanent). This is the only response that is uniformly interpreted by major search engines as a request to index the new page and transfer the qualities of the old page to it. So before launching a new site, we must in place:

* A transition strategy to map old pages over to new ones
* Identification of existing pages that have valuable inbound links
* Redirecting links using method that issue a header response of 301 (Permanent)

Search Engine Optimization

Search Engine Optimization SEO is the fondation stone to make a site successful on the Internet.
To get higher SERPs of the site in the organic search results, it is necessary to to “On Site Optimization” & “Off Site Optimization”.  Keyword Research & Analysis is the first basic step.  If the keywords are selected after proper research in search engines like google etc. then you need to have the conent interwoven with these selected keywords.
This is the checklist for your site to do the optimization:-

• Html analysis, Identification & Rectification of Html tags if any.
• Broken Link Analysis
• Site Structure Report including Complete Optimization of Java Script, Flash on Index Pages which hinder on High Ranking & Positioning.
• Creation of Meta Tags in conformity with Keywords & Site Content
• Re-writing of Rich & Keywords dense Site Contents Pages for search engines to find and rank site highly.
• Optimize the site up to keywords most relevant to the site supported by multiple keywords rich text links placement.
• Creation of Keyword Rich Alt-labels
• Creation of side & Footer links if needed

Once you have your site optimized then re-host the site and start promoting the site. You need to do “Off Site Optimization” keeping in mind the anchor text must be your keywords. You may start with submission in

• Google,
• Yahoo,
• Dmoz,
• MSN,
• HotBot