The basics of SEO could actually be traced to the words Google Crawling and indexing of a website or blog. What I mean is that there is no way we can discuss search engine optimization without using these two terms- Crawling and Indexing.
To Crawl a blog or website by Google means to visit the web page of that blog or website. This is done through Google spider or crawler. Google Crawling can also be said to mean that when your links are followed and your blogs or websites are crawled by google.
Naturally, Google crawls every aspects or parts of your blog or website but to prevent google from crawling certain parts like your admin dashboard, feeds, trackback as well as comments on your blog or website you can stop this with the help of your robot.txt file.
Google indexing a blog or website on the other hands means that after the blog or website must have been crawled by the search engine, the result would appear in Google index. It also means adding a blog or website WebPages to Google search results based on the meta tags used- that is (index or noindex.).
Note that for those on wordpress platform, all posts and pages are indexable by default but using noindex meta tag, you are saying that certain pages or post on your blog or website should not be added to google search results.
However, Google Crawling and Indexing of blogs and website are not always carried out equally on every site due to the following reasons;
Backlinks
Backlinks play an important role on how blogs and websites are crawled and indexed by the search engines. No matter how rich your contents are, if no blog or website is linking back to your site or blog, Google would assume that your blog posts are less relevant. The more quality backlinks you have, the more your site or blog gain trust and reputation in the eyes of google and all other search engines. Also, regardless of your blog or website rankings without backlinks for your website or blog, Search engine may assume that your blog or website posts are of low quality .
Domain Name
The rate of crawling and indexing a website or blog could also depend on the kind of domain. Domain names with high PR are more crawled and index by goggle compare to those domains with less PR.
Internal Linking
Like I said in one of my previous posts that using a sentence or sentences as a link in a post known as the anchor text is very good for SEO and this would increase the crawling and indexing of your blog or website.
Sitemap
The use of XML sitemap especially those of us on wordpress, enables the search engines to be notify of our blog or website and your site pages can be crawled and indexed.
Website/Blog pinging
Pinging your blog or website immediately after each posts or contents is a good way to also get the search engines notified about your site and this will definitely increase the crawling rate of your blog/website. Note that for those on wordpress, after publishing a post, your blogs are automatically pinged to all the search engines.
