Quick Answer
No Index is a setting you select for a webpage that that instructs search engines not to index it, and therefore, prevents it from showing up in search results. This impacts your SEO because the search engine will not not consider that page a part of your site. You want to use this feature to make search engines focus on the key pages of your site and ignore unimportant page and pages with low quality or duplicate content.
Longer Answer
As stated above “No Index” is a setting for a webpage that tells a search engine “don’t pay attention to me”. However, it really isn’t a “setting”. It is a tag in the header of the page that has a directive of “No Index”. If that doesn’t make that much sense to you, I wouldn’t worry too much about it. If you are using WordPress or some other CMS then there is probably some tool on your site that allows you to select not to index the page; I personally use All In One SEO. If you aren’t using a CMS or are just interested in the details, what is really going on is a robots tag is being added with the “No Index” directive. It looks like this…
<meta name=”robots” content=”noindex” />
With this tag you are explisitely saying that you don’t want search engine to index the page. It does not mean that the page isn’t being crawled. Tthat is a diffrent setting/directive (no follow). At this point you should have a good idea of what no index is doing, so let’s dig into the weeds a little bit.
What exactly is an “index”
After a page is discovered and crawled, Google tries to understand what the page is about. This process is called indexing. Google analyzes the content of the page, catalogs images and video files embedded on the page, and otherwise tries to understand the page. This information is stored in the Google index, a huge database stored in many computers.
So an index is simply a database of information that the search engine has about a given webpage that it uses to demine when to show it to users.
When to use No Index
In the never ending quest to optimize your site it can be difficult to determine the exact pages you should index and which ones you shouldn’t. As with most web based questions the answer isn’t very satisfying but you should look to your analytics and search console to determine what pages have high value and make informed decisions from there. However, there are a few hard set rules in my opinion you can apply and then a few questions that I always ask myself when trying to make a decision on a page.
Hard Set Rules
times I always feel you should no index a page
Admin Pages: There is no reason any page that requires a user to be logged in to show up in search results
Duplicate Pages: If you have pages that are exactly the same then only one should be indexed
Community Posts: If you have a website that allows community members to create a profile to contribute blog posts or content, this is something you will want to regulate. You only want the profiles who contribute the best content to show up on SERPs, because this is an indicator of website authority and good UX (User Experience).
Paid Content: This is in the same vein as the admin pages, only your freely available content should be indexed.
Low Content Pages: If you have a page with just a few sentences on it then it should not be indexed, even if it’s a relevant page. Google has an algorithm, the Panda Algorithm, that seeks out low quality pages and flags them and you want to avoid that.
Flexible Rules
Times where you will have to make a judgement call
This one is tougher becuase there is no hard set rule and you are usually trying to make a judgement call on a page that might apply to one of the hard set rules but might not. A difficult one I always struggle with is product pages. They are relevant pages, but they aren’t always what I would consider “content rich”. So, how do you decide if you should index it or not? Personally, I ask my self a few questions and I feel the answers to them are a pretty good judgement.
The Questions I ask are
Does this content exist elsewhere already, and is that page more important? : Maybe the content isn’t an exact match but similar content exists elsewhere on my site. I’ll think to myself what makes this content different from that other page? is there a way to publish this content differently through keywords so both pages are indexed and target different but similar keywords? If the answer is yes, then I may index it, but if I feel the content is too similar then I will pick what I think is the most valuable of the pages and index that one.
What would someone search to find this content?: If I can’t think of anything that someone would search to find the content then I don’t index it. However, using the example above it gets a little more tricky, but the answer is the same. If I think that both pages are relevant to the keywords that this content could show up for, then I either try to index both relevant to a particular targeted phrase, or only index one of them. An example of this would be if I have two web design pages, one could be targeting “St. Louis web design services” and one could target “top St. Louis web design agencies”.
They are both about web design but I can make the content different enough to target both. However, say I have a web design services landing page and my general St. Louis Web Design page of my site. The content of these are probably similar because they are both about my web design services. Because of this I would only want to index the most important page, which would probably be my general page.
Do I want this content to be found?: Sometimes is as simple as that.
In the end you are the best judge of the page. Is it good content, would you want to see it in our search results, could it interfere with some more important pages, etc. After asking yourself these you should have a good feel for if you should index it or not.
