In this article we’ll look at How to Remove Meta Robots Tags, the ones which cause search engines not to index your website and is a follow-up article to my previous post on the ‘Importance of META Robots Tag in Search Engine Optimisation‘ where this problem was first explained.

I am indebted to Leland from ThemeLib for the solution (read the discussion comments on the post mentioned), because to tell you the truth, I was wrong in what I thought was happening and the solution to my problem was far simpler than I had envisaged.  So join me as I go through the problems I faced and how I fixed them.

So, to start off with, its a good thing to check your production code for your blog by doing the View | Source commands to look through the source HTML for your website and check the Header section to make sure these META Robots tags of NOINDEX and NOFOLLOW are not present.  Now in my case, they were present, and once I’d found them, I had to take a drastic step of removing a PHP function, which affected other things.  It solved my problem, but it was n’t the solution, because I still had n’t got to the bottom of where and how these tags came to be in my code in the first place.

The PHP function I removed in the Header section, was a hook-in point for the various widgets and plug-ins to insert code into the header.  Now, I had systematically removed all these, and put them back in and found none of these were the cause of the problem.  This is when Lelands advice to go back to the Wordpress Dashboard and check my ‘Privacy’ settings came in.  Checking these, you will find Wordpress has two options:

You can see my Privacy settings were set to ‘I would like to block search engines, but allow normal visitors’.  I’m not sure how this came to be set to this option, I may have set it whilst I was working on my website pages and felt it was n’t ready to publish or promote as yet, or it may have come as the default.  This is the part of the Wordpress program that inserts the META Robots Tags for NOINDEX and NOFOLLOW into your production HTML code which search engines will see.   Anyway, the solution was to simply select the top option, which says ‘I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archives’ and save the changes:

I then re-inserted the PHP funtion I had removed from my Header file, updated it and went back to my browser.  Reloaded my website, checked the source code and there were no META Robots Tags for NOINDEX or NOFOLLOW anymore, and boy was I glad I’d found the solution to my problem.  Because, now, my plug-ins for Post Views started working again, and I was ready to take a backup of all my Wordpress files and keep them safe, just incase anything went wrong again!!!

So in the end, the problem was not with the Wordpress THEME template files or even the function that brought in the statements into the source code, and that’s why it was difficult to locate and find.  With all the different plug-ins and widgets that we end up loading to make our blog pages user friendly and informative, it’s always best to go through all your SETTINGS for these widgets/plug-ins and of course, always go back and check your Dashboard default Wordpress settings.

It was a tough lesson, but I think an important one, if you never come across these META Robots tags, then you’ll be fine, nothing to worry about, but if you do, this is one of the solutions to the problem if you’re using a Wordpress theme.  I would welcome, any readers thoughts on this and if they’ve come across this problem themselves and how they resolve it.

Daniel Hunt

........................................................................................................................................................

Previous / Next Articles

« « Importance of META Robots Tag in Search Engine Optimisation

How to start your own Blog on an external host » »

........................................................................................................................................................

technicalarticles.co.uk features regular articles on my blogging journey of creating a unique web presence and sharing with my readers ideas and tips for helping them do the same. Keep up to date with the latest posts from Daniel Hunt - Join the Community by Subscribing.