Noindexing a WordPress page means telling search engine bots to exclude it from the search results and content analysis. That seems counterproductive since the goal is to publish as many high-quality pages and have the search engine index and rank them, enhancing the website’s search engine authority. While you shouldn’t publish low-quality content, sometimes pages should be public but unindexed. You can keep them available to visitors or only for user roles such as writers, editors, and administrators. We will explain how excluding web pages from SERPs (search engine results pages) works. Then, we show how to noindex a page in WordPress.
What is the noindex meta tag?
As the name suggests, the noindex tag is an HTML meta tag that tells crawlers and bots employed by search engines to not index the content. That means a noindexed page cannot appear in search results but may remain visible to you or on the front end. It will also not be used for discovering new content through URLs on it. Consequently, such a page will not affect your SEO (Search Engine Optimization) in any way, even if publicly available. That’s because no link equity from the source page to the destination will go through the link. You don’t have to worry about other pages on your site linking to it.
Search engines, especially Google, are advanced enough to immediately drop the page even if indexed pages point to it. However, we warn you that research shows a page might still be indexed if an indexed page on an external page links to it. The same applies if it was mentioned in a sitemap. Additionally, if that page was already indexed in the past, adding nofollow won’t remove it from search results unless you submit a manual request. It’s easy to implement, as it only requires website access, not root server access. Therefore, other sites are unaffected, and you don’t have to require help from a web provider or do any configurations.
Explanation of the disallow directive
In contrast, the disallow directive is a more radical approach. Instead of letting bots crawl the page, disallow blocks them from both crawling and indexing. However, it does not prevent the page from being indexed in other ways, such as through internal and external links pointing to it. Nofollow and disallow should not be combined. That’s because, if a page is blocked by disallow, crawlers can never see the “nofollow” tag and its instructions, much less respect it. Thus, your page may still appear in search results if you use disallow and nofollow.
1. Hide a WordPress page from search engines via the robots.txt file
We will start with the shortest method to block access to a page. If a page still appears in search results for reasons we mentioned, it will have a “No information is available for this page” message. You can follow the robots.txt editing via Yoast SEO plugin guide to do so. Using that particular plugin is beneficial for method 4, too. If you aren’t a user, you may access the site via FTP and find the robots.txt file in the root folder (may be named “public_html” or “www”). If it isn’t there, make it. Afterward, open the file and paste one of these codes:
User-agent: *
Disallow: /wordpress-page-name/
User-agent: * means that it will stop all search engines from accessing the page. You can also block specific crawlers. For example, to remove Google’s search engine from crawling, replace * with googlebot. Also, URL or domain name is unnecessary, only page name with a slash before and after.
2. Use a meta tag in HTML or PHP to noindex a page
Content on your website is displayed in your browser in an HTML format with the help of PHP, CSS, JavaScript, JSON, and so on. Thus, you can modify the HTML for the page you want to conceal from search engines and add a noindex tag this way:
- Open the page you wish to edit in HTML, whether via text editor or WordPress Admin Section’s editor.
- Locate thе
<header>tag at the top, between<html>and<body>and put a cursor to start typing. - Add the following line:
<meta name="robots" content="noindex">and save changes.
An example is this:
<!DOCTYPE html>
<html><head>
<meta name="robots" content="noindex" />
(…)
</head>
<body>(…)</body>
</html>
Like above, robots applies to all search engine bots. You can block a page from appearing in Google Search results by replacing robots with googlebot.
Using a PHP code snippet to noindex a page
While effective and recommended by Google, the method above can be a bit inconvenient. It is hard to remember which pages were noindexed after a while. Plus, some newcomers may be unable to post the code in the right place. An alternative is to use a PHP code in the functions.php file in your theme’s folder (root/wp-content/theme-name/). Open it, scroll to the bottom, and paste this code:
<?php
$url = $_SERVER['REQUEST_URI'];
if(stristr($url,"your-page-name.html")){
echo '<meta name="robots" content="noindex" />';
} ?>
3. Modify an HTTP header in WordPress to noindex a resource page
Another method that Google recommends is using an HTTP response header. It is applicable to all non-HTML files such as PDF files you embed on pages, or files in your Media Library such as images and videos. You can use the X-Robots-Tag to instruct engines not to index your page. Here’s how that would look:
HTTP/1.1 200 OK
(…)
X-Robots-Tag: noindex
(…)
Since users often want other engines to still index them, an example of multiple uses is to add another below. For example:
HTTP/1.1 200 OK
(…)
X-Robots-Tag: googlebot: noindex
X-Robots-Tag: anotherbot: noindex
(…)
4. Use a WordPress plugin to noindex a page
We customarily mention installing WordPress plugins to solve all kinds of problems. In this case, you likely have one installed already—Yoast SEO. If you do, you can noindex a page via Yoast SEO in WordPress this way:
- Open a page with Block Editor or Classic Editor.
- Find the Yoast SEO meta box at the bottom and select Settings (gears/cogwheel icon).
- Open the Advanced tab.
- Choose No under “Allow search engines to show this Page in search results”.
- Select the empty circle near No in front of the “Should search engines follow links on this page?” option.
- Publish the page if you haven’t, whether as Public or Private.
Note. Want to noindex tags, categories, content types, archives, breadcrumbs, media, and RSS feed pages? Go to SEO in the left sidebar → Pick an appropriate tab → Toggle Yes under Show X in search results?