How to Stop a WordPress Staging Site Being Indexed

Headline: How to stop a WordPress staging site being indexed. A crawler visiting a grid of pages that includes a staging copy.

Short answer: use a password if your tools allow it. If they do not, use noindex and leave robots.txt open so Google can read it. Then add a check after every push, because a staging setting that reaches production can hide your live site.

staging and liveGooglebota staging copy next to the live site
Illustration, not a screenshot.

If Google has indexed your WordPress staging site, your worry is justified: anyone can find those pages in search. I found this on a WordPress plugin site where a staging host was public and indexed, with 406 impressions in 90 days and about 10% of the crawl.

noindex means do not list this page. Disallow in robots.txt means do not fetch this page.

Key takeaways

  • Find out whether staging is indexed and how much crawl it takes before you change anything.
  • Use a password if your tools allow it. If not, set noindex and confirm it on a live response.
  • Leave robots.txt open until the pages drop out, and consider a Removals request for speed.
  • Check the production robots meta tag and robots.txt after every push.
  • Remember every install on the staging host, including a blog installed in a folder.

What you’ll need

  • Search Console access to the staging host
  • A way to view page source

Check whether you have a problem

  • Search Console: filter the performance report by the staging host’s URLs. Any impressions mean it is indexed.
  • Crawl Stats: look at the share of requests that go to the staging host.
  • View the page source on staging and look for index, follow or no robots tag at all.
  • Check whether staging has canonical tags pointing at itself. That tells Google it is a page in its own right.

The three options

OptionWhat it doesWhen to use it
Password (HTTP auth)Nobody, including Google, can see the pagesBest default, if your tools can get past it
noindexGoogle can see the page but is told not to list itWhen tools or colleagues need open access
robots.txt DisallowAsks crawlers not to request pagesNot as a fix for pages already indexed

Why a Disallow in robots.txt backfires

Google has to fetch a page to see a noindex tag on it. If robots.txt blocks the page, Google never sees the noindex. URLs that are already known can stay in the index, usually with no description.

So for a staging site that is already indexed, the order matters: noindex first, wait for the pages to drop out, and only then consider a block.

Animation comparing two options. With noindex, Googlebot fetches the page, reads the tag and drops the page. With a Disallow in robots.txt, Googlebot never fetches the page, never sees the noindex and the URL can linger in the index. The order is noindex first, wait for the pages to drop out, then consider a block.
Google has to fetch a page to see a noindex tag, which is why a Disallow can backfire.

Check what your robots.txt actually returns after you turn on a “discourage search engines” setting. Some setups add a blanket Disallow: / along with the noindex. If yours does, you have the problem above.

What I did when a password was not possible

The team’s tooling needed access to staging, so I could not add a login. I used the CMS setting that asks search engines not to index the site, then checked a live response for noindex, nofollow. I did not touch robots.txt.

Do not trust the toggle. Fetch the page and read the tag.

Getting the pages out faster

noindex works as Google recrawls, which can take a while. You can also file a request in the Search Console Removals tool. Treat that as temporary hiding while noindex does the permanent work.

The push that ruins production

This is the part that is easy to miss. If you push from staging to production in a way that copies the site’s options, the “discourage search engines” setting can go with it. Production then serves noindex, and nobody notices until traffic drops.

Add a check after every push:

  1. Fetch the production home page and read the robots meta tag. It should not contain noindex.
  2. Fetch production robots.txt and confirm it has no Disallow: /.
  3. Check one important URL in the Search Console URL Inspection tool if anything looks wrong.

It takes a minute. Write it into the deploy steps so it does not depend on remembering.

Animation of a push from staging to production that copies the site's options, including the discourage search engines setting, so production starts serving noindex. A three-step check after every push catches it: production robots meta should not contain noindex, robots.txt has no Disallow slash, and URL Inspection on one important URL if anything looks wrong.
A staging setting can ride along with a push. A short check afterwards catches it.

Frequently Asked Questions About Staging Site Indexing

A password is the strongest option if your tools can get past it. If they cannot, set noindex and confirm it in a live response. The table of the three options above shows which fits.

How do I stop a staging site from being indexed by Google?

Google can only obey a noindex it is able to read. A Disallow stops it fetching the page, so URLs it already knows can linger in results. Put noindex in place first and block later, if at all.

Why shouldn’t I block an indexed staging site in robots.txt?

Set noindex and wait for Google to recrawl. For something quicker, the Search Console Removals tool can hide the pages for a while, as the section on getting pages out faster describes.

How do I get staging pages out of Google faster?

Yes. With HTTP authentication nobody, Google included, can see the pages, which is why it is the best default. The catch is that your own tools and colleagues need access too. If they cannot work behind a login, use noindex.

Does a password protect a staging site from Google?

Unlock Growth Secrets!

Get actionable insights on SEO, content strategy, email marketing, and more, straight to your inbox.

We don’t spam! Read our privacy policy for more info.

Leave a Reply

Your email address will not be published. Required fields are marked *

Text copied to clipboard with attribution!