azure-storage-net: Why can't I add this HTTP header to my blobs?

I just learned that Googlebot-Image has been indexing images at my site, presumably because a robots.txt entry is only honored for LOCAL images … those served from the site’s domain. Images that are in Azure Storage (served by the Azure CDN) are on a different domain, thus Google indexes the images.

To stop this, Googlebot-Image needs to see this header on the blobs:

X-Robots-Tag: noindex

However, when I try to add this header (as a user-defined header) in CloudBerry Explorer for Azure Blob Storage, the header cannot be added.

So, what’s up with this? Are you preventing me from adding the header? If so, why? If there is no good reason to stop me from adding the header, can you change your policy and allow its use? … or do you have any other ideas?

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 16 (3 by maintainers)

Most upvoted comments

@GuardRex don’t forget items in the root directory of storage are served from a container called $root which you’ll have to create.

Google Image bot does respect robots.txt https://support.google.com/webmasters/answer/35308?hl=en