This article explores the different ways to prevent indexing of media files in Umbraco, both for local/own-hosted environments and Umbraco Cloud setups. It includes working code examples, links to official documentation, and insights from Umbraco support.

The goal

Prevent search engines from indexing documents (e.g. .pdf, .docx, .jpg) stored in the /media/ folder of your Umbraco site, while still allowing legitimate users to access them directly.

The recommended and most effective method is to add an HTTP response header:

X-Robots-Tag: noindex, nofollow

Google documentation: X-Robots-Tag

Option 1: Local / Own Hosting (Static File Middleware)

If you're hosting Umbraco yourself, you can intercept static file requests and apply the X-Robots-Tag header directly using middleware.

Code snippet (Program.cs):

app.UseStaticFiles(new StaticFileOptions
{
    OnPrepareResponse = ctx =>
    {
        var path = ctx.File.PhysicalPath;
        if (!string.IsNullOrEmpty(path) &&
            path.Contains("\\media\\", StringComparison.OrdinalIgnoreCase) &&
            (path.EndsWith(".pdf") || path.EndsWith(".docx") || path.EndsWith(".jpg")))
        {
            ctx.Context.Response.Headers["X-Robots-Tag"] = "noindex, nofollow";
        }
    }
});

Pros

• Easy to implement
• Works for all static files served from disk

Cons

• Doesn’t work in Umbraco Cloud (media served from Azure Blob Storage)

Option 2: Umbraco Cloud-Compatible Solution

Because Umbraco Cloud serves media via Azure Blob Storage, custom middleware or standard StaticFileOptions in Program.cs will not affect these files.

Instead, you should use Umbraco’s built-in pipeline with a registered IConfigureOptions<StaticFileOptions> class.

Code file: ConfigureMediaStaticFileOptions.cs

using System.IO;
using System;
using System.Collections.Generic;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Options;
using Microsoft.Net.Http.Headers;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.Hosting;

public class ConfigureMediaStaticFileOptions : IConfigureOptions<StaticFileOptions>
{
    private static readonly HashSet<string> _blockedExtensions = new(StringComparer.OrdinalIgnoreCase)
    {
        ".pdf", ".docx", ".xlsx", ".pptx", ".jpg", ".jpeg", ".png"
    };

    private readonly string _backOfficePath;

    public ConfigureMediaStaticFileOptions(IOptions<GlobalSettings> globalSettings, IHostingEnvironment hostingEnvironment)
    {
        _backOfficePath = globalSettings.Value.GetBackOfficePath(hostingEnvironment);
    }

    public void Configure(StaticFileOptions options)
    {
        options.OnPrepareResponse = ctx =>
        {
            var path = ctx.Context.Request.Path;
            var ext = Path.GetExtension(ctx.File.Name);

            if (path.StartsWithSegments(_backOfficePath)) return;

            if (_blockedExtensions.Contains(ext))
            {
                ctx.Context.Response.Headers["X-Robots-Tag"] = "noindex, nofollow";
            }
        };
    }
}

Register in Program.cs:

builder.Services.AddTransient<IConfigureOptions<StaticFileOptions>>, ConfigureMediaStaticFileOptions>();

Official docs

• Custom response headers for static files
• Umbraco PR #11783

Pros

• Works correctly in Umbraco Cloud
• Applies to files stored in Blob Storage

Cons

• Slightly more advanced setup

Bonus: Serving truly private files

If you need to restrict access entirely, or serve files with permissions and logging, don’t use /media/ at all. Instead, store files in a protected folder and stream them via a controller:

[Route("private-docs/{filename}")]
public class PrivateDocsController : Controller
{
    [HttpGet]
    public IActionResult GetFile(string filename)
    {
        var path = Path.Combine("wwwroot", "private-docs", filename);
        if (!System.IO.File.Exists(path)) return NotFound();

        Response.Headers["X-Robots-Tag"] = "noindex, nofollow";
        var mime = MimeMapping.MimeUtility.GetMimeMapping(filename);
        return PhysicalFile(path, mime);
    }
}

Summary

For sensitive files, combine this with:

• robots.txt exclusions
• Password protection or user-based gating
• Secure file storage outside /media/

Final thought

The X-Robots-Tag header is a powerful but subtle tool. When implemented correctly, especially in Umbraco Cloud, it allows you to protect your clients from accidental leakage of internal documentation while keeping their sites SEO-optimised.

If you're unsure which approach is right, start with the static header injection via IConfigureOptions, then move to controller-based delivery for more control.

Need help with implementation or auditing your current setup? Get in touch.