A Complete Guide to Configuring Friendly Error Pages in Sitecore – Part 1 – 404 Pages

July 27, 2017

Blog | Development | A Complete Guide to Configuring Friendly Error Pages in Sitecore – Part 1 – 404 Pages

Setting up friendly error pages is a subject that comes up often within the Sitecore community and among our internal GeekHive developers. As this Stack Exchange question illustrates, there are many takes on the subject. In my opinion, none of the proposed answers are as complete as they should be.  This post is part one of a two part series intended to serve as a complete guide to setting up error pages within Sitecore. It is considered the bare minimum required to ensure primary use-cases are met.  In part one, I will cover the steps required to configure friendly 404 error pages properly within Sitecore.

The code from this post has been packaged up as a NuGet package.

Add 404 Item to Site

The first step is to add a new 404 item beneath all defined sites. The path would resemble something such as /sitecore/content/Site1/home/404. Each site should have a unique 404 page defined beneath its root item. The 404 item requires a defined layout so that when this page is set as the Context.Item, it renders properly.

Update Settings

With the item in place, the next step is to update Sitecore settings to call upon our page when needed. An example patch is shown below:

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <settings>
      <setting name="LayoutNotFoundUrl">
        <patch:attribute name="value">/404</patch:attribute>
      </setting>
      <setting name="ItemNotFoundUrl">
        <patch:attribute name="value">/404</patch:attribute>
      </setting>
    </settings>
  </sitecore>
</configuration>

Note that a LayoutNotFoundUrl response will retain the URL and ultimately return a 404 status code. On any Sitecore site, you can quickly test the LayoutNotFoundUrl setting by visiting http://mysite.com/sitecore/content. This is because the ItemResolver processor checks if the LocalPath directly resolves to a Sitecore path. This means you can access any item by its full path in the URL.

Transfer Bad Requests to Friendly 404 Page

With the two previous settings defined, Sitecore will automatically redirect the user to the 404 page. This is not ideal, however, since the server will be issuing a 302 (temporary redirect) first, then a 404 status code. To solve this dilemma, we need to patch an existing Sitecore setting:

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <settings>
      <setting name="RequestErrors.UseServerSideRedirect">
        <patch:attribute name="value">true</patch:attribute>
      </setting>
    </settings>
  </sitecore>
</configuration>

Next, we need to read this config value as part of the Sitecore.Pipelines.HttpRequest.ExecuteRequest processor. To do this, we create a new class that inherits the ExecuteRequest class and updates the PerformRedirect method:

public class CustomExecuteRequest : global::Sitecore.Pipelines.HttpRequest.ExecuteRequest
{
    private readonly BaseLinkManager _baseLinkManager;

    public CustomExecuteRequest(BaseSiteManager baseSiteManager, BaseItemManager baseItemManager, BaseLinkManager baseLinkManager) : base(baseSiteManager, baseItemManager)
    {
        _baseLinkManager = baseLinkManager;
    }
        
    protected override void PerformRedirect(string url)
    {
        if (Context.Site == null || Context.Database == null || Context.Database.Name == "core")
        {
            _404Logger.Log.Info("Attempting to redirect url {0}, but no Context Site or DB defined (or core db redirect attempted)".Fmt(url));
            return;
        }

        // need to retrieve not found item to account for sites utilizing virtualFolder attribute
        var notFoundItem = Context.Database.GetItem(Context.Site.StartPath + Settings.ItemNotFoundUrl);

        if (notFoundItem == null)
        {
            _404Logger.Log.Info("No 404 item found on site: {0}".Fmt(Context.Site.Name));
            return;
        }

        var notFoundUrl = _baseLinkManager.GetItemUrl(notFoundItem);

        if (string.IsNullOrWhiteSpace(notFoundUrl))
        {
            _404Logger.Log.Info("Found 404 item for site, but no URL returned: {0}".Fmt(Context.Site.Name));
            return;
        }

        _404Logger.Log.Info("Redirecting to {0}".Fmt(notFoundUrl));
        if (Settings.RequestErrors.UseServerSideRedirect)
        {
            HttpContext.Current.Server.TransferRequest(notFoundUrl);
        }
        else
            WebUtil.Redirect(notFoundUrl, false);
    }
}

This code performs a TransferRequest which retains the incoming URL properly, yet executes the 404 page behind the scenes. We then patch in our update:

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <httpRequestBegin>
        <!-- Reads updated "RequestErrors.UseServerSideRedirect" value and transfers request to LayoutNoutFoundUrl or ItemNotFoundUrl, preserving requested URL -->
        <processor type="MyDll.CustomExecuteRequest, MyDll" resolve="true" patch:instead="*[@type='Sitecore.Pipelines.HttpRequest.ExecuteRequest, Sitecore.Kernel']"/>
      </httpRequestBegin>
    </pipelines>
  </sitecore>
</configuration>

Return Proper HTTP Status Code

With all previous updates, most requests will serve up the friendly 404 page and retain the URL that was requested. Next, we need to modify the status code returned to the browser. The HTTP Status Code for the response must be altered from 200 (success) to 404 (page not found). This ensures search engine crawlers properly index 404 errors at a given URL as an improper URL. The following processor is added to the httpRequestEnd pipeline immediately after the EndDiagnostics processor. This processor checks if the current URL is equal to the value set in ItemNotFoundUrl. If so, it changes the response from 200 to 404 as outlined below. Note, this example inherits an HttpRequestBase class, also shown below.

public class Set404StatusCode : HttpRequestBase
{
    protected override void Execute(HttpRequestArgs args)
    {
        // retain 500 response if previously set
        if (HttpContext.Current.Response.StatusCode >= 500 || args.Context.Request.RawUrl == "/")
            return;

        // return if request does not end with value set in ItemNotFoundUrl, i.e. successful page
        if (!args.Context.Request.Url.LocalPath.EndsWith(Settings.ItemNotFoundUrl, StringComparison.InvariantCultureIgnoreCase))
            return;

        _404Logger.Log.Warn("Page Not Found: " + args.Context.Request.RawUrl + ", current status: " + HttpContext.Current.Response.StatusCode);
        HttpContext.Current.Response.TrySkipIisCustomErrors = true;
        HttpContext.Current.Response.StatusCode = (int)HttpStatusCode.NotFound;
        HttpContext.Current.Response.StatusDescription = "Page not found";
    }
}
public abstract class HttpRequestBase : HttpRequestProcessor
{
	/// <summary>
	/// allowedSites and disallowedSites are mutually exclusive, use one or the other
	/// in the event both are used, disallowedSites is enforced
	/// </summary>
	public string allowedSites { get; set; }
	public string disallowedSites { get; set; }
	public string ignoredPaths { get; set; }
	public string ignoredModes { get; set; }
	/// <summary>
	/// allowedDatabases and disallowedDatabases are mutually exclusive, use one or the other
	/// in the event both are used, disallowedDatabases is enforced
	/// </summary>
	public string allowedDatabases { get; set; }
	public string disallowedDatabases { get; set; }

	private const string EditMode = "Edit";

	/// <summary>
	/// Overridden HttpRequestProcessor method
	/// </summary>
	/// <param name="args"></param>
	public override void Process(HttpRequestArgs args)
	{
		if (IsValid(args))
		{
			Execute(args);
		}
	}

	protected abstract void Execute(HttpRequestArgs args);

	protected virtual bool IsValid(HttpRequestArgs hArgs)
	{
		return SitesAllowed()
			&& PathNotIgnored(hArgs)
			&& ModeNotIgnored()
			&& DatabaseAllowed();
	}

	private bool SitesAllowed()
	{
		// httpRequest processors should never run without a context site
		if (Context.Site == null)
			return false;

		var contextSiteName = Context.GetSiteName();

		if (string.IsNullOrWhiteSpace(contextSiteName))
			return false;

		// disallow checked first to trump an allowance
		if (!string.IsNullOrWhiteSpace(disallowedSites))
		{
			return !disallowedSites
				.Split(',')
				.Select(i => i.Trim())
				.Any(siteName => string.Equals(siteName, contextSiteName, StringComparison.CurrentCultureIgnoreCase));
		}

		if (!string.IsNullOrWhiteSpace(allowedSites))
		{
			return allowedSites
				.Split(',')
				.Select(i => i.Trim())
				.Any(siteName => string.Equals(siteName, contextSiteName, StringComparison.CurrentCultureIgnoreCase));
		}
		
		return true;
	}

	private bool PathNotIgnored(HttpRequestArgs hArgs)
	{
		if (string.IsNullOrWhiteSpace(ignoredPaths))
			return true;
		
		var ignoredPath = ignoredPaths
			.Split(',')
			.Select(i => i.Trim())
			.Any(path => hArgs.Context.Request.RawUrl.StartsWith(path, StringComparison.CurrentCultureIgnoreCase));

		return !ignoredPath;
	}

	private bool ModeNotIgnored()
	{
		if (string.IsNullOrWhiteSpace(ignoredModes))
			return true;

		var modes = ignoredModes.Split(',').Select(i => i.Trim());

		var isEditor = Context.PageMode.IsExperienceEditor;

		return !modes.Any(mode =>
		  (mode == "Edit" && isEditor) ||
		  (mode == "Preview" && Context.PageMode.IsPreview)
		);
	}

	private bool DatabaseAllowed()
	{
		// httpRequest processors should never run without a context database
		if (Context.Database == null)
			return false;
		
		var contextDatabaseName = Context.Database.Name;
		
		// disallow checked first to trump an allowance
		if (!string.IsNullOrWhiteSpace(disallowedDatabases))
		{
			return !disallowedDatabases
				.Split(',')
				.Select(i => i.Trim())
				.Any(database => string.Equals(database, contextDatabaseName, StringComparison.CurrentCultureIgnoreCase));
		}

		if (!string.IsNullOrWhiteSpace(allowedDatabases))
		{
			return allowedDatabases
				.Split(',')
				.Select(i => i.Trim())
				.Any(database => string.Equals(database, contextDatabaseName, StringComparison.CurrentCultureIgnoreCase));
		}
		
		return true;
	}
}

Patch Config

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <httpRequestEnd>
        <!-- Sets a 404 status code on the response -->
        <processor type="MyDll.Set404StatusCode, MyDll" patch:after="*[@type='Sitecore.Pipelines.HttpRequest.EndDiagnostics, Sitecore.Kernel']">
          <disallowedDatabases>core</disallowedDatabases>
          <disallowedSites>shell</disallowedSites>
        </processor>
      </httpRequestEnd>
    </pipelines>
  </sitecore>
</configuration>

Notice that if the incoming request has already set a 500 status code, the response is not altered. This will be covered in part two of this series. It also only sets the status to 404 if the requested URL matches the ItemNotFoundUrl setting.

Log URLs to Fine-Tune Detection

With all previous steps in place, Sitecore will begin serving up the friendly 404 page properly when a URL is requested and no matching page is found. This is a great starting point, but to ensure we aren’t introducing additional errors into our instance of Sitecore, we need to log all not-found requests to ensure we aren’t causing any native functionality to fail. To do so, we add a custom log file and write a warning on all not-found requests.

Patch Config

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <log4net>
      <appender name="404Appender" type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
        <file value="$(dataFolder)/logs/404.log.{date}.txt"/>
        <appendToFile value="true"/>
        <layout type="log4net.Layout.PatternLayout">
          <conversionPattern value="%4t %d{ABSOLUTE} %-5p %m%n"/>
        </layout>
        <encoding value="utf-8"/>
      </appender>
      <logger name="CustomErrors._404Logger" additivity="false">
        <level value="INFO"/>
        <appender-ref ref="404Appender"/>
      </logger>
    </log4net>
  </sitecore>
</configuration>

Code

public static class _404Logger
{
    public static ILog Log => LogManager.GetLogger("CustomErrors._404Logger") ?? LoggerFactory.GetLogger(typeof(_404Logger));
}

Log Warning in Set404StatusCode.cs

...
        _404Logger.Log.Warn("Page Not Found: " + args.Context.Request.RawUrl);
        HttpContext.Current.Response.TrySkipIisCustomErrors = true;
        HttpContext.Current.Response.StatusCode = (int)HttpStatusCode.NotFound;
        HttpContext.Current.Response.StatusDescription = "Page not found";
    }
...

Failed Media Requests Return Friendly 404 Error Page

Even with all of the previous updates, a failed media request will still give an improper response.  This is because media requests do not execute the same httpRequestBegin pipeline as standard requests. By default, failed requests will redirect to the ItemNotFoundUrl setting, which performs a 302 redirect instead of a 404. To ensure the URL is retained, we need to replace the existing MediaRequestHandler handler located in the Web.config. The below code was adapted from this article. The replacement code is below:

public class MediaRequestHandler : global::Sitecore.Resources.Media.MediaRequestHandler
{
    protected override bool DoProcessRequest(HttpContext context)
    {
        Assert.ArgumentNotNull(context, "context");

        var request = MediaManager.ParseMediaRequest(context.Request);

        if (request == null)
            return false;

        var media = MediaManager.GetMedia(request.MediaUri);

        if (media != null)
            return DoProcessRequest(context, request, media);

        using (new SecurityDisabler())
            media = MediaManager.GetMedia(request.MediaUri);

        string str;

        if (media == null)
        {
            str = Settings.ItemNotFoundUrl;
        }
        else
        {
            Assert.IsNotNull(Context.Site, "site");
            str = Context.Site.LoginPage != string.Empty ? Context.Site.LoginPage : Settings.NoAccessUrl;
        }
        if (Settings.RequestErrors.UseServerSideRedirect)
            HttpContext.Current.Server.TransferRequest(str);
        else
            HttpContext.Current.Response.Redirect(str);
        return true;
    }
}

Changes in web.config:

      <!--
      Commented original MediaRequestHandler to account for 404
      <add verb="*" path="sitecore_media.ashx" type="Sitecore.Resources.Media.MediaRequestHandler, Sitecore.Kernel" name="Sitecore.MediaRequestHandler"/>-->
      <add verb="*" path="sitecore_media.ashx" type="MyDll.MediaRequestHandler, MyDll" name="Sitecore.MediaRequestHandler"/>

The above code executes the native code, and if it fails to return a media item, it then checks if the item is blocked via permissions or if it does not exist. In the event it does not exist, it performs a TransferRequest to the 404 URL.

Handling Requests for Improper Extensions

Even with all previous modifications, a request made with an extension not understood by IIS will still return the default IIS 404 page. For example, http://mysite.com/test.asdfjkl will return the default IIS 404 page. To account for this, we need to add an httpErrors section to the web.config.

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <system.webServer>
    <httpErrors errorMode="DetailedLocalOnly" existingResponse="Auto" xdt:Transform="Insert" >
      <remove statusCode="404" subStatusCode="-1" />
      <error statusCode="404" path="404.html" responseMode="File" />
    </httpErrors>
  </system.webServer>
</configuration>

Note, the above patch is done via a config transformation as opposed to directly modifying the web.config. It is extremely important that this use-case refers to a static 404 page. Do not set this to the same /404 page as configured previously as it will cause an infinite redirect. The server will take care of setting the status code to 404 for us. The 404.html document in the web root is a valid HTML page. While this page cannot be site-specific (in the case of a multi-site environment) it is an extremely rare use-case.

Who would have thought things would get so complex?

Every implementation is different, which is likely why Sitecore does not provide this functionality out-of-the-box. As exhausting as this post is, everything included is essential to properly handle all 404 use-cases. Every piece plays a specific function.

Stay tuned for Part 2, where I outline the steps needed to address 500 pages. This use-case is much simpler than that of the friendly 404, but has similar “gotchas” to look out for.

Read Part 2 – 500 Pages- here.

 

John Rappel

Technical Lead
Tags
  • 404
  • Best Practices
  • Sitecore
subscribe to GeekHive's newsletter

Never miss out on a Sitecore update again!

Our newsletter features a blog roundup of our top posts so you can stay up to date with industry trends, tutorials, and best practices.

Recent Work

Check out what else we've been working on