Useful "Page not found" error pages

Posted on December 17, 2000

in IA/Usability

by Madhu Menon (themadman)

Rated 4.1 (Ratings: 20) (Add your rating)

Want more?

More articles in IA/Usability
More articles by themadman

Madhu Menon

Member info | Full bio

User since: February 06, 2000

Last login: October 27, 2008

Articles written: 3

Ever encountered a "404 error" message on a site? Also known as a "page not found" error, it can really annoy visitors. Some of these folks may never return to your site. If you don't handle these people with care, you could drive important traffic away from your site. Now, you really don't want to do that, do you?

Links go "missing" for a whole bunch of reasons:

You moved the page elsewhere. Instead of grouping "mutual funds" under "Investments", perhaps you moved it to "Personal Finance" (for example)
Some other site linked to a page on your site. Either they got the link wrong or the linked page no longer exists (tip: please� try not to re-architect your site unless absolutely necessary).
A search engine still maintains an old index of your site and hasn't updated it in years.
Somebody made a minor mistake typing the URL in e.g., product.htm instead of product.html

But that's no reason to allow visitors to leave disappointed! Every site must have a custom "404 page" that tries to help people find what they thought they'd find. Experienced web developers (of whom there are lots on the thelist) know that such a page is essential for a large, high-traffic site. Or at least they ought to. If that's the case, why do some of the most popular sites on the Internet not have one? Here are some examples...

... and still more well-known sites have practically useless 404 error pages. Some examples are:

http://www.usatoday.com/AnErrorPage.html (ugh! Stops you cold.)
http://www.aol.com/AnErrorPage.html (as if hitting the "reload" button on your browser will bring back a page that's been "renamed, moved, or deleted".)
http://www.fastcompany.com/AnErrorPage.html

Very few sites actually trap the URL causing the problem and allow people to enter it into a form without copying, pasting, pressing the Back button to check the referrering page, etc. Why is that so hard? Instead of asking questions like "Write to us, telling us what URL caused the problem, where you came from, what operating system and browser you're using", why not just sniff all this through a server-side script and use that info? See: http://www.ask.com/AnErrorPage.html as an annoying example.

Don't drive people away. Here are some ideas for designing more user-friendly error pages::

Don't redirect them automatically to the home page! This is evil!
Show a friendlier error message. Use a more informal tone. "Oops! We couldn't find this page for you. Try these options:" sounds a lot better than "Error. Missing page".
Offer a link to the site map, or better still, reproduce the site map on the error page. This will help those users that might have been just looking for a particular section, say, the "Services" page.
Offer a link to the site search. Better still, put a search form right there on the error page. It's good to try and minimize the number of clicks for a visitor.
Offer a link to a guided tour (if you have one) or the "About the site" page.
Trap the refererring page (Use the "HTTP_REFERER" server variable) and automatically send an email message to the Webmaster of the site, notifying him/her of i) The URL with the error and ii) The referring page
On the custom 404 page, have a contact form that the user can fill in if he/she was looking for something specific. For a particularly nice touch, pre-fill the "Missing page:" field with the URL of the page that caused the error, and the "Referring page:" field with the URL of page they came from.
Please reply promptly to all mail from users who got a 404 page. Don't stop at the autoresponder (you do have one, don't you?). Write back to them, tell them how to find what they were looking for, and do it within 24 hours of the error. Prompt customer service is so rare on the Web that it will really be appreciated by your visitors.

Using some or all of the above measures will spread that "they really care about me" feeling amongst your site's visitors, and we all know that trust is a scarce commodity on the Web.

Are there any decent 404 pages out there? Yes. Here are a few:

Fool.com - http://www.fool.com/AnErrorPage.html (Very nice. I like the informal, reassuring tone of this page)
PC Magazine - http://www.pcmag.com/AnErrorPage.html (Includes a site search, site map, etc.)
Martin Burns' web site: http://www.easyweb.co.uk/AnErrorPage.html (employs most of the techniques I've mentioned)

More resources

ASPToday has a detailed article about setting up custom 404 error pages on Windows NT. If you're on NT, read this.
ASP 101 has a good ASP script that traps the referring page and sends a mail to the Webmaster about the error. You could modify this to log it to a database instead.
As Martin points out below, Anthony has written an article about doing the same for Apache on Unix/Linux.
The 404 research lab is a site devoted to just 404 errors. Highly recommended. Includes examples of good 404 pages, whacky pages, instructional pages, etc.

Madhu Menon is a chef and restaurant owner based in Bangalore, India - the so-called "Silicon Valley of India". He has been online since 1994 when people used gopher more than the WWW. Madhu used to be a user experience consultant in times long gone but dumped his entire career in Web development to pursue the other passion in his life - cooking. He started an upmarket gourmet South-east Asian restaurant called Shiok in Bangalore where he serves food from Thailand, Malaysia, Singapore, and Indonesia.

In a past life, Madhu has been the Webmaster of several large sites, including the now defunct CNET India. He was last seen masquerading as the head of the user experience department at an Internet solutions company. When not doing anything useful, he can be found ranting on his weblog at http://madmanweb.com.

Oh, and he's also the Language Nazi admin at evolt.org

15 comments on this article. Log in to add your comment | Rate this article:

What about shared hosts?

Submitted by aardvark on December 18, 2000 - 11:27.

Some people are stuck on shared hosts (primarily NT) where the ISP won't allow you to have custom error pages. For example, I created this truly useless 404 page on my domain, but nobody will ever see it because the host won't map 404s to it.

So what do you do in those cases? Well, in my case, every page that ever existed on the site that would otherwise have been deleted has been replaced with a redirect to the 404 page or an appropriate content page. Directories that have directory browsing disabled (all of them) but no content (like the /images directory) have a default page that redirects the user as well. So even if your host doesn't allow custom 404s, you should still try to plan for the pages that are most likely to return 404 errors: directories and deleted or moved pages.

It's also worth noting that 404s aren't the only pages you should have custom pages for. There are many others you should customize, especially since you have a better idea what would return those errors.

Submitted by themadman on December 18, 2000 - 11:43.

Thanks for pointing that out, Adrian. Possibly because of my background managing (only) large sites, I was a little biased. The effect of 404 errors is especially bad for big sites and may lead to a direct revenue loss through lost traffic, and these sites typically have their own servers. If I were to set up a smaller web site though, I'd certainly ensure that the hosting company allows a custom 404 page. Most of the good ones seem to do so now. And you're right, other errors should have custom error pages too, but the 404 is by far the most commonly found. If you're a decent sysadmin, don't ever let people get an error like "Server busy". Keep the box tuned, mate.

404 and other tips

Submitted by socketboy on December 18, 2000 - 13:57.

Custom 404 pages of the types you mention, I believe are very important. Across my sites, I have every type you mention, including one which is less functional, but rather fun, which is a Lost on the Internet Postcard site, which you get when going to a url that doesn't exist at VanityURL.com, such as http://vanityURL.com/evolt/is/god.html I can't talk for NT (I prefer not to use it), but on Unix shared servers, you can often do the 404 mapping in your .htaccess files. Dreamhost.com allows me to do this. More importantly, there are a lot of things one can do in order to make sure that dead links don't happen, or once you discover them, that they do not continue to be a problem. This could range from doing redirects for old pages (which could range from pointer html pages, to symbolic links, or best of all, mod rewrite commands) which would allow you to automatically forward someone on to the new address. I use this a lot on The Firm List (www.firmlist.com) as it's grown and changed over time and there's all sorts of redirections to keep old bookmarks happy. There are also cgi scripts that will "sniff" for the referrer page so that when a 404 error page is generated based on a link within your own site (which can happen) that you know about it immediately. This is essential in a site with more than 4,000 pages like mine. In my experience, the only acceptable broken links should be the ones you can't help, like when someone misspells things. Otherwise, after one time (when you find out they exist) they should be fixed.

404 howtos

Submitted by MartinB on December 18, 2000 - 17:06.

While many evoltians will know how to do this, there are full instructions in the following places:

Anthony's article A Cheesy .htaccess Tutorial

The 404 Research Lab, along with a whole bunch of examples. This has instructions for IIS and Netscape servers as well as Apache.

Another example of a fun and nice 404

Submitted by pboersma on January 19, 2001 - 09:49.

Another example can be found at: http://www.ftrain.com/anErrorPage.html (of course I found it first ;-))

Good NT host

Submitted by jasonwhitman on March 6, 2001 - 15:54.

Great article. It's amazing how many people don't even know that you can create a custom 404 or other error page (I've educated teams at two jobs now).

I did want to mention the company I host with, SoftCom. They are all NT on shared servers but allow for a custom ASP or HTML page for your site. Very nice and not expensive.

Try W3C's

Submitted by Martin Tsachev on June 29, 2001 - 07:47.

W3C have a really good 404 page it automatically performs a search for a similar phrase and shows you the result with the error message. I hope other sites follow this practice.

Re: Try W3C's

Submitted by Jeff Howden on November 17, 2001 - 02:29.

shaggy,

did you know that evolt.org does this by default?

try going to http://evolt.org/javascript to see what i mean. if you're browser supports it you can even perform multi-word searches from the address bar. for example, using ie5+ type "http://evolt.org/javascript .jeff" in the address bar. it will automatically escape the address resulting in this link and search results: http://evolt.org/javascript%20jeff

enjoy,

.jeff

User reporting of 404s

Submitted by MartinB on November 21, 2001 - 10:42.

Why should a user have to report a 404 at all? Although offering the opportunity will give the user an opportunity to vent about it, and will give you a sense of which 404s are most annoying to users, you can trap and report errors serverside without the user having to intervene.

You can either:

Run regular reports on your error log (A Good ThingTM anyway)
Have an SSI exec call (or ASP or CF or whatever) in the page to send you an email notification. This can trap and report all the platform, requested page, referer etc info through environmental/server variables.

Yes, Do it Yourself!

Submitted by socketboy on November 21, 2001 - 11:14.

I agree with MartinB, it's no problem to have your system notify you automatically. I originally set up a .cgi script to be called when an error occured by simply having the .htaccess call that .cgi script for the 404 error page. The script would send me the request page, the referrer page and any other environmental information I wanted as well as redirecting the user to an error page with navigation choices.

Now I am transferring to a PHP system which does a similar thing, emailing any environmental variables I want when the page is called and I simply have the .htaccess file point to the appropriate 404 error page as I have different ones for different regions of the site so that the error page provides them with contextual navigation choices rather than just a generic set of choices.

RE: User reporting of 404s

Submitted by themadman on November 21, 2001 - 11:27.

This article was from a user's perspective, not tips for Webmasters :)
That's why I left out the recording and analysis part of it. That's something I assumed would be done.
(If not, that Webmaster doesn't deserve to have a job.)

I'm not saying that reporting 404s should be required (not that you could force it anyway); just that if a person wants to say, "My friend sent me a link to an article on [xyz], but it's not here. Where has it gone?", then that option would be a good idea. See step 6 in my article.
Before that, you should follow steps 1-5. They're not mutually exclusive.

Madhu

So go ahead and do it.

Submitted by persist1 on May 22, 2002 - 15:39.

In Apache, it's just one line in an .htaccess file:

ErrorDocument 404 /your_error_page.html

As the example implies, you can also provide a separate page for requests that return a 403 response code... also, the path you apply depends on your site setup.

On my own personal site, I:

Redirect for recently moved pages
Apply art direction that preserves section ID, so that the hapless visitor knows that they haven't messed up completely

And of course, I apply a sense of humor. People aren't quite so perturbed after they've had a chuckle.

Find refering page in PHP/Apache

Submitted by plush on January 16, 2003 - 22:56.

If you are trying to find the refering page (the one that caused the 404 error) with a PHP/Apache server you must use relative links in the .htaccess file.

  ErrorDocument 404 /your_error_page.html
     // This will return the correct URL of the page that caused the error

   ErrorDocument 404 http://www.your_domain.com/your_error_page.html
     // This will return the URL of the custom 404 page as the referer. No good!

-----

$_SERVER['REQUEST_URI'] will find the URL in PHP 4.1.0+.
$HTTP_SERVER_VARS['REQUEST_URI'] will find the URL in earlier versions.

Just send'em onward....??

Submitted by tdave365 on May 4, 2006 - 17:52.

Notice that one of the examples actually has the errant URL pre-filled in the feedback form. If that's possible (to grab the intended URL out of thin air to use in the form) it should be possible to do a re-direct to an equivalent page or an even friendlier custom page, based on the same. I think this is definetely doable in classic ASP.

I'm not talking about moving visitors on *in general*, which the article already points out is in bad taste; I'm talking about moving them on someplace relevant based on the attempted URL.

Doable, but...

Submitted by Marcel Feenstra on June 13, 2007 - 22:06.

Retrieving the URL that was entered by the visitor is certainly doable in classic ASP! However, I would be careful not to "assume" too much about the "intended" URL.

In some cases, it may be "obvious" what was meant (user typed "javascipt", must have meant "javascript") --you could use a list of "common typos" here.

However, if it is not completely clear what the user "must have meant", it would be better to display a few "candidates" (and/or a search form) than to redirect a user to the page you think is best/most relevant, IMO...

Start of page header

Other Fine Evolt.org Sites

Navigation Starts

Submit

Article Categories

Highest rated articles

Recent comments

Main Page Content