Here are some specific htaccess examples taken mostly from my WordPress Password Protection plugin, which does alot more than password protection as you will see from the following mod_rewrite examples. These are a few of the mod_rewrite uses that BlogSecurity declared pushed the boundaries of Mod_Rewrite! Some of these snippets are quite exotic and unlike anything you may have seen before, also only for those who understand them as they can kill a website pretty quick.

Directory Protection

Enable the DirectoryIndex Protection, preventing directory index listings and defaulting. [Disable]

Options -Indexes
DirectoryIndex index.html index.php /index.php

Password Protect wp-login.php

Requires a valid user/pass to access the login page[401]

<Files wp-login.php>
Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user
</Files>

Password Protect wp-admin

Requires a valid user/pass to access any non-static (css, js, images) file in this directory.[401]

Options -ExecCGI -Indexes +FollowSymLinks -Includes
DirectoryIndex index.php /index.php
Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$">
Allow from All
</FilesMatch>
<FilesMatch "(async-upload)\.php$">
<IfModule mod_security.c>
SecFilterEngine Off
</IfModule>
Allow from All
</FilesMatch>

Protect wp-content

Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [401]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-content/.*$ [NC]
RewriteCond %{REQUEST_FILENAME} !^.+flexible-upload-wp25js.php$
RewriteCond %{REQUEST_FILENAME} ^.+\.(php|html|htm|txt)$
RewriteRule .* - [F,NS,L]

Protect wp-includes

Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-includes/.*$ [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /wp-includes/js/.+/.+\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} ^.+\.php$
RewriteRule .* - [F,NS,L]

Common Exploits

Block common exploit requests with 403 Forbidden. These can help alot, may break some plugins. [403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ ///.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\=?(http|ftp|ssl|https):/.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\?.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(asp|ini|dll).*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(htpasswd|htaccess|aahtpasswd).*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

Stop Hotlinking

Denies any request for static files (images, css, etc) if referrer is not local site or empty. [403]

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_REFERER} !^http://www.askapache.com.*$ [NC]
RewriteRule \.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$ - [F,NS,L]

Safe Request Methods

Denies any request not using GET,PROPFIND,POST,OPTIONS,PUT,HEAD[403]

RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|POST|PROPFIND|OPTIONS|PUT)$ [NC]
RewriteRule .* - [F,NS,L]

Forbid Proxies

Denies any POST Request using a Proxy Server. Can still access site, but not comment. See Perishable Press [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:VIA}%{HTTP:FORWARDED}%{HTTP:USERAGENT_VIA}%{HTTP:X_FORWARDED_FOR}%{HTTP:PROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION}%{HTTP:HTTP_PC_REMOTE_ADDR}%{HTTP:HTTP_CLIENT_IP} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

Real wp-comments-post.php

Denies any POST attempt made to a non-existing wp-comments-post.php[403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

HTTP PROTOCOL

Denies any badly formed HTTP PROTOCOL in the request, 0.9, 1.0, and 1.1 only[403]

RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ .+\ HTTP/(0\.9|1\.0|1\.1) [NC]
RewriteRule .* - [F,NS,L]

SPECIFY CHARACTERS

Denies any request for a url containing characters other than "a-zA-Z0-9.+/-?=&" - REALLY helps but may break your site depending on your links. [403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ [a-zA-Z0-9\.\+_/\-\?\=\&]+\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

BAD Content Length

Denies any POST request that doesnt have a Content-Length Header[403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Length} ^$
RewriteCond %{REQUEST_URI} !^/(wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

BAD Content Type

Denies any POST request with a content type other than application/x-www-form-urlencoded|multipart/form-data[403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Type} !^(application/x-www-form-urlencoded|multipart/form-data.*(boundary.*)?)$ [NC]
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

Missing HTTP_HOST

Denies requests that dont contain a HTTP HOST Header.[403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_HOST} ^$
RewriteRule .* - [F,NS,L]

Bogus Graphics Exploit

Denies obvious exploit using bogus graphics[403]

RewriteCond %{HTTP:Content-Disposition} \.php [NC]
RewriteCond %{HTTP:Content-Type} image/.+ [NC]
RewriteRule .* - [F,NS,L]

No UserAgent, Not POST

Denies POST requests by blank user-agents. May prevent a small number of visitors from POSTING. [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

No Referer, No Comment

Denies any comment attempt with a blank HTTP_REFERER field, highly indicative of spam. May prevent some visitors from POSTING. [403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteCond %{HTTP_REFERER} ^-?$
RewriteRule .* - [F,NS,L]

Trackback Spam

Denies obvious trackback spam. See Holy Shmoly! [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^.*(opera|mozilla|firefox|msie|safari).*$ [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.+/trackback/?\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

Map all URIs except those corresponding to existing files to a handler

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteRule . /script.php

Map any request to a handler

In the case where all URIs should be sent to the same place (including potentially requests for static content) the method to use depends on the type of the handler. For php scripts, use: For other handlers such as php scripts, use:

RewriteEngine On
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php

And for CGI scripts:

ScriptAliasMatch .* /var/www/script.cgi

Map URIs corresponding to existing files to a handler instead

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php

If the existing files you wish to have handled by your script have a common set of file extensions distinct from that of the hander, you can bypass mod_rewrite and use instead mod_actions. Let's say you want all .html and .tpl files to be dealt with by your script:

Action foo-action /script.php
AddHandler foo-action html tpl

Deny access if var=val contains the string foo.

RewriteCond %{QUERY_STRING} foo
RewriteRule ^/url - [F]

Removing the Query String

RewriteRule ^/url /url?

Adding to the Query String

Keep the existing query string using the Query String Append flag, but add var=val to the end.

RewriteRule ^/url /url?var=val [QSA]

Rewriting For Certain Query Strings

Rewrite URLs like http://askapache.com/url1?var=val to http://askapache.com/url2?var=val but don't rewrite if val isn't present.

RewriteCond %{QUERY_STRING} val
RewriteRule ^/url1 /url2

Modifying the Query String

Change any single instance of val in the query string to other_val when accessing /path. Note that %1 and %2 are back-references to the matched part of the regular expression in the previous RewriteCond.

RewriteCond %{QUERY_STRING} ^(.*)val(.*)$
RewriteRule /path /path?%1other_val%2

Mod_Rewrite Security originally appeared on AskApache.com

PirateBay SOPA Press Release

INTERNETS, 18th of January 2012.

PRESS RELEASE, FOR IMMEDIATE RELEASE

Over a century ago Thomas Edison got the patent for a device which would "do for the eye what the phonograph does for the ear". He called it the Kinetoscope. He was not only amongst the first to record video, he was also the first person to own the copyright to a motion picture.

Because of Edisons patents for the motion pictures it was close to financially impossible to create motion pictures in the North american east coast. The movie studios therefor relocated to California, and founded what we today call Hollywood. The reason was mostly because there was no patent. There was also no copyright to speak of, so the studios could copy old stories and make movies out of them - like Fantasia, one of Disneys biggest hits ever.

So, the whole basis of this industry, that today is screaming about losing control over immaterial rights, is that they circumvented immaterial rights. They copied (or put in their terminology: "stole") other peoples creative works, without paying for it. They did it in order to make a huge profit. Today, they're all successful and most of the studios are on the Fortune 500 list of the richest companies in the world. Congratulations - it's all based on being able to re-use other peoples creative works. And today they hold the rights to what other people create. If you want to get something released, you have to abide to their rules. The ones they created after circumventing other peoples rules.

The reason they are always complainting about "pirates" today is simple. We've done what they did. We circumvented the rules they created and created our own. We crushed their monopoly by giving people something more efficient. We allow people to have direct communication between eachother, circumventing the profitable middle man, that in some cases take over 107% of the profits (yes, you pay to work for them). It's all based on the fact that we're competition. We've proven that their existance in their current form is no longer needed. We're just better than they are.

And the funny part is that our rules are very similar to the founding ideas of the USA. We fight for freedom of speech. We see all people as equal. We believe that the public, not the elite, should rule the nation. We believe that laws should be created to serve the public, not the rich corporations.

The Pirate Bay is truly an international community. The team is spread all over the globe - but we've stayed out of the USA. We have Swedish roots and a swedish friend said this: The word SOPA means "trash" in Swedish. The word PIPA means "a pipe" in Swedish. This is of course not a coincidence. They want to make the internet inte a one way pipe, with them at the top, shoving trash through the pipe down to the rest of us obedient consumers. The public opinion on this matter is clear. Ask anyone on the street and you'll learn that noone wants to be fed with trash. Why the US government want the american people to be fed with trash is beyond our imagination but we hope that you will stop them, before we all drown.

SOPA can't do anything to stop TPB. Worst case we'll change top level domain from our current .org to one of the hundreds of other names that we already also use. In countries where TPB is blocked, China and Saudi Arabia springs to mind, they block hundreds of our domain names. And did it work? Not really. To fix the "problem of piracy" one should go to the source of the problem. The entertainment industry say they're creating "culture" but what they really do is stuff like selling overpriced plushy dolls and making 11 year old girls become anorexic. Either from working in the factories that creates the dolls for basically no salary or by watching movies and tv shows that make them think that they're fat.

In the great Sid Meiers computer game Civilization you can build Wonders of the world. One of the most powerful ones is Hollywood. With that you control all culture and media in the world. Rupert Murdoch was happy with MySpace and had no problems with their own piracy until it failed. Now he's complainting that Google is the biggest source of piracy in the world - because he's jealous. He wants to retain his mind control over people and clearly you'd get a more honest view of things on Wikipedia and Google than on Fox News.

Some facts (years, dates) are probably wrong in this press release. The reason is that we can't access this information when Wikipedia is blacked out. Because of pressure from our failing competitors. We're sorry for that.

THE PIRATE BAY, (K)2012

Anonymous Hackers SOPA Statement

Bright, and Clear: The Future of Free Speech

A rallying cry on the occassion of the Web's first mass blackout

As we watch the web go dark today in protest against the SOPA/PIPA censorship bills, let's take a moment and reflect on why this fight is so important. We may have learned that free speech is what makes America great, or instinctively resist attempts at silencing our voices. But these are abstract principles, divorced from the real world and our daily lives.

Free speech is the foundation of a free society. We can have the vote all we want. We can donate money wherever we want. But unless we're able to talk to each other and figure out collectively _what_ we want, those things don't matter.

We believe a healthy society doesn't allow its artists, musicians and other creators to starve. The copyright industry has been justly criticized for abusing the political process in a desperate attempt to maintain its role as a cultural gatekeeper, a business model made obsolete by a digital age of free copies. But the RIAA, MPAA & IFPI deserve our opprobrium for making enormous profits while often leaving the very artists it claims to represent *poorer* than they would be as independents.[1] While the public may have greater access to the few artists deemed sufficiently marketable to gain mass media promotion, fewer and fewer of us are making art and music in our own lives.

It's time we make a stand for a better world - not merely take rearguard actions to preserve a status quo that is _already_ failing us. Accordingly, we present the following list of demands:

• We call on national legislatures to not only reject ACTA efforts to globalize the American intellectual property regime, but to abolish the WIPO.
• We demand the elimination of the DMCA's registration requirement for qualification under the "safe harbor" provision. It's absurd that a website owner needs to mail in a form and pay a $100+ fee to the government to register a contact for copyright violations. A web page at a standard location (a la robots.txt) should suffice.
• We expect courts to apply penalties just as severe to rightsholders who issue abusive takedown notices as those applied to copyright violators.
• No more Jammie Thomases. Any penalties for copyright infringement must be sane and reasonable and not based unsubstantiated, outlandish claims of harm.
• The Department of Justice must begin an anti-trust investigation into the copyright industry, with a specific focus on collusion between rightsholders and ISPs in monitoring Internet users, and payola and cross ownership with mass media.
• We demand an end to sales of radio frequencies into private hands. We hold that spectrum is a form of speech - it rightly belongs to the people and is not the government's to auction off to begin with.
• We demand that ISPs stop interfering with file sharing via BitTorrent or any other protocol.
• We recognize a right of total ownership, not merely licensing, of products we have purchased and a right to tinker and modify them as we see fit. The Library of Congress should not be determining the acceptable boundaries of technology.
• We reject the principle of contributory infringement entirely. While there may be bad uses, there is no bad code.
• We expect legislators and judges to make efforts to educate themselves about the technologies they oversee, and to call on and respect the opinions of technical experts when necessary. The Internet makes nerds of us all.
• All research receiving any public funding, directly or indirectly, must be placed in the public domain upon publication.
• For the sake of innovation and competitiveness, the US Patent & Trade Office must immediately cease issuing software and business method patents, and declare all such existing patents null and void. We unequivocally reject any patents on mathematical formulas and genes or other naturally-occurring substances (human or otherwise).
• Copyright and patent terms must be reduced to reasonable lengths (two and five years from the time of creation, respectively). Works should only be eligible for the length of protection in the effect when created - the continuing extension of terms to protect Disney's ownership of Mickey Mouse must cease.
• We recognize a broad right of "fair use" as essential to a vibrant and creative culture. We will remix, sample, mash up, translate, perform, parody and otherwise create derivative works as we see fit.
• Courts must accord bloggers the same rights as mainstream reporters. The right to a free press originally meant a literal, physical printing press - not membership in some government sanctioned elite. Blogs are the modern day digital equivalent.

We call upon all freedom loving Internauts to join us. We further call upon our legislators, bureaucrats and the media & telecommunications industries to immediately begin implementing our demands. The future of free speech is bright, and clear - either stand with us or get out of the way. [1]

KeepTheWebOpen - Fight Back!

The KeepTheWebOpen site shows how the world may fundamentally be changed for the better... soon.. maybe...

PirateBay and Anonymous SOPA Press Releases originally appeared on AskApache.com

Hi there! I'm a bike messenger by day, aspiring actor by night, and this is my blog. I live in Los Angeles, have a great dog named Jack, and I like piña coladas. (And gettin' caught in the rain.)

The XYZ Doohickey Company was founded in 1971, and has been providing quality doohickies to the public ever since. Located in Gotham City, XYZ employs over 2,000 people and does all kinds of awesome things for the Gotham community.

Sample Page originally appeared on AskApache.com

This is part of the GRUB article on the Arch Linux wiki that I contributed. As of now, just a copy.

The grub menu.lst provides for a convenient way to add a number of entries with extended kernel parameters to configure all sorts of advanced settings to enable you to quickly and conveniently boot into your existing system with varying levels of debugging output. It's very easy and useful to create several levels of debugging just by adding additional entries to your grub configuration. And if you ever have issues or problems down the road due to a power-failure or hardware failure, it can save you hours of trouble, and of course nothing can beat debugging output when it comes to learning about your system.

Useful Menu.lst Entries

If you are interested in debugging, then you deserve some grub entries for powerusers, here are a few that I like (just add to your menu.lst).

title Shutdown the Computer
halt
 
title Reboot the Computer
reboot
 
title Command Line
commandline
 
title Install GRUB to hd0 MBR
root (hd0,0)
setup (hd0)
 
title Matrix
color green/black light-green/green
 
title Scan for /boot/grub/menu.lst
find --set-root --ignore-floppies /boot/grub/menu.lst
configfile /boot/grub/menu.lst
 
title Scan for /boot/menu.lst
find --set-root --ignore-floppies /menu.lst
configfile /boot/menu.lst
 
# http://www.vortex.prodigynet.co.uk/x86test/
title    Run x86test (CPU Info)
kernel /boot/x86test_zImage.bin
#wget http://www.vortex.prodigynet.co.uk/x86test/x86test_zImage.bin
 
# http://www.memtest.org/
title    Run memtest86+ (Memory Testing)
kernel /boot/memtest86+-1.70.bin

Light Debug

A quick way to see more verbose messages on your console is to bootup your normal grub entry after appending verbose to the kernel line. This simple word added to your kernel line turns on more logging thanks to the /etc/rc.sysinit file, which at the top of the file runs:

if /bin/grep -q " verbose" /proc/cmdline; then /bin/dmesg -n 8; fi

Very simple way to get a bit more messages and debug output in your logs.

title  Arch Linux DEBUG Light
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait verbose
initrd /kernel26.img

Medium Debug

This example menu.lst entry turns on real logging that is set by the kernel and not in an init script. Adding the debug kernel parameter to your kernel line is recognized by a lot of linux internals and enables quite a bit of debugging compared to the default.

title Arch Linux DEBUG Medium
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootdelay=5 panic=10 debug
initrd /kernel26.img

Heavy Debug

An even more impressive kernel parameter is the ignore_loglevel, which causes the system to ignore any loglevel and keeps the internal loglevel at the maximum debugging level, basically rendering dmesg unable to lower the debug level.

title Arch Linux DEBUG Heavy
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootdelay=5 panic=10 debug ignore_loglevel
initrd /kernel26.img

Extreme Debug

If the "Heavy Debug" seemed like a lot of output, thats about 1/2 of the logging that occurs with this example. This does a couple things, it uses the earlyprintk parameter to setup your kernel for "early" "printing" of messages to your "vga" screen. The ,keep just lets it stay on the screen longer. This will let you see logs that normally are hidden due to the boot-up process. This also changes the log buffer length to 10MB, and also instructs that any fatal signals be printed with print_fatal_signals. The last one, sched_debug, you can look up in the very excellent kernel documentation on kernel parameters.

title Arch Linux DEBUG Extreme
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro debug ignore_loglevel log_buf_len=10M print_fatal_signals=1 LOGLEVEL=8 earlyprintk=vga,keep sched_debug
initrd /kernel26.img

Insane Debug

The first few debugging examples showed some really nice kernel parameters to turn on really verbose debugging. This kind of debugging is absolutely critical if you want to max out your system or just learn more about what is going on behind the scenes. But there is a final trick that is my favorite, it's the ability to set both environment variables, and more importantly, module parameters at boot.

As an example, here is the boot line that I am using at the moment on an older Dell Desktop, just to illustrate module parameters and environment vars.

title  Arch Linux X-256
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait pause_on_oops=5 panic=60 i915.modeset=1 no_console_suspend ipv6.disable=1 TERM=xterm-256color quiet 5
initrd /kernel26.img

Since it's low on both memory and CPU, I disable ipv6. I also turn on kernel modesetting for the i915 video card, set my terminal to be xterm-256color, and boot straight into X. This lets me use a very optimized arch-linux configuration, amazing how fast thanks to using slim as the login manager, ratpoison as my window manager, and terminal with tmux as my login shell, all from boot, as the pstree shows (plus Synergy!).

init,1
  |-slim,3096
  |   |-X,3098 -nolisten tcp vt07 -auth /var/run/slim.auth
  |   `-ratpoison,3107,askapache
  |       |-terminal,5341 -x sh -c exec /usr/bin/tmux -2 -l -u -q attach -d -t tmux-askapache
  |       |   |-bash,11165
  |       |   |-tmux,5345 -2 -l -u -q attach -d -t tmux-askapache
  |       |   `-{terminal},5346
  |       `-xscreensaver,3113 -no-splash
  |-synergyc,6121,galileo -f --name galileo-fire --restart 10.66.66.2:26666
  |
  `-tmux,5348,askapache -2 -l -u -q attach -d -t tmux-askapache
      |-bash,5351
      |   `-ssh,9969 lug@askapache.com
      `-bash,5868
         `-vim,11149 -p sda1/grub/menu.lst /boot/grub/menu.lst

That kind of optimized system is only possible if you first can figure out your system, by debugging both the kernel as previously illustrated, debugging the init process, and most importantly, by debugging the modules enabled for your system's hardware/firmware/software. Debugging modules is challenging but worth the effort, and then you are able to do some truly insane debugging from grub like the following example, note that the actual grub entry is all on one line, but I split it into 4 lines so you could see it all. This basically turns on every module on this little Dell desktop to be at the absolute max debug level. There is so much logging when I boot this that the system grinds to a halt and is slower than a TI-89 calculator (See Improve Boot Performance).

title  Arch Linux DEBUG INSANE
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait ignore_loglevel debug debug_locks_verbose=1 sched_debug initcall_debug mminit_loglevel=4 udev.log_priority=8
       loglevel=8 earlyprintk=vga,keep log_buf_len=10M print_fatal_signals=1 apm.debug=Y i8042.debug=Y drm.debug=1 scsi_logging_level=1 usbserial.debug=Y
       option.debug=Y pl2303.debug=Y firewire_ohci.debug=1 hid.debug=1 pci_hotplug.debug=Y pci_hotplug.debug_acpi=Y shpchp.shpchp_debug=Y apic=debug
       show_lapic=all hpet=verbose lmb=debug pause_on_oops=5 panic=10 sysrq_always_enabled
initrd /kernel26.img

A couple key items from that grub entry are sysrq_always_enabled which forces on the sysrq magic, which really is a lifesaver when debugging at this level as your machine will freeze/stop-responding sometimes and it's nice to use sysrq to kill all tasks, change the loglevel, unmount all filesystems, or do a hard reboot. Another key parameter is the initcall_debug, which debugs the init process in excruciating detail. Very useful at times. The last parametery I find very useful is the udev.log_priority=8 to turn on udev logging.

Break Into Init

For instance, If you add break=y to your kernel cmdline, init will pause early in the boot process (after loading modules) and launch an interactive sh shell which can be used for troubleshooting purposes. (Normal boot continues after logout.) This is very similar to the shell that shows up if your computer gets turned off before it is able to shutdown properly. But using this parameter lets you enter into this mode differently at will.

title  Arch Linux Init Break
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait break=y
initrd /kernel26.img

Debugging init

This awesome parameter udev.log_priority=8 does the same thing as editing the file /etc/udev/udev.conf except it executes earlier, turning on debugging output for udev. If you want to know your hardware, that is the key parameter right there. Another trick is if you change the /etc/udev/udev.conf to be verbose, then you can make your initrd image include that file to turn on verbose udeb debugging by adding it to your {{Filename|/etc/mkinitcpio.conf} like:

FILES="/etc/modprobe.d/modprobe.conf /etc/udev/udev.conf"

, which on arch is as easy as

# mkinitcpio -p kernel26

Debugging udev is key because the initrd performs a root change at the end of its run to usually launch a program like /sbin/init as part of a chroot, and unless the new file system has a valid /dev directory, udev must be initialized before invoking chroot in order to provide /dev/console.

exec chroot . /sbin/init <dev/console >dev/console 2>&1

So basically, you aren't able to view the logs that are generated before /dev/console is initialized by udev or by a special initrd you compiled yourself. One method the kernel developers use to be able to still get the log messages generated before /dev/console is available is to provide an alternative console that you can enable or disable from grub.

Net Console

If you read through the kernel documentation regarding debugging, you will hear about Netconsole, which can be loaded from the kernel line in GRUB, compiled into your kernel, or loaded at runtime as a module. Having a netconsole entry in your menu.lst is most excellent for debugging slower computers like old laptops or thin-clients. It's easy to use. Just setup a 2nd computer (running arch) to accept syslog requests on a remote port, very fast and quick to do on arch-linux, 1 line to syslog.conf. Then you could use a log-color-parser like ccze to view all syslog logs, or just tail your everything.log. Then on your laptop, boot up and select the netconsole entry from the grub menu, and you will start seeing as much logging as you want on your syslog system. This logging lets you view even earlier log output than is available with the earlyprintk=vga kernel parameter, as netconsole is used by kernel hackers and developers, so it's very powerful.

title  Arch Linux DEBUG Netconsole
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro netconsole=514@10.0.0.2/12:34:56:78:9a:bc debug ignore_loglevel
initrd /kernel26.img

Hijacking cmdline

If you do not have access to GRUB or the kernel boottime cmdline, like on a server or virtual machine, as long as you have root permissions you can still enable this kind of simplistic verbose logging using a neat hack. While you cannot modify the /proc/cmdline even as root, you can place your own cmdline file on top of /proc/cmdline, so that accessing /proc/cmdline actually accesses your file.

For example if I cat /proc/cmdline, I have the following:

root=/dev/disk/by-label/ROOT ro console=tty1 logo.nologo quiet

So I use a simple sed command to replace quiet with verbose like:

sed 's/ quiet/ verbose/' /proc/cmdline > /root/cmdline

Then I bind mount /root/cmdline so that it becomes /proc/cmdline, using the -n option to mount so that this mount won't be recorded in the systems mtab.

mount -n --bind -o ro /root/cmdline /proc/cmdline

Now if I cat /proc/cmdline, I have the following:

root=/dev/disk/by-label/ROOT ro console=tty1 logo.nologo verbose

Advanced Linux Debugging using a Bootloader (GRUB) originally appeared on AskApache.com

I was trying to find an official, authoritative list of HTTP Status Codes but I kept finding lists that weren't authoritative or complete. So I searched and found my answer in the Apache HTTP Server source code. Once I had the exact HTTP Status Codes and resulting Error Documents sent by Apache, I researched deeper into HTTP Status Codes by reading as many related RFC's as I could find, and several other software source codes were explored. This is the most authoritative list I know of, if you can do better leave a comment and I'll update it. Another thing to keep in mind, the Status code number itself is what is used by software and hardware to make determinations, the phrase returned by the status code is for the human only and does not have any weight other than informing the user.. So "503 Service Unavailable", "503 Service Temporarily Unavailable", and "503 Get the heck outta here" are all completely valid.

Update March 9, 2009: A lot of sites on the web have updated their HTTP status code lists to include the HTTP Status codes listed on this page, including Wikipedia, IANA, W3C, and others, so rest assured this info is accurate and complete. If you'd like to see how to create custom error pages for all of these errors like mine /show-error-506 , then check out this detailed tutorial I just posted.

Contents

• List of All 57 HTTP Response Status Code
• Quick Start to triggering ErrorDocuments for each Status Code
• Automate the ErrorDocument Triggering
  • The htaccess Code
  • PHP script that gets and outputs the Headers/Content
• Headers and Content Returned
• Apache Source Code
  • httpd.h
  • http_protocol.c

57 APACHE HTTP STATUS RESPONSE CODES

Once I compiled the list of Apache recognized HTTP Status Codes, I was dying to see them all in action (i.e. the corresponding ErrorDocument). At first I thought I would have to create a php or perl script emulating each of the 57 HTTP Status Codes, a tedious undertaking I wasn't about to do. Instead I "asked Apache" by searching the Apache HTTP Documentation for ambiguity sending Status Codes and/or triggering ErrorDocuments with an Apache Directive.
While reading up on mod_alias and the Redirect directive I found:

Apache Docs

Syntax: Redirect [status] URL-path URL The status argument can be used to return other HTTP status codes. Other status codes can be returned by giving the numeric status code as the value of status. If the status is between 300 and 399, the URL argument must be present, otherwise it must be omitted.

100 Continue

ErrorDocument Continue | Sample 100 Continue
This means that the server has received the request headers, and that the client should proceed to send the request body (in case of a request which needs to be sent; for example, a POST request). If the request body is large, sending it to a server when a request has already been rejected based upon inappropriate headers is inefficient. To have a server check if the request could be accepted based on the requests headers alone, a client must send Expect: 100-continue as a header in its initial request (see RFC 2616 14.20 Expect header) and check if a 100 Continue status code is received in response before continuing (or receive 417 Expectation Failed and not continue).

101 Switching Protocols

ErrorDocument Switching Protocols | Sample 101 Switching Protocols
This means the requester has asked the server to switch protocols and the server is acknowledging that it will do so.[3]

102 Processing

ErrorDocument Processing | Sample 102 Processing
(WebDAV) - (RFC 2518 )

200 OK

ErrorDocument OK | Sample 200 OK
Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request the response will contain an entity describing or containing the result of the action.

201 Created

ErrorDocument Created | Sample 201 Created
The request has been fulfilled and resulted in a new resource being created.

202 Accepted

ErrorDocument Accepted | Sample 202 Accepted
The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place.

203 Non-Authoritative Information

ErrorDocument Non-Authoritative Information | Sample 203 Non-Authoritative Information
The server successfully processed the request, but is returning information that may be from another source.

204 No Content

ErrorDocument No Content | Sample 204 No Content
The server successfully processed the request, but is not returning any content.

205 Reset Content

ErrorDocument Reset Content | Sample 205 Reset Content
The server successfully processed the request, but is not returning any content. Unlike a 204 response, this response requires that the requester reset the document view.

206 Partial Content

ErrorDocument Partial Content | Sample 206 Partial Content
The server is delivering only part of the resource due to a range header sent by the client. This is used by tools like wget to enable resuming of interrupted downloads, or split a download into multiple simultaneous streams.

207 Multi-Status

ErrorDocument Multi-Status | Sample 207 Multi-Status
(WebDAV) - The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made.

226 IM Used

ErrorDocument IM Used | Sample 226 IM Used
The server has fulfilled a GET request for the resource, and the response is a representation of the result of one or more instance-manipulations applied to the current instance. The actual current instance might not be available except by combining this response with other previous or future responses, as appropriate for the specific instance-manipulation(s).

300 Multiple Choices

ErrorDocument Multiple Choices | Sample 300 Multiple Choices
Indicates multiple options for the resource that the client may follow. It, for instance, could be used to present different format options for video, list files with different extensions, or word sense disambiguation.

301 Moved Permanently

ErrorDocument Moved Permanently | Sample 301 Moved Permanently
This and all future requests should be directed to the given URI.

302 Found

ErrorDocument Found | Sample 302 Found
This is the most popular redirect code[citation needed], but also an example of industrial practice contradicting the standard. HTTP/1.0 specification (RFC 1945 ) required the client to perform a temporary redirect (the original describing phrase was "Moved Temporarily"), but popular browsers implemented it as a 303 See Other. Therefore, HTTP/1.1 added status codes 303 and 307 to disambiguate between the two behaviours. However, the majority of Web applications and frameworks still use the 302 status code as if it were the 303.

303 See Other

ErrorDocument See Other | Sample 303 See Other
The response to the request can be found under another URI using a GET method. When received in response to a PUT, it should be assumed that the server has received the data and the redirect should be issued with a separate GET message.

304 Not Modified

ErrorDocument Not Modified | Sample 304 Not Modified
Indicates the resource has not been modified since last requested. Typically, the HTTP client provides a header like the If-Modified-Since header to provide a time against which to compare. Utilizing this saves bandwidth and reprocessing on both the server and client, as only the header data must be sent and received in comparison to the entirety of the page being re-processed by the server, then resent using more bandwidth of the server and client.

305 Use Proxy

ErrorDocument Use Proxy | Sample 305 Use Proxy
Many HTTP clients (such as Mozilla[4] and Internet Explorer) do not correctly handle responses with this status code, primarily for security reasons.

306 Switch Proxy

ErrorDocument Switch Proxy | Sample 306 Switch Proxy
No longer used.

307 Temporary Redirect

ErrorDocument Temporary Redirect | Sample 307 Temporary Redirect
In this occasion, the request should be repeated with another URI, but future requests can still use the original URI. In contrast to 303, the request method should not be changed when reissuing the original request. For instance, a POST request must be repeated using another POST request.

400 Bad Request

ErrorDocument Bad Request | Sample 400 Bad Request
The request contains bad syntax or cannot be fulfilled.

401 Unauthorized

ErrorDocument Unauthorized | Sample 401 Unauthorized
Similar to 403 Forbidden, but specifically for use when authentication is possible but has failed or not yet been provided. The response must include a WWW-Authenticate header field containing a challenge applicable to the requested resource. See Basic access authentication and Digest access authentication.

402 Payment Required

ErrorDocument Payment Required | Sample 402 Payment Required
The original intention was that this code might be used as part of some form of digital cash or micropayment scheme, but that has not happened, and this code has never been used.

403 Forbidden

ErrorDocument Forbidden | Sample 403 Forbidden
The request was a legal request, but the server is refusing to respond to it. Unlike a 401 Unauthorized response, authenticating will make no difference.

404 Not Found

ErrorDocument Not Found | Sample 404 Not Found
The requested resource could not be found but may be available again in the future. Subsequent requests by the client are permissible.

405 Method Not Allowed

ErrorDocument Method Not Allowed | Sample 405 Method Not Allowed
A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource.

406 Not Acceptable

ErrorDocument Not Acceptable | Sample 406 Not Acceptable
The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request.

407 Proxy Authentication Required

ErrorDocument Proxy Authentication Required | Sample 407 Proxy Authentication Required
Required

408 Request Timeout

ErrorDocument Request Timeout | Sample 408 Request Timeout
The server timed out waiting for the request.

409 Conflict

ErrorDocument Conflict | Sample 409 Conflict
Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.

410 Gone

ErrorDocument Gone | Sample 410 Gone
Indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed; however, it is not necessary to return this code and a 404 Not Found can be issued instead. Upon receiving a 410 status code, the client should not request the resource again in the future. Clients such as search engines should remove the resource from their indexes.

411 Length Required

ErrorDocument Length Required | Sample 411 Length Required
The request did not specify the length of its content, which is required by the requested resource.

412 Precondition Failed

ErrorDocument Precondition Failed | Sample 412 Precondition Failed
The server does not meet one of the preconditions that the requester put on the request.

413 Request Entity Too Large

ErrorDocument Request Entity Too Large | Sample 413 Request Entity Too Large
The request is larger than the server is willing or able to process.

414 Request-URI Too Long

ErrorDocument Request-URI Too Long | Sample 414 Request-URI Too Long
The URI provided was too long for the server to process.

415 Unsupported Media Type

ErrorDocument Unsupported Media Type | Sample 415 Unsupported Media Type
The request did not specify any media types that the server or resource supports. For example the client specified that an image resource should be served as image/svg+xml, but the server cannot find a matching version of the image.

416 Requested Range Not Satisfiable

ErrorDocument Requested Range Not Satisfiable | Sample 416 Requested Range Not Satisfiable
The client has asked for a portion of the file, but the server cannot supply that portion (for example, if the client asked for a part of the file that lies beyond the end of the file).

417 Expectation Failed

ErrorDocument Expectation Failed | Sample 417 Expectation Failed
The server cannot meet the requirements of the Expect request-header field.

418 I'm a teapot

ErrorDocument I'm a teapot | Sample 418 I'm a teapot
The HTCPCP server is a teapot. The responding entity MAY be short and stout. Defined by the April Fools specification RFC 2324. See Hyper Text Coffee Pot Control Protocol for more information.

422 Unprocessable Entity

ErrorDocument Unprocessable Entity | Sample 422 Unprocessable Entity
(WebDAV) (RFC 4918 ) - The request was well-formed but was unable to be followed due to semantic errors.

423 Locked

ErrorDocument Locked | Sample 423 Locked
(WebDAV) (RFC 4918 ) - The resource that is being accessed is locked

424 Failed Dependency

ErrorDocument Failed Dependency | Sample 424 Failed Dependency
(WebDAV) (RFC 4918 ) - The request failed due to failure of a previous request (e.g. a PROPPATCH).

425 Unordered Collection

ErrorDocument Unordered Collection | Sample 425 Unordered Collection
Defined in drafts of WebDav Advanced Collections, but not present in "Web Distributed Authoring and Versioning (WebDAV) Ordered Collections Protocol" (RFC 3648).

426 Upgrade Required

ErrorDocument Upgrade Required | Sample 426 Upgrade Required
(RFC 2817 ) - The client should switch to TLS/1.0.

449 Retry With

ErrorDocument Retry With | Sample 449 Retry With
A Microsoft extension. The request should be retried after doing the appropriate action.

500 Internal Server Error

ErrorDocument Internal Server Error | Sample 500 Internal Server Error
A generic error message, given when no more specific message is suitable.

501 Not Implemented

ErrorDocument Not Implemented | Sample 501 Not Implemented
The server either does not recognise the request method, or it lacks the ability to fulfil the request.

502 Bad Gateway

ErrorDocument Bad Gateway | Sample 502 Bad Gateway
The server was acting as a gateway or proxy and received an invalid response from the upstream server.

503 Service Unavailable

ErrorDocument Service Unavailable | Sample 503 Service Unavailable
The server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state.

504 Gateway Timeout

ErrorDocument Gateway Timeout | Sample 504 Gateway Timeout
The server was acting as a gateway or proxy and did not receive a timely request from the upstream server.

505 HTTP Version Not Supported

ErrorDocument HTTP Version Not Supported | Sample 505 HTTP Version Not Supported
The server does not support the HTTP protocol version used in the request.

506 Variant Also Negotiates

ErrorDocument Variant Also Negotiates | Sample 506 Variant Also Negotiates
(RFC 2295 ) - Transparent content negotiation for the request, results in a circular reference.

507 Insufficient Storage

ErrorDocument Insufficient Storage | Sample 507 Insufficient Storage
(WebDAV) (RFC 4918 )

509 Bandwidth Limit Exceeded

ErrorDocument Bandwidth Limit Exceeded | Sample 509 Bandwidth Limit Exceeded
(Apache bw/limited extension) - This status code, while used by many servers, is not specified in any RFCs.

510 Not Extended

ErrorDocument Not Extended | Sample 510 Not Extended
(RFC 2774 ) - Further extensions to the request are required for the server to fulfil it.

1xx Info / Informational

HTTP_INFO - Request received, continuing process. Indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line.

• 100 Continue - HTTP_CONTINUE
• 101 Switching Protocols - HTTP_SWITCHING_PROTOCOLS
• 102 Processing - HTTP_PROCESSING

2xx Success / OK

HTTP_SUCCESS - The action was successfully received, understood, and accepted. Indicates that the client's request was successfully received, understood, and accepted.

• 200 OK - HTTP_OK
• 201 Created - HTTP_CREATED
• 202 Accepted - HTTP_ACCEPTED
• 203 Non-Authoritative Information - HTTP_NON_AUTHORITATIVE
• 204 No Content - HTTP_NO_CONTENT
• 205 Reset Content - HTTP_RESET_CONTENT
• 206 Partial Content - HTTP_PARTIAL_CONTENT
• 207 Multi-Status - HTTP_MULTI_STATUS

3xx Redirect

HTTP_REDIRECT - The client must take additional action to complete the request. Indicates that further action needs to be taken by the user-agent in order to fulfill the request. The action required may be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A user agent should not automatically redirect a request more than 5 times, since such redirections usually indicate an infinite loop.

• 300 Multiple Choices - HTTP_MULTIPLE_CHOICES
• 301 Moved Permanently - HTTP_MOVED_PERMANENTLY
• 302 Found - HTTP_MOVED_TEMPORARILY
• 303 See Other - HTTP_SEE_OTHER
• 304 Not Modified - HTTP_NOT_MODIFIED
• 305 Use Proxy - HTTP_USE_PROXY
• 306 unused - UNUSED
• 307 Temporary Redirect - HTTP_TEMPORARY_REDIRECT

4xx Client Error

HTTP_CLIENT_ERROR - The request contains bad syntax or cannot be fulfilled. Indicates case where client seems to have erred. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition.

• 400 Bad Request - HTTP_BAD_REQUEST
• 401 Authorization Required - HTTP_UNAUTHORIZED
• 402 Payment Required - HTTP_PAYMENT_REQUIRED
• 403 Forbidden - HTTP_FORBIDDEN
• 404 Not Found - HTTP_NOT_FOUND
• 405 Method Not Allowed - HTTP_METHOD_NOT_ALLOWED
• 406 Not Acceptable - HTTP_NOT_ACCEPTABLE
• 407 Proxy Authentication Required - HTTP_PROXY_AUTHENTICATION_REQUIRED
• 408 Request Time-out - HTTP_REQUEST_TIME_OUT
• 409 Conflict - HTTP_CONFLICT
• 410 Gone - HTTP_GONE
• 411 Length Required - HTTP_LENGTH_REQUIRED
• 412 Precondition Failed - HTTP_PRECONDITION_FAILED
• 413 Request Entity Too Large - HTTP_REQUEST_ENTITY_TOO_LARGE
• 414 Request-URI Too Large - HTTP_REQUEST_URI_TOO_LARGE
• 415 Unsupported Media Type - HTTP_UNSUPPORTED_MEDIA_TYPE
• 416 Requested Range Not Satisfiable - HTTP_RANGE_NOT_SATISFIABLE
• 417 Expectation Failed - HTTP_EXPECTATION_FAILED
• 418 I'm a teapot - UNUSED
• 419 unused - UNUSED
• 420 unused - UNUSED
• 421 unused - UNUSED
• 422 Unprocessable Entity - HTTP_UNPROCESSABLE_ENTITY
• 423 Locked - HTTP_LOCKED
• 424 Failed Dependency - HTTP_FAILED_DEPENDENCY
• 425 No code - HTTP_NO_CODE
• 426 Upgrade Required - HTTP_UPGRADE_REQUIRED

5xx Server Error

HTTP_SERVER_ERROR - The server failed to fulfill an apparently valid request. Indicate cases in which the server is aware that it has erred or is incapable of performing the request. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. User agents should display any included entity to the user. These response codes are applicable to any request method.

• 500 Internal Server Error - HTTP_INTERNAL_SERVER_ERROR
• 501 Method Not Implemented - HTTP_NOT_IMPLEMENTED
• 502 Bad Gateway - HTTP_BAD_GATEWAY
• 503 Service Temporarily Unavailable - HTTP_SERVICE_UNAVAILABLE
• 504 Gateway Time-out - HTTP_GATEWAY_TIME_OUT
• 505 HTTP Version Not Supported - HTTP_VERSION_NOT_SUPPORTED
• 506 Variant Also Negotiates - HTTP_VARIANT_ALSO_VARIES
• 507 Insufficient Storage - HTTP_INSUFFICIENT_STORAGE
• 508 unused - UNUSED
• 509 unused - UNUSED
• 510 Not Extended - HTTP_NOT_EXTENDED

Quick Start to triggering ErrorDocuments for each Status Code

Let start with a quick and easy example. Add the following Redirect rules to your htaccess file, then open your browser and goto each url like yoursite.com/e/400. Don't create an /e/ directory or any files.

Redirect 400 /e/400
Redirect 503 /e/503
Redirect 405 /e/405

Automate the ErrorDocument Triggering

The htaccess Redirects

When a Status code is encountered, Apache outputs the Header and the ErrorDocument for that error code. So you can view any Header and the default ErrorDocument, by causing that numerical error code, which is caused by the Status Code.

For instance, if you request a file that doesn't exist, a 404 Not Found Header is issued and the corresponding ErrorDocument is served with the 404 Not Found Header.

Redirect 100 /e/100
Redirect 101 /e/101
Redirect 102 /e/102
Redirect 200 /e/200
Redirect 201 /e/201
Redirect 202 /e/202
Redirect 203 /e/203
Redirect 204 /e/204
Redirect 205 /e/205
Redirect 206 /e/206
Redirect 207 /e/207
Redirect 300 /e/300 http://www.askapache.com/?s=300
Redirect 301 /e/301 http://www.askapache.com/?s=301
Redirect 302 /e/302 http://www.askapache.com/?s=302
Redirect 303 /e/303 http://www.askapache.com/?s=303
Redirect 304 /e/304 http://www.askapache.com/?s=304
Redirect 305 /e/305 http://www.askapache.com/?s=305
Redirect 306 /e/306 http://www.askapache.com/?s=306
Redirect 307 /e/307 http://www.askapache.com/?s=307
Redirect 400 /e/400
Redirect 401 /e/401
Redirect 402 /e/402
Redirect 403 /e/403
Redirect 404 /e/404
Redirect 405 /e/405
Redirect 406 /e/406
Redirect 407 /e/407
Redirect 408 /e/408
Redirect 409 /e/409
Redirect 410 /e/410
Redirect 411 /e/411
Redirect 412 /e/412
Redirect 413 /e/413
Redirect 414 /e/414
Redirect 415 /e/415
Redirect 416 /e/416
Redirect 417 /e/417
Redirect 418 /e/418
Redirect 419 /e/419
Redirect 420 /e/420
Redirect 421 /e/421
Redirect 422 /e/422
Redirect 423 /e/423
Redirect 424 /e/424
Redirect 425 /e/425
Redirect 426 /e/426
Redirect 500 /e/500
Redirect 501 /e/501
Redirect 502 /e/502
Redirect 503 /e/503
Redirect 504 /e/504
Redirect 505 /e/505
Redirect 506 /e/506
Redirect 507 /e/507
Redirect 508 /e/508
Redirect 509 /e/509
Redirect 510 /e/510

PHP script that gets and outputs the Headers/Content

Now all I have to do is add 57 Redirect Directives to my htaccess, and then request each of them 1 at a time from my browser to see the result, and use a packet sniffing program like WireShark to see the Headers. Uh, scratch that, that would take way too long!

Instead I hacked up a simple php script using cURL to automate sending GET Requests to each of the 57 Redirect URL-paths. A side benefit of using the php script is that it performs all 57 Requests concurrently and saves each Requests returned headers and content to an output buffer. After all 57 have been queried, the output buffer is flushed to the browser.

<?php
$SITENAME='http://www.askapache.com';
 
$CODES = array(array('100','101','102'),
array('200','201','202','203','204','205','206','207'),
array('300','301','302','303','304','305','306','307'),
array('400','401','402','403','404','405','406','407','408','409','410','411','412','413',
'414','415','416','417','418','419','420','421','422','423','424','425','426'),
array('500','501','502','503','504','505','506','507','508','509','510'));
 
$TMPSAVETO='/tmp/'.time().'.txt';
 
# if file exists then delete it
if(is_file($TMPSAVETO))unlink($TMPSAVETO);
 
foreach($CODES as $keyd => $res)
{
foreach($res as $key)
{
$ch = curl_init("$SITENAME/e/$key");
$fp = fopen ($TMPSAVETO, "a");
curl_setopt ($ch, CURLOPT_FILE, $fp);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION ,1);
curl_setopt ($ch, CURLOPT_HEADER ,1);
curl_exec ($ch);
curl_close ($ch);
fclose ($fp);
}
}
$OUT='';
ob_start();
header ("Content-Type: text/plain;");
readfile($TMPSAVETO);
$OUT=ob_get_clean();
echo $OUT;
unlink($TMPSAVETO);
exit;
?>

Headers and Content Returned

100 Continue

HTTP/1.1 100 Continue
<html>
<head>
<title>100 Continue</title>
</head>
<body>
<h1>Continue</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

101 Switching Protocols

HTTP/1.1 101 Switching Protocols<html>
<head>
<title>101 Switching Protocols</title>
</head>
<body>
<h1>Switching Protocols</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

102 Processing

HTTP/1.1 102 Processing
X-Pad: avoid browser bug<html>
<head>
<title>102 Processing</title>
</head>
<body>
<h1>Processing</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

200 OK

HTTP/1.1 200 OK
<html>
<head>
<title>200 OK</title>
</head>
<body>
<h1>OK</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

201 Created

HTTP/1.1 201 Created
<html>
<head>
<title>201 Created</title>
</head>
<body>
<h1>Created</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

202 Accepted

HTTP/1.1 202 Accepted
<html>
<head>
<title>202 Accepted</title>
</head>
<body>
<h1>Accepted</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

203 Non-Authoritative Information

HTTP/1.1 203 Non-Authoritative Information
<html>
<head>
<title>203 Non-Authoritative Information</title>
</head>
<body>
<h1>Non-Authoritative Information</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

204 No Content

HTTP/1.1 204 No Content
Content-Type: text/plain; charset=UTF-8

205 Reset Content

HTTP/1.1 205 Reset Content<html>
<head>
<title>205 Reset Content</title>
</head>
<body>
<h1>Reset Content</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

206 Partial Content

HTTP/1.1 206 Partial Content<html>
<head>
<title>206 Partial Content</title>
</head>
<body>
<h1>Partial Content</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

207 Multi-Status

HTTP/1.1 207 Multi-Status
X-Pad: avoid browser bug<html>
<head>
<title>207 Multi-Status</title>
</head>
<body>
<h1>Multi-Status</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

300 Multiple Choices

HTTP/1.1 300 Multiple Choices
Location: http://www.askapache.com/?s=300<html>
<head>
<title>300 Multiple Choices</title>
</head>
<body>
<h1>Multiple Choices</h1>
</body>
</html>

301 Moved Permanently

HTTP/1.1 301 Moved Permanently
Location: http://www.askapache.com/?s=301<html>
<head>
<title>301 Moved Permanently</title>
</head>
<body>
<h1>Moved Permanently</h1>
<p>The document has moved  <a href="http://www.askapache.com/?s=301">here</a> .</p>
</body>
</html>

302 Found

HTTP/1.1 302 Found
Location: http://www.askapache.com/?s=302<html>
<head>
<title>302 Found</title>
</head>
<body>
<h1>Found</h1>
<p>The document has moved  <a href="http://www.askapache.com/?s=302">here</a> .</p>
</body>
</html>

303 See Other

HTTP/1.1 303 See Other
Location: http://www.askapache.com/?s=303<html>
<head>
<title>303 See Other</title>
</head>
<body>
<h1>See Other</h1>
<p>The answer to your request is located  <a href="http://www.askapache.com/?s=303">here</a> .</p>
</body>
</html>

304 Not Modified

HTTP/1.1 304 Not Modified

305 Use Proxy

HTTP/1.1 305 Use Proxy
Location: http://www.askapache.com/?s=305<html>
<head>
<title>305 Use Proxy</title>
</head>
<body>
<h1>Use Proxy</h1>
<p>This resource is only accessible through the proxy
    http://www.askapache.com/?s=305<br />You will need to configure your client to use that proxy.</p>
</body>
</html>

306 unused

HTTP/1.1 306 unused
Location: http://www.askapache.com/?s=306<html>
<head>
<title>306 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

307 Temporary Redirect

HTTP/1.1 307 Temporary Redirect
Location: http://www.askapache.com/?s=307<html>
<head>
<title>307 Temporary Redirect</title>
</head>
<body>
<h1>Temporary Redirect</h1>
<p>The document has moved  <a href="http://www.askapache.com/?s=307">here</a> .</p>
</body>
</html>

400 Bad Request

HTTP/1.1 400 Bad Request
Connection: close<html>
<head>
<title>400 Bad Request</title>
</head>
<body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.</p>
</body>
</html>

401 Authorization Required

HTTP/1.1 401 Authorization Required<html>
<head>
<title>401 Authorization Required</title>
</head>
<body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
    are authorized to access the document
    requested.  Either you supplied the wrong
    credentials (e.g., bad password), or your
    browser doesn't understand how to supply
    the credentials required.</p>
<p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body>
</html>

402 Payment Required

HTTP/1.1 402 Payment Required<html>
<head>
<title>402 Payment Required</title>
</head>
<body>
<h1>Payment Required</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

403 Forbidden

HTTP/1.1 403 Forbidden<html>
<head>
<title>403 Forbidden</title>
</head>
<body>
<h1>Forbidden</h1>
<p>You don't have permission to access /e/403
    on this server.</p>
<p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body>
</html>

404 Not Found

HTTP/1.1 404 Not Found<html>
<head>
<title>404 Not Found</title>
</head>
<body>
<h1>Not Found</h1>
<p>The requested URL /e/404 was not found on this server.</p>
</body>
</html>

NOTE:
You will most definately want to check out and use the Google 404 Error Page if you run WordPress.

405 Method Not Allowed

HTTP/1.1 405 Method Not Allowed
Allow: TRACE
<html>
<head>
<title>405 Method Not Allowed</title>
</head>
<body>
<h1>Method Not Allowed</h1>
<p>The requested method GET is not allowed for the URL /e/405.</p>
</body>
</html>

406 Not Acceptable

HTTP/1.1 406 Not Acceptable
<html>
<head>
<title>406 Not Acceptable</title>
</head>
<body>
<h1>Not Acceptable</h1>
<p>An appropriate representation of the requested resource /e/406 could not be found on this server.</p>
</body>
</html>

407 Proxy Authentication Required

HTTP/1.1 407 Proxy Authentication Required<html>
<head>
<title>407 Proxy Authentication Required</title>
</head>
<body>
<h1>Proxy Authentication Required</h1>
<p>This server could not verify that you
    are authorized to access the document
    requested.  Either you supplied the wrong
    credentials (e.g., bad password), or your
    browser doesn't understand how to supply
    the credentials required.</p>
</body>
</html>

408 Request Time-out

HTTP/1.1 408 Request Time-out
Connection: close
<html>
<head>
<title>408 Request Time-out</title>
</head>
<body>
<h1>Request Time-out</h1>
<p>Server timeout waiting for the HTTP request from the client.</p>
</body>
</html>

409 Conflict

HTTP/1.1 409 Conflict
<html>
<head>
<title>409 Conflict</title>
</head>
<body>
<h1>Conflict</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

410 Gone

HTTP/1.1 410 Gone
<html>
<head>
<title>410 Gone</title>
</head>
<body>
<h1>Gone</h1>
<p>The requested resource<br />/e/410<br />is no longer available on this server and there is no forwarding address.
    Please remove all references to this resource.</p>
</body>
</html>

411 Length Required

HTTP/1.1 411 Length Required
Connection: close
<html>
<head>
<title>411 Length Required</title>
</head>
<body>
<h1>Length Required</h1>
<p>A request of the requested method GET requires a valid Content-length.</p>
</body>
</html>

412 Precondition Failed

HTTP/1.1 412 Precondition Failed
<html>
<head>
<title>412 Precondition Failed</title>
</head>
<body>
<h1>Precondition Failed</h1>
<p>The precondition on the request for the URL /e/412 evaluated to false.</p>
</body>
</html>

413 Request Entity Too Large

HTTP/1.1 413 Request Entity Too Large
Connection: close<html>
<head>
<title>413 Request Entity Too Large</title>
</head>
<body>
<h1>Request Entity Too Large</h1>
The requested resource<br />/e/413<br />does not allow request data with GET requests, or the amount of data provided in
the request exceeds the capacity limit.
</body>
</html>

414 Request-URI Too Large

HTTP/1.1 414 Request-URI Too Large
Connection: close<html>
<head>
<title>414 Request-URI Too Large</title>
</head>
<body>
<h1>Request-URI Too Large</h1>
<p>The requested URL's length exceeds the capacity
    limit for this server.</p>
</body>
</html>

415 Unsupported Media Type

HTTP/1.1 415 Unsupported Media Type<html>
<head>
<title>415 Unsupported Media Type</title>
</head>
<body>
<h1>Unsupported Media Type</h1>
<p>The supplied request data is not in a format
    acceptable for processing by this resource.</p>
</body>
</html>

416 Requested Range Not Satisfiable

HTTP/1.1 416 Requested Range Not Satisfiable

417 Expectation Failed

HTTP/1.1 417 Expectation Failed<html>
<head>
<title>417 Expectation Failed</title>
</head>
<body>
<h1>Expectation Failed</h1>
<p>The expectation given in the Expect request-header
    field could not be met by this server.</p>
<p>The client sent<pre>
    Expect: </p>
</body>
</html>

418 I'm a teapot

HTTP/1.1 418 I'm a teapot<html>
<head>
<title>418 I'm a teapot</title>
</head>
<body>
<h1>I'm a teapot</h1>
<p>Unfortunately this coffee machine is out of coffee.</p>
</body>
</html>

419 unused

HTTP/1.1 419 unused<html>
<head>
<title>419 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

420 unused

HTTP/1.1 420 unused<html>
<head>
<title>420 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

421 unused

HTTP/1.1 421 unused<html>
<head>
<title>421 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

422 Unprocessable Entity

HTTP/1.1 422 Unprocessable Entity<html>
<head>
<title>422 Unprocessable Entity</title>
</head>
<body>
<h1>Unprocessable Entity</h1>
<p>The server understands the media type of the
    request entity, but was unable to process the
    contained instructions.</p>
</body>
</html>

423 Locked

HTTP/1.1 423 Locked<html>
<head>
<title>423 Locked</title>
</head>
<body>
<h1>Locked</h1>
<p>The requested resource is currently locked.
    The lock must be released or proper identification
    given before the method can be applied.</p>
</body>
</html>

424 Failed Dependency

HTTP/1.1 424 Failed Dependency<html>
<head>
<title>424 Failed Dependency</title>
</head>
<body>
<h1>Failed Dependency</h1>
<p>The method could not be performed on the resource
    because the requested action depended on another
    action and that other action failed.</p>
</body>
</html>

425 No code

HTTP/1.1 425 No code<html>
<head>
<title>425 No code</title>
</head>
<body>
<h1>No code</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

426 Upgrade Required

HTTP/1.1 426 Upgrade Required<html>
<head>
<title>426 Upgrade Required</title>
</head>
<body>
<h1>Upgrade Required</h1>
<p>The requested resource can only be retrieved
    using SSL.  The server is willing to upgrade the current
    connection to SSL, but your client doesn't support it.
    Either upgrade your client, or try requesting the page
    using https:// </p>
</body>
</html>

500 Internal Server Error

HTTP/1.1 500 Internal Server Error
Connection: close<html>
<head>
<title>500 Internal Server Error</title>
</head>
<body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
<p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body>
</html>

501 Method Not Implemented

HTTP/1.1 501 Method Not Implemented
Allow: TRACE
Connection: close<html>
<head>
<title>501 Method Not Implemented</title>
</head>
<body>
<h1>Method Not Implemented</h1>
<p>GET to /e/501 not supported.</p>
</body>
</html>

502 Bad Gateway

HTTP/1.1 502 Bad Gateway
X-Pad: avoid browser bug<html>
<head>
<title>502 Bad Gateway</title>
</head>
<body>
<h1>Bad Gateway</h1>
<p>The proxy server received an invalid
    response from an upstream server.</p>
</body>
</html>

503 Service Temporarily Unavailable

HTTP/1.1 503 Service Temporarily Unavailable
Connection: close<html>
<head>
<title>503 Service Temporarily Unavailable</title>
</head>
<body>
<h1>Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your
    request due to maintenance downtime or capacity
    problems. Please try again later.</p>
</body>
</html>

504 Gateway Time-out

HTTP/1.1 504 Gateway Time-out<html>
<head>
<title>504 Gateway Time-out</title>
</head>
<body>
<h1>Gateway Time-out</h1>
<p>The proxy server did not receive a timely response
    from the upstream server.</p>
</body>
</html>

505 HTTP Version Not Supported

HTTP/1.1 505 HTTP Version Not Supported<html>
<head>
<title>505 HTTP Version Not Supported</title>
</head>
<body>
<h1>HTTP Version Not Supported</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

506 Variant Also Negotiates

HTTP/1.1 506 Variant Also Negotiates<html>
<head>
<title>506 Variant Also Negotiates</title>
</head>
<body>
<h1>Variant Also Negotiates</h1>
<p>A variant for the requested resource
    <pre>
    /e/506
    </pre>
    is itself a negotiable resource. This indicates a configuration error.</p>
</body>
</html>

507 Insufficient Storage

HTTP/1.1 507 Insufficient Storage<html>
<head>
<title>507 Insufficient Storage</title>
</head>
<body>
<h1>Insufficient Storage</h1>
<p>The method could not be performed on the resource
    because the server is unable to store the
    representation needed to successfully complete the
    request.  There is insufficient free space left in
    your storage allocation.</p>
</body>
</html>

508 unused

HTTP/1.1 508 unused<html>
<head>
<title>508 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

509 unused

HTTP/1.1 509 unused
<html>
<head>
<title>509 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

510 Not Extended

HTTP/1.1 510 Not Extended
X-Pad: avoid browser bug
<html>
<head>
<title>510 Not Extended</title>
</head>
<body>
<h1>Not Extended</h1>
<p>A mandatory extension policy in the request is not
    accepted by the server for this resource.</p>
</body>
</html>

Apache Source Code

httpd.h

From httpd.h

/**
* The size of the static array in http_protocol.c for storing
* all of the potential response status-lines (a sparse table).
* A future version should dynamically generate the apr_table_t at startup.
*/
#define RESPONSE_CODES 57
#define HTTP_CONTINUE            100
#define HTTP_SWITCHING_PROTOCOLS       101
#define HTTP_PROCESSING          102
#define HTTP_OK              200
#define HTTP_CREATED             201
#define HTTP_ACCEPTED            202
#define HTTP_NON_AUTHORITATIVE       203
#define HTTP_NO_CONTENT          204
#define HTTP_RESET_CONTENT         205
#define HTTP_PARTIAL_CONTENT         206
#define HTTP_MULTI_STATUS          207
#define HTTP_MULTIPLE_CHOICES        300
#define HTTP_MOVED_PERMANENTLY       301
 
#define HTTP_MOVED_TEMPORARILY       302
#define HTTP_SEE_OTHER           303
#define HTTP_NOT_MODIFIED          304
#define HTTP_USE_PROXY           305
#define HTTP_TEMPORARY_REDIRECT      307
#define HTTP_BAD_REQUEST           400
#define HTTP_UNAUTHORIZED          401
#define HTTP_PAYMENT_REQUIRED        402
#define HTTP_FORBIDDEN           403
#define HTTP_NOT_FOUND           404
#define HTTP_METHOD_NOT_ALLOWED      405
#define HTTP_NOT_ACCEPTABLE        406
#define HTTP_PROXY_AUTHENTICATION_REQUIRED 407
#define HTTP_REQUEST_TIME_OUT        408
#define HTTP_CONFLICT            409
#define HTTP_GONE              410
#define HTTP_LENGTH_REQUIRED         411
#define HTTP_PRECONDITION_FAILED       412
#define HTTP_REQUEST_ENTITY_TOO_LARGE    413
#define HTTP_REQUEST_URI_TOO_LARGE     414
#define HTTP_UNSUPPORTED_MEDIA_TYPE    415
#define HTTP_RANGE_NOT_SATISFIABLE     416
#define HTTP_EXPECTATION_FAILED      417
#define HTTP_UNPROCESSABLE_ENTITY      422
#define HTTP_LOCKED            423
#define HTTP_FAILED_DEPENDENCY       424
#define HTTP_UPGRADE_REQUIRED        426
#define HTTP_INTERNAL_SERVER_ERROR     500
#define HTTP_NOT_IMPLEMENTED         501
#define HTTP_BAD_GATEWAY           502
#define HTTP_SERVICE_UNAVAILABLE       503
#define HTTP_GATEWAY_TIME_OUT        504
#define HTTP_VERSION_NOT_SUPPORTED     505
#define HTTP_VARIANT_ALSO_VARIES       506
#define HTTP_INSUFFICIENT_STORAGE      507
#define HTTP_NOT_EXTENDED          510
 
/** is the status code informational */
#define ap_is_HTTP_INFO(x)     (((x) >= 100)&&((x) < 200))
/** is the status code OK ?*/
#define ap_is_HTTP_SUCCESS(x)    (((x) >= 200)&&((x) < 300))
/** is the status code a redirect */
#define ap_is_HTTP_REDIRECT(x)   (((x) >= 300)&&((x) < 400))
/** is the status code a error (client or server) */
#define ap_is_HTTP_ERROR(x)    (((x) >= 400)&&((x) < 600))
/** is the status code a client error  */
#define ap_is_HTTP_CLIENT_ERROR(x) (((x) >= 400)&&((x) < 500))
/** is the status code a server error  */
#define ap_is_HTTP_SERVER_ERROR(x) (((x) >= 500)&&((x) < 600))
/** is the status code a (potentially) valid response code?  */
#define ap_is_HTTP_VALID_RESPONSE(x) (((x) >= 100)&&((x) < 600))
 
/** should the status code drop the connection */
#define ap_status_drops_connection(x) \
(((x) == HTTP_BAD_REQUEST)       || \
((x) == HTTP_REQUEST_TIME_OUT)    || \
((x) == HTTP_LENGTH_REQUIRED)     || \
((x) == HTTP_REQUEST_ENTITY_TOO_LARGE) || \
((x) == HTTP_REQUEST_URI_TOO_LARGE) || \
((x) == HTTP_INTERNAL_SERVER_ERROR) || \
((x) == HTTP_SERVICE_UNAVAILABLE) || \
((x) == HTTP_NOT_IMPLEMENTED))

HTTP_INFO

Is the status code (x) informational?

x >= 100 && x < 200

HTTP_SUCCESS

Is the status code (x) OK?

x >= 200 && x < 300

HTTP_REDIRECT

Is the status code (x) a redirect?

x >= 300 && x < 400

HTTP_ERROR

Is the status code (x) a error (client or server)?

x >= 400 && x < 600

HTTP_CLIENT_ERROR

Is the status code (x) a client error?

x >= 400 && x < 500

HTTP_SERVER_ERROR

Is the status code (x) a server error?

x >= 500 && x < 600

HTTP_VALID_RESPONSE

Is the status code (x) a (potentially) valid response code?

x >= 100 && x < 600

http_protocol.c

From http_protocol.c

static const char * status_lines[RESPONSE_CODES] =
static const char * const status_lines[RESPONSE_CODES] =
"100 Continue",
"101 Switching Protocols",
"102 Processing",
"200 OK",
"201 Created",
"202 Accepted",
"203 Non-Authoritative Information",
"204 No Content",
"205 Reset Content",
"206 Partial Content",
"207 Multi-Status",
"300 Multiple Choices",
"301 Moved Permanently",
"302 Found",
"303 See Other",
"304 Not Modified",
"305 Use Proxy",
"306 unused",
"307 Temporary Redirect",
"400 Bad Request",
"401 Authorization Required",
"402 Payment Required",
"403 Forbidden",
"404 Not Found",
"405 Method Not Allowed",
"406 Not Acceptable",
"407 Proxy Authentication Required",
"408 Request Time-out",
"409 Conflict",
"410 Gone",
"411 Length Required",
"412 Precondition Failed",
"413 Request Entity Too Large",
"414 Request-URI Too Large",
"415 Unsupported Media Type",
"416 Requested Range Not Satisfiable",
"417 Expectation Failed",
"418 unused",
"419 unused",
"420 unused",
"421 unused",
"422 Unprocessable Entity",
"423 Locked",
"424 Failed Dependency",
"425 No code",
"426 Upgrade Required",
"500 Internal Server Error",
"501 Method Not Implemented",
"502 Bad Gateway",
"503 Service Temporarily Unavailable",
"504 Gateway Time-out",
"505 HTTP Version Not Supported",
"506 Variant Also Negotiates",
"507 Insufficient Storage",
"508 unused",
"509 unused",
"510 Not Extended"

IANA HTTP Status Code Registry

WordPress 2.8 Changes

I just learned that my modification to the WordPress core was accepted and will be implemented for version 2.8! This may mean WordPress is the only 100% HTTP/1.1 compliant software on the net! Below is the new list (I thought someone out there could use the php array) and as you can see, unfortunately418 I'm a teapotdidn't make it ;)

$wp_header_to_desc = array(
  100 => 'Continue',
  101 => 'Switching Protocols',
  102 => 'Processing',
 
  200 => 'OK',
  201 => 'Created',
  202 => 'Accepted',
  203 => 'Non-Authoritative Information',
  204 => 'No Content',
  205 => 'Reset Content',
  206 => 'Partial Content',
  207 => 'Multi-Status',
  226 => 'IM Used',
 
  300 => 'Multiple Choices',
  301 => 'Moved Permanently',
  302 => 'Found',
  303 => 'See Other',
  304 => 'Not Modified',
  305 => 'Use Proxy',
  306 => 'Reserved',
  307 => 'Temporary Redirect',
 
  400 => 'Bad Request',
  401 => 'Unauthorized',
  402 => 'Payment Required',
  403 => 'Forbidden',
  404 => 'Not Found',
  405 => 'Method Not Allowed',
  406 => 'Not Acceptable',
  407 => 'Proxy Authentication Required',
  408 => 'Request Timeout',
  409 => 'Conflict',
  410 => 'Gone',
  411 => 'Length Required',
  412 => 'Precondition Failed',
  413 => 'Request Entity Too Large',
  414 => 'Request-URI Too Long',
  415 => 'Unsupported Media Type',
  416 => 'Requested Range Not Satisfiable',
  417 => 'Expectation Failed',
  422 => 'Unprocessable Entity',
  423 => 'Locked',
  424 => 'Failed Dependency',
  426 => 'Upgrade Required',
 
  500 => 'Internal Server Error',
  501 => 'Not Implemented',
  502 => 'Bad Gateway',
  503 => 'Service Unavailable',
  504 => 'Gateway Timeout',
  505 => 'HTTP Version Not Supported',
  506 => 'Variant Also Negotiates',
  507 => 'Insufficient Storage',
  510 => 'Not Extended'
);

RIPE WHOIS

All the status codes are standard HTTP codes ( http://www.iana.org/assignments/http-status-codes ).

Clients should avoid any form of coupling with the the text/plain error message contained in response body since it may change between different releases of the API and is only intended as a starting point for indentifying the real causes of the exception event.

The following table gives a brief description of the mapping between standard Whois V.3 responses and the related REST services status codes. Consider this table as just an example of the error mapping strategy, it may change with future releases.

Clients will have to define error messages generic enough to represent the four main error conditions, that are Bad Request, Forbidden, Not Found and Internal Server Error.

For example a possible mapping for client side error messages may be:

Helpful HTTP Links

1. IANA registry
2. Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0)
3. Adobe Flash status code definitions (ie 408)
4. Microsoft Internet Information Server Status Codes and Sub-Codes
5. httplint
6. HTTP Headers, brief intro.
7. Common User-Agent Issues
8. [RFC2295] Holtman, K. and A.H. Mutz , " Transparent Content Negotiation in HTTP ", RFC 2295, March 1998.
9. [RFC2518] Goland, Y. , Whitehead, E. , Faizi, A. , Carter, S.R. , and D. Jensen , " HTTP Extensions for Distributed Authoring -- WEBDAV ", RFC 2518, February 1999.
10. [RFC2616] Fielding, R. , Gettys, J. , Mogul, J. , Frystyk, H. , Masinter, L. , Leach, P. , and T. Berners-Lee , " Hypertext Transfer Protocol -- HTTP/1.1 ", RFC 2616, June 1999.
11. [RFC2774] Nielsen, H. , Leach, P. , and S. Lawrence , " An HTTP Extension Framework ", RFC 2774, February 2000.
12. [RFC2817] Khare, R. and S. Lawrence, " Upgrading to TLS Within HTTP/1.1 ", RFC 2817, May 2000.
13. [RFC3229] Mogul, J., Krishnamurthy, B., Douglis, F., Feldmann, A., Goland, Y., van Hoff, A., and D. Hellerstein, " Delta encoding in HTTP ", RFC 3229, January 2002.
14. [RFC4918] Dusseault, L., Ed. , " HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV) ", RFC 4918, June 2007.

HTTP Status Codes and Htaccess ErrorDocuments originally appeared on AskApache.com

The php part of this article is based on my Advanced WordPress 404.php article from 2008. Many of the following ideas came out of the research performed to enumerate every single Apache ErrorDocument, including learning how to view the defaults and many cool tricks for htaccess.

The PHP HTTP ErrorDocument Handler

Just save this as /err.php or whatever. The best is to put it in a cgi-bin script-alias directory under your DOCUMENT_ROOT like /cgi-bin/e.php but most people don't know how. That way you can setup some advanced stuff in a /cgi-bin/.htaccess file. If you are interested in locking it down, I recommend reading Securing php.ini and php-cgi with .htaccess.

Advantages and Reasons for Using

Fast, HTTP Protocol Compliance, protection. If you are reading this article, you already know and just want to check out the code!

<?php
ob_start();
@set_time_limit(5);
@ini_set('memory_limit', '64M');
@ini_set('display_errors', 'Off');
error_reporting(0);
 
function print_error_page()
{
 
  $status_reason = array(
  100 => 'Continue',
  101 => 'Switching Protocols',
  102 => 'Processing',
  200 => 'OK',
  201 => 'Created',
  202 => 'Accepted',
  203 => 'Non-Authoritative Information',
  204 => 'No Content',
  205 => 'Reset Content',
  206 => 'Partial Content',
  207 => 'Multi-Status',
  226 => 'IM Used',
  300 => 'Multiple Choices',
  301 => 'Moved Permanently',
  302 => 'Found',
  303 => 'See Other',
  304 => 'Not Modified',
  305 => 'Use Proxy',
  306 => 'Reserved',
  307 => 'Temporary Redirect',
  400 => 'Bad Request',
  401 => 'Unauthorized',
  402 => 'Payment Required',
  403 => 'Forbidden',
  404 => 'Not Found',
  405 => 'Method Not Allowed',
  406 => 'Not Acceptable',
  407 => 'Proxy Authentication Required',
  408 => 'Request Timeout',
  409 => 'Conflict',
  410 => 'Gone',
  411 => 'Length Required',
  412 => 'Precondition Failed',
  413 => 'Request Entity Too Large',
  414 => 'Request-URI Too Long',
  415 => 'Unsupported Media Type',
  416 => 'Requested Range Not Satisfiable',
  417 => 'Expectation Failed',
  422 => 'Unprocessable Entity',
  423 => 'Locked',
  424 => 'Failed Dependency',
  426 => 'Upgrade Required',
  500 => 'Internal Server Error',
  501 => 'Not Implemented',
  502 => 'Bad Gateway',
  503 => 'Service Unavailable',
  504 => 'Gateway Timeout',
  505 => 'HTTP Version Not Supported',
  506 => 'Variant Also Negotiates',
  507 => 'Insufficient Storage',
  510 => 'Not Extended'
  );
 
  $status_msg = array(
  400 => "Your browser sent a request that this server could not understand.",
  401 => "This server could not verify that you are authorized to access the document requested.",
  402 => 'The server encountered an internal error or misconfiguration and was unable to complete your request.',
  403 => "You don't have permission to access %U% on this server.",
  404 => "We couldn't find <acronym title='%U%'>that uri</acronym> on our server, though it's most certainly not your fault.",
  405 => "The requested method is not allowed for the URL %U%.",
  406 => "An appropriate representation of the requested resource %U% could not be found on this server.",
  407 => "An appropriate representation of the requested resource %U% could not be found on this server.",
  408 => "Server timeout waiting for the HTTP request from the client.",
  409 => 'The server encountered an internal error or misconfiguration and was unable to complete your request.',
  410 => "The requested resource %U% is no longer available on this server and there is no forwarding address. Please remove all references to this resource.",
  411 => "A request of the requested method GET requires a valid Content-length.",
  412 => "The precondition on the request for the URL %U% evaluated to false.",
  413 => "The requested resource %U% does not allow request data with GET requests, or the amount of data provided in the request exceeds the capacity limit.",
  414 => "The requested URL's length exceeds the capacity limit for this server.",
  415 => "The supplied request data is not in a format acceptable for processing by this resource.",
  416 => 'Requested Range Not Satisfiable',
  417 => "The expectation given in the Expect request-header field could not be met by this server. The client sent <code>Expect:</code>",
  422 => "The server understands the media type of the request entity, but was unable to process the contained instructions.",
  423 => "The requested resource is currently locked. The lock must be released or proper identification given before the method can be applied.",
  424 => "The method could not be performed on the resource because the requested action depended on another action and that other action failed.",
  425 => 'The server encountered an internal error or misconfiguration and was unable to complete your request.',
  426 => "The requested resource can only be retrieved using SSL. Either upgrade your client, or try requesting the page using https://",
  500 => 'The server encountered an internal error or misconfiguration and was unable to complete your request.',
  501 => "This type of request method to %U% is not supported.",
  502 => "The proxy server received an invalid response from an upstream server.",
  503 => "The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.",
  504 => "The proxy server did not receive a timely response from the upstream server.",
  505 => 'The server encountered an internal error or misconfiguration and was unable to complete your request.',
  506 => "A variant for the requested resource <code>%U%</code> is itself a negotiable resource. This indicates a configuration error.",
  507 => "The method could not be performed.  There is insufficient free space left in your storage allocation.",
  510 => "A mandatory extension policy in the request is not accepted by the server for this resource."
  );
 
  // Get the Status Code
  if (isset($_SERVER['REDIRECT_STATUS']) && ($_SERVER['REDIRECT_STATUS'] != 200))$sc = $_SERVER['REDIRECT_STATUS'];
  elseif (isset($_SERVER['REDIRECT_REDIRECT_STATUS']) && ($_SERVER['REDIRECT_REDIRECT_STATUS'] != 200)) $sc = $_SERVER['REDIRECT_REDIRECT_STATUS'];
  $sc = (!isset($_GET['error']) ? 404 : $_GET['error']);
 
  $sc=abs(intval($sc));
 
  // Redirect to server home if called directly or if status is under 400
  if( ( (isset($_SERVER['REDIRECT_STATUS']) && $_SERVER['REDIRECT_STATUS'] == 200) && (floor($sc / 100) == 3) )
     || (!isset($_GET['error']) && $_SERVER['REDIRECT_STATUS'] == 200)  )
  {
      @header("Location: http://{$_SERVER['SERVER_NAME']}",1,302);
      die();
  }
 
  // Check range of code or issue 500
  if (($sc < 200) || ($sc > 599)) $sc = 500;
 
  // Check for valid protocols or else issue 505
  if (!in_array($_SERVER["SERVER_PROTOCOL"], array('HTTP/1.0','HTTP/1.1','HTTP/0.9'))) $sc = 505;
 
  // Get the status reason
  $reason = (isset($status_reason[$sc]) ? $status_reason[$sc] : '');
 
  // Get the status message
  $msg = (isset($status_msg[$sc]) ? str_replace('%U%', htmlspecialchars(strip_tags(stripslashes($_SERVER['REQUEST_URI']))), $status_msg[$sc]) : 'Error');
 
  // issue optimized headers (optimized for your server)
  @header("{$_SERVER['SERVER_PROTOCOL']} {$sc} {$reason}", 1, $sc);
  if( @php_sapi_name() != 'cgi-fcgi' ) @header("Status: {$sc} {$reason}", 1, $sc);
 
  // A very small footprint for certain types of 4xx class errors and all 5xx class errors
  if (in_array($sc, array(400, 403, 405)) || (floor($sc / 100) == 5))
  {
    @header("Connection: close", 1);
    if ($sc == 405) @header('Allow: GET,HEAD,POST,OPTIONS', 1, 405);
  }
 
  echo "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html>";
  echo "<head>\n<title>{$sc} {$reason}</title>\n<h1>{$reason}</h1>\n<p>{$msg}<br />\n</p>\n";
}
 
function askapache_global_debug()
{
  # http://www.php.net/manual/en/function.array-walk.php#100681
  global $_GET,$_POST,$_ENV,$_SERVER;  $g=array('_ENV','_SERVER','_GET','_POST');
  array_walk_recursive($g, create_function('$n','global $$n;if( !!$$n&&ob_start()&&(print "[ $"."$n ]\n")&&array_walk($$n,
    create_function(\'$v,$k\', \'echo "[$k] => $v\n";\'))) echo "<"."p"."r"."e>".htmlspecialchars(ob_get_clean())."<"."/"."pr"."e>";') );
}
 
print_error_page();
//if($_SERVER['REMOTE_ADDR']=='youripaddress')askapache_global_debug();
echo "</body>\n</html>";
echo ob_get_clean();
exit;
?>

Note: If you are installing this on a non-linux/non-apache machine/server, you will need to read your products documentation for custom error documents. It will work on any machine that can run php.

Htaccess ErrorDocument Tips

The thing is, how do you setup your website to use this php file to be able to handle all those HTTP Status Codes gracefully? You just need to configure your server to use that php file for any Status Codes you want. If you are building an ErrorDocument handling system for a server-wide, multi-site, setup, you will want to instead use the method I use. Instead of using a separate language like PHP, Python, Ruby, Perl, etc, to handle errors, I rely on the very safe and fast SSI method. I detailed the advanced ErrorDocument SSI (.htaccess or httpd.conf)..

If you instead like most of us, you will be setting this up for 1 site, or 1 DOCUMENT_ROOT serving virtual hosts. For that the best method is to modify your .htaccess file.

Using Redirect in .htaccess to trigger Errors

This is one of my all time favorite discoveries from my apache studies. It's documented elsewhere on this site, but if you want to test it out, just request www.askapache.com/show-error-402. Of course the error handling that I have in place is quite nice.

Redirect 400 /show-error-400
Redirect 401 /show-error-401
Redirect 402 /show-error-402
Redirect 403 /show-error-403
Redirect 405 /show-error-405
Redirect 406 /show-error-406
Redirect 407 /show-error-407
Redirect 408 /show-error-408
Redirect 409 /show-error-409
Redirect 410 /show-error-410
Redirect 411 /show-error-411
Redirect 412 /show-error-412
Redirect 413 /show-error-413
Redirect 414 /show-error-414
Redirect 415 /show-error-415
Redirect 416 /show-error-416
Redirect 417 /show-error-417
Redirect 418 /show-error-418
Redirect 419 /show-error-419
Redirect 420 /show-error-420
Redirect 421 /show-error-421
Redirect 422 /show-error-422
Redirect 423 /show-error-423
Redirect 424 /show-error-424
Redirect 425 /show-error-425
Redirect 426 /show-error-426
Redirect 500 /show-error-500
Redirect 501 /show-error-501
Redirect 502 /show-error-502
Redirect 503 /show-error-503
Redirect 504 /show-error-504
Redirect 505 /show-error-505
Redirect 506 /show-error-506
Redirect 507 /show-error-507
Redirect 508 /show-error-508
Redirect 509 /show-error-509
Redirect 510 /show-error-510

Powerful Mod_Rewrite Trick

Here's how to combine the power of mod_rewrites ability to parse requests and environment variables with the above Redirect trick to trigger a specific ErrorDocument based on the query_string parameter error. This trick is only on AskApache.com, very powerful trick if you need to force ErrorDocuments.

RewriteCond %{QUERY_STRING} error=([4|5][0-9][0-9]) [NC]
RewriteCond %{QUERY_STRING} !404
RewriteRule . /show-error-%1 [L]

ErrorDocument Example for .htaccess

So if you save the php file as err.php in your DOCUMENT_ROOT, these are the htaccess commands that will enable its use.

The addition of the ?error=num should be unneccessary on a good linux machine, it's a way for lesser OS's and webhosts to still be able to use errordocuments. Basically Apache handles ErrorDocuments by setting special DEBUGGING variables (Start with REDIRECT_) so it's very easy to determine the STATUS CODE by just viewing $_SERVER['REDIRECT_STATUS']. If a recursive type of redirect is going on, you may see $_SERVER['REDIRECT_REDIRECT_STATUS']. Dumb (consistently) OS's like a Windows server almost always have problems with things like that, because they don't give a hoot about POSIX or standards, why should they when no one can view their code anyway. Here are some more details on the REDIRECT_STATUS and other ways to use these variables.

If you want to learn how to enumerate and view the different variables that are in your Apache environment, I think I have the best tutorial on the planet for how to do this with PHP and mod_rewrite with mod_headers. That article is the basis for anyone who is hired to do mod_rewrites on a new server without root access. I would say that one article will inform you more about mod_rewrite then any other article on this site.

###
# ErrorDocument: In the event of a problem or error, what the server will return to the client. URLs
# can begin with a / for local web-paths (relative to DocumentRoot), or be a full URL which the client
# can resolve. Alternatively, a message can be displayed.  If a malformed request is detected, normal
# request processing will be immediately halted and the internal error message returned.
#
# Prior to version 2.0, messages were indicated by prefixing them with a
# single unmatched double quote character.
#
# The special value default can be used to specify Apache's simple hardcoded message and
# will restore Apache's simple hardcoded message.
#
ErrorDocument 400 /err.php?error=400
ErrorDocument 401 /err.php?error=401
ErrorDocument 402 /err.php?error=402
ErrorDocument 403 /err.php?error=403
ErrorDocument 404 /err.php?error=404
ErrorDocument 405 /err.php?error=405
ErrorDocument 406 /err.php?error=406
ErrorDocument 407 /err.php?error=407
ErrorDocument 408 /err.php?error=408
ErrorDocument 409 /err.php?error=409
ErrorDocument 410 /err.php?error=410
ErrorDocument 411 /err.php?error=411
ErrorDocument 412 /err.php?error=412
ErrorDocument 413 /err.php?error=413
ErrorDocument 414 /err.php?error=414
ErrorDocument 415 /err.php?error=415
ErrorDocument 416 /err.php?error=416
ErrorDocument 417 /err.php?error=417
ErrorDocument 422 /err.php?error=422
ErrorDocument 423 /err.php?error=423
ErrorDocument 424 /err.php?error=424
ErrorDocument 426 /err.php?error=426
ErrorDocument 500 /err.php?error=500
ErrorDocument 501 /err.php?error=501
ErrorDocument 502 /err.php?error=502
ErrorDocument 503 /err.php?error=503
ErrorDocument 504 /err.php?error=504
ErrorDocument 505 /err.php?error=505
ErrorDocument 506 /err.php?error=506
ErrorDocument 507 /err.php?error=507
ErrorDocument 510 /err.php?error=510

PHP to handle HTTP Status Codes for ErrorDocument originally appeared on AskApache.com

A large client has a secure website where they assemble and create presentations consisting of a single Table of Contents page with many pdf's attached to it. They use the site to make presentations. This is a big client, a rich client, and they needed a way to guarantee they would be able to access the site. So I got this request:

Can you make an offline version of the Page that is always updated and available for download so we can download the offline version and present using that in case the website is down or more often, in case Internet Access is unavailable?

Update: I used COOKIE based authentication to secure this clients site, so that only logged in users can see anything at all, so how do I enable the curl requests to authenticate as well using the COOKIE of the requesting user in each request made by curl? Just add the users HTTP_COOKIE to the headers array used by curl like so:

array(
...
"Cookie: {$_SERVER['HTTP_COOKIE']}"
)

That now means the scraped version of the page is an exact duplicate that the user is looking at. Very sweet!

I can GET anything

So, here's what I hacked together last night, that is being used today. It's essentially 2 files.

1. A php file that scrapes uses curl to scrape all the urls for the page (favicon, css, images, pdfs, etc..)
2. A simple bash shell script acting as a cgi that creates a zip file of all the urls, and a self-extracting exe file for those without a winzip tool

The PHP File

This is a simple script that is given 2 parameters:

1. The url to scrape
2. The type of download to return

scrapeit.php

<?php
ob_start();
 
/**
* gogeturl2() - grabs a url with curl, and saves it to disk, works for all media types, pdf, js, img, etc.
*
* @return
*/
function gogeturl2($url, $saveto)
{
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
if ($fp = fopen($saveto, 'w')) // {curl_setopt($ch,CURLOPT_STDERR, $fp); curl_setopt($ch,CURLOPT_VERBOSE,1);}
{
  curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
  curl_setopt($ch, CURLOPT_MAXCONNECTS, 4);
  curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
  curl_setopt($ch, CURLOPT_FAILONERROR, 1);
  curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
  curl_setopt($ch, CURLOPT_FILE, $fp);
  curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
  curl_setopt($ch, CURLOPT_HTTPHEADER, array(
    "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3",
    "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
    "Accept-Language: en-us,en;q=0.5",
    "Accept-Encoding: none",
    "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7",
    "Keep-Alive: 300",
    "Connection: Keep-Alive",
    "Pragma:"));
  $r = curl_exec($ch);
  $ch_info = curl_getinfo($ch);
  if (curl_errno($ch)) error_log(print_r($ch_info, 1) . print_r(curl_errno($ch), 1) . print_r(curl_error($ch), 1));
  else curl_close($ch);
  fclose($fp);
}
}
/**
* gogeturl()  returns the source of the requested url (thanks to accept-encoding: none)
*
* @return
*/
function gogeturl($url)
{
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_MAXCONNECTS, 4);
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
  "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 GTB6",
  "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
  "Accept-Language: en-us,en;q=0.5",
  "Accept-Encoding: none",
  "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7",
  "Keep-Alive: 115",
  "Connection: keep-alive",
  "Pragma:"));
$g = curl_exec($ch);
if (curl_errno($ch)) error_log(print_r(array(
'chinfo' => $ch_info,
'curl_errno' => curl_errno($ch),
'curl_error' => curl_error($ch)
), 1));
curl_close($ch);
return $g;
}
 
/**
* _mkdir() makes a directory
*
* @return
*/
function _mkdir($path, $mode = 0755)
{
$old = umask(0);
$res = @mkdir($path, $mode);
umask($old);
return $res;
}
 
/**
* rmkdir()  recursively makes a directory tree
*
* @return
*/
function rmkdir($path, $mode = 0755)
{
return is_dir($path) || (rmkdir(dirname($path), $mode) && _mkdir($path, $mode));
}

The following should be in a couple functions, but I was running on a tight time schedule, and hey this $hitt aint free... wait yes it is, always.

// Ok lets get it on!
// first lets setup some variables
if (!isset($_GET['url']) || empty($_GET['url']))die();
$td = $th = $urls = array();
$FDATE = date("m-d-y-Hms");
$FTMP = '/home/askapache/sites/askapache.com/tmp';
$fetch_url = $_GET['url'];
$fu = parse_url($fetch_url);
$fd = substr($FTMP . $fu['path'], 0, - 1);
$FEXE = "{$fd}-{$FDATE}.exe";
$FZIP = "{$fd}-{$FDATE}.zip";
 
// now this is a shortcut to download the css file and add all the images in it to the img_urls array
$img_urls = array();
$gg = preg_match_all("/url\(([^\)]*?)\)/Ui", gogeturl('https://www.askapache.com/askapache-0128770124.css'), $th);
$imgs = array_unique($th[1]);
foreach($imgs as $img)
{
// only because all the links are relative
$img_urls[] = 'https://www.askapache.com' . $img;
}
 
// now fetch the main page, and assemble an array of all the external resources into the urls array
$gg = preg_match_all("/(background|href)=([\"\'])([^\"\'#]+?)([\"\'])/Ui", gogeturl($fetch_url), $th);
foreach($th[3] as $url)
{
if (strpos($url, '.js') !== false)continue;
if (strpos($url, 'wp-login.php') !== false || $url == 'https://www.askapache.com/') continue;
if (strrpos($url, '/') == strlen($url) - 1)continue;
if (strpos($url, 'https://www.askapache.com/') === false)
{
  if ($url[0] == '/') $urls[] = 'https://www.askapache.com' . $url;
  else continue;
}
else $urls[] = $url;
}
 
// now create a uniq array of urls, then download and save each of them
$urls = array_merge(array_unique($img_urls), array_unique($urls));
foreach($urls as $url)
{
  $pu = parse_url($url);
  rmkdir(substr($fd . $pu['path'], 0, strrpos($fd . $pu['path'], '/')));
  gogeturl2($url, $fd . $pu['path']);
}
 
// deletes dir ie. /this-page/this-page/ when it should be /this-page/index.html
if (is_dir($fd . $fu['path'])) rmdir($fd . $fu['path']);
 
// now save the page as index.html
gogeturl2($fetch_url, $fd . '/index.html');
 
// fixup to be able3 to parse
$g = file_get_contents($fd . '/index.html');
$g = str_replace('<script', '<!--<script', $g);
$g = str_replace('script>', 'script>!-->', $g);
$g = str_replace('href="https://www.askapache.com/', 'href="/', $g);
$g = str_replace('src="https://www.askapache.com/', 'src="/', $g);
$g = str_replace('href="/', 'href="', $g);
$g = str_replace('src="/', 'src="', $g);
$g = str_replace("href='https://www.askapache.com/", "href='/", $g);
$g = str_replace("src='https://www.askapache.com/", "src='/", $g);
$g = str_replace("href='/", "href='", $g);
$g = str_replace("src='/", "src='", $g);
file_put_contents($fd . '/index.html', $g);
 
// fixup for css file
foreach($urls as $url)
{
if (strpos($url, '.css') !== false)
{
  $fuu = parse_url($url);
  $css = file_get_contents($fd . $fuu['path']);
  $css = str_replace('url(/', 'url(../', $css);
  file_put_contents($fd . $fuu['path'], $css);
}
}
 
// my favorite technique, using fsockopen to initiate a shell script server-side.
// passing the args in the HTTP Headers... genius!!
// close the sucker fast with HTTP/1.0 and connection: close
$fp = fsockopen ($_SERVER['SERVER_NAME'], 80, $errno, $errstr, 5);
fwrite($fp, "GET /cgi-bin/sh/zip.sh HTTP/1.0\r\nHost: www.askapache.com\r\nX-Pad: {$fd}\r\nX-Allow: {$FDATE}\r\nConnection: Close\r\n\r\n");
fclose($fp);
 
// loop until the file created by /cgi-bin/sh/zip.sh is found
$c = 0;
do
{
$c++;
sleep(1);
clearstatcache();
if (is_file("{$FEXE}")) continue;
}
while ($c < 20);
 
// either zip or exe
$type = $_GET['type'];
if ($type == 'zip') $file = $FZIP;
else $file = $FEXE;
 
// wow great debugging dude
error_log($file);
 
// if the file is there, do a 302 redirect to initiate download
if (file_exists("{$file}"))
{
  @header("HTTP/1.1 302 Moved", 1, 302);
  @header("Status: 302 Moved", 1, 302);
  @header('Location: https://www.askapache.com/' . basename($file));
  exit;
}
 
exit;
?>

zip.sh

#!/bin/sh
 
# all you need for cgi
  echo -e "Content-type: text/plain\n\n"
 
# blank that run log
  echo "" > /home/askapache/sites/askapache.com/tmp/run.log
 
# redirect 1 and 2 to the run log for the whole script
  exec &>/home/askapache/sites/askapache.com/tmp/run.log
 
# basename
  N=${HTTP_X_PAD//*\/}
 
# date-based
  NN=${HTTP_X_ALLOW}
 
# create recursively the dir tree
  mkdir -pv $HTTP_X_PAD
 
# cd to the tmp
  cd /home/askapache/sites/askapache.com/tmp
 
# the zip version with date
  F=$N-$NN.zip
 
# the exe version with date
  NN=$N-$NN.exe
 
# for debugging, only goes to run log
  echo "F=$F"
  echo "NN=$NN"
  echo "N=$N"
 
# create a relative (r is recursive) archive of the entire dir
  /usr/bin/zip -rvv $F $N
 
# add the self-extracting stub to the archive
  /bin/cat unzipsfx.exe $F > $NN
 
# fix the sfx stub
  /usr/bin/zip -A $NN
 
# move both the exe and zip to the web-docroot to be dl'd directly
  cp -vf $NN /home/askapache/sites/askapache.com/htdocs/
  cp -vf $F /home/askapache/sites/askapache.com/htdocs/
 
# sleep for 60 seconds and then rm all the files, so you better download that file fast
  sleep 60 && rm -rvf $HTTP_X_PAD $F $NN /home/askapache/sites/askapache.com/htdocs/$NN /home/askapache/sites/askapache.com/htdocs/$F
 
#suh suh cya
exit 0;

Creating SFX Archives

The best way is 7z, but I couldn't get p7zip's sfx module to work and didn't have time to compile it. Instead I just used the stub available here: curl -O -L ftp://ftp.info-zip.org/pub/infozip/win32/unz552xn.exe which works great but no customization like password, icons, etc.. oh well.

Implementation

Simple, just create a link to the php file with the url and type parameters. /cgi-bin/php/scrapeit.php?url=http://www.askapache.com/htaccess/htaccess.html&type=exe or if you integrate into wordpress like I did you can add this to your header or admin_bar and use the get_permalink() for the url arg.

Lock It Down

This was used on a private site so what I did was add some code to the scrapeit.php file that just copied the HTTP_COOKIE value sent by the requesting user and sent that as part of the request in the fsockopen request. That means only logged in users can do this, and furthermore, if a user doesn't have access to a page and tries to use this to circumvent, they can't. And also htaccess was used to limit the scripts to only allow the ip's running the server to make connections.

Conclusion

What can't be done with linux, bash, http, php, and a little server-side finesse? My clients are very happy, and I had some fun!

Creating an Offline Version of Page originally appeared on AskApache.com