This is part 2 of the exhaustive Htaccess Tutorial. I realized it was so lengthy that search engines and visitors were having real problems with it, so I moved half of it here. But this gave me the opportunity to add a ton of new stuff that I hadn't been able to add to the main htaccess tutorial. And now this new part 2 is already twice as big as the original! To sum up, this is a work in progress.

Moving Half of Original Tutorial to a new URL - SEO Rewriting

Well, the original tutorial was at /htaccess/apache-htaccess.html, but it was such a huge article that search engines were dropping it! So I split it into 3 new urls.

1. /htaccess/htaccess.html - 1
2. /htaccess/htaccess-rewrites.html - Rewrites
3. /htaccess/apache-htaccess-2.html - Part 2

The /htaccess/apache-htaccess-2.html is a lot of unfinished new stuff, and the /htaccess/htaccess-rewrites.html (this very page) still needs to be split several times to get the filesize down.

301 Link Juice

The /apache-htaccess.html url has been around since 2006, and it was full of original ideas and examples for using htaccess. Almost every htaccess guide or tutorial published since then has many of the examples and ideas from that tutorial, (I encourage people to modify and republish everything on this site according to copyright). I know it, the authors sometimes know it, but the main point is Google for sure knows it. That's the beauty of creating unique content, Google sees that. So that /apache-htaccess.html link has extreme link juice, from all the sites, books, papers, and presentations that have linked to it since 2006.

Filesize Is important

I like to use my blog as a way to keep notes about my research organized, and I am extremely good at doing research, unfortunately, that means I have a huge article. The filesize for the html alone is larger than all the other resources like javascript and images, combined.

• So that means it is very difficult to view on a mobile device, or a slow connection.
• For google-bot and other search engine crawlers and robots, this is a huge problem (I made a big mistake letting it get that size).
• The robot has to parse an enormous single html file, containing hundreds/thousands of external and internal links, and its such an issue they can decide to just skip indexing that url until the filesize is manageable.
• Once again, it's all about the human experience, a huge single file is not good for anyone who isn't printing it out to read offline.

Starting Fresh without losing juice

So since the page wasn't being indexed much since it was so huge, I decided to split up the content into new separate urls and utilize a 301 Redirect to transfer all the link juice from /apache-htaccess.html to /htaccess.html.

301 Redirect Timeline

1. Now, the idea is to take the first pages from the original multi-page guide, do a little improvement on that content, and save it to the new url /htaccess.html.
2. Then just continue taking the next page from the remaining original guide and creating new pages using the original links structure. /apache-htaccess-2.html, /apache-htaccess-3.html, etc.. This is a secondary backup to the new /htaccess.html url, which will receive the 301 link juice from the old url, but these secondary pages will help keep the links on external sites good.
3. Then, I setup a 301 Redirect in my .htaccess file to redirect the old url to the new /htaccess.html url.
4. Finally, I delete the old url and it is replaced forever by a 301 Redirect pointing to my new location, filesize problems eliminated.

301 Redirect Code Used

This is so easy to do with RedirectMatch, way faster and easier than using mod_rewrite to handle this, and much less overhead. Note that this is a general command that I will leave up for a few weeks and slowly tighten it up by looking at my Google Analytics and Apache Logs. For instance, this first redirectmatch rule is an older RedirectMatch I still have active to redirect all the old links pointing at /2006/htaccess/apache-htaccess.html to /htaccess/apache-htaccess.html from when I ditched the date-based permalinks back in 2007, and you can see it is a little tighter than the 2nd one which also redirects requests for apache-htaccess.html/feed/ or trackback or whatever.

RedirectMatch 301 ^/2006/.*apache-htaccess.html$ http://www.askapache.com/htaccess/apache-htaccess.html
RedirectMatch 301 ^/.*apache-htaccess.html(.*)$ http://www.askapache.com/htaccess/htaccess.html$1

Advanced 301 Redirects for SEO

I wrote a couple of articles that go into detail about maximizing the SEO with linking and redirects, it remains one of my most helpful articles to anyone trying to rank higher the right way, the Google way.. SEO Secrets of AskApache Part 2

Htaccess Rewrites for Moving Urls originally appeared on AskApache.com

Are you an advanced mod_rewrite expert or guru? This article is for YOU too! Just make sure to read all the way to the bottom..

The following undocumented techniques and methods will allow you to utilize mod_rewrite at an "expert level" by showing you how to unlock its secrets.

Most if not all web developers and server administrators struggle with Apache mod_rewrite. It's very tough and only gets a little easier with practice. Until Now! Get ready to explode your learning curve, I figured something out.

Why mod_rewrite is so tough

I have come to the conclusion, after many hours of zenful thought, that the reason mod_rewrite is so tough is pretty obvious, people are trying to apply regular-expressions to URLs and Variables that they don't really understand. They understand what they want, but they don't understand what the URLS and Variables are that they are trying to rewrite.

Hit-Or-Miss with mod_rewrite

A lot of the mod_rewrite "experts" and "gurus" floating around the net absolutely know their mod_rewrite, but what separates them from a beginner or novice is for the most part an understanding of what the URLS and Variables look like that are targeted by the regular expressions. Take this simple rewriterule that rewrites requests made without the www to www.

RewriteEngine On
RewriteBase /
 
RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC]
RewriteRule .+ http://www.askapache.com%{REQUEST_URI}

Pretty simple right? WRONG. Most people could not figure that out..

Why?

The reason intelligent people can't figure that out is because they have no idea what HTTP_HOST or REQUEST_URI actually looks like. How can you write a rule for something if you don't know what it looks like? You can't.

When Not To Use Mod_Rewrite

Ok so heres an important concept that alot of people haven't heard. You should only use mod_rewrite's rewriterule when you use a rewritecond or if you are rewriting internally like my feedcount hack.

If you are simply redirecting one url to another, you should definately be using the much easier mod_alias's redirect and redirectmatch, which is enabled on most Apache servers.

When To Use Mod_Rewrite

So then, you should only use mod_rewrite's rewriterule when you are checking against one of the Apache Environment Variables to determine whether to rewrite or not. This is where the Apache Documentation is grossly lacking. They don't tell you what those variables look like, leaving us completely incapable of creating rewrites based on them. Not anymore.

Mod_Rewrite Environment Variables (The Secret)

Here's the variables I have found accessible by mod_rewrite (both documented and undocumented). A thing to note is that you can set these variables early in an .htaccess file using SetEnv, RewriteRule, Header, etc.. and they will be accessible at the end of the .htaccess file.

Decoding Mod_Rewrite Variables

So when I realized my problem was that I didn't know the value of the variable being tested by the RewriteCond, I set out to try and discover how to view those variables.. Keep in mind you can also use RewriteLogging, but its only allowed for root users who can edit the httpd.conf, this is .htaccess.

Setting Environment Variables with RewriteRule

I discovered a multitude of methods to set and view apache environment variables, using various modules and some core tricks, but the method that allows me to view the most environment variables is RewriteRule.. I wanted to use SetEnvIf more, but its just not as powerful as mod_rewrite, due to programming.

This code sets the variable INFO_REQUEST_URI to have the value of REQUEST_URI.

RewriteEngine On
RewriteBase /
RewriteRule .* - [E=INFO_REQUEST_URI:%{REQUEST_URI},NE]

Saving the Apache Variable Values

Now the trick is how to view that environment variable... The method I came up with is nice... We will send the environment variable value in an HTTP Header, as there isn't much data manipulation/validation so you get an accurate look at the actual value.. At first I tried adding the variable value to a redirection using the query_string.. but a HTTP_USER_AGENT value doesn't play well as a query_string.

Using RequestHeader in .htaccess

This code takes advantage of the incredible mod_headers apache module to actually ADD a whole new header to YOUR request. Seriously one of the coolest tricks I've found yet.. Its almost the same as being able to spoof POST requests! Since Headers can be protected data... especially the HTTP_COOKIE header..

RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"

Viewing the Variable Values

Now you can use any kind of server-run interpreter like perl, php, ruby, etc., to view all the variable values. All cgi-script handlers like those are able to view request headers..

PHP Code to access Apache Variables

Works even in safe-mode... any interpreter can view HTTP Headers! Note that each of these variables are added as HTTP headers to the request for the script.. kinda confusing.. So each variable sent as a header is prefixed with HTTP_ to denote it was a header.

<?php
header("Content-Type: text/plain");
$INFO=$MISS=array();
foreach($_SERVER as $v=>$r)
{
  if(substr($v,0,9)=='HTTP_INFO')
  {
    if(!empty($r))$INFO[substr($v,10)]=$r;
    else $MISS[substr($v,10)]=$r;
  }
}
 
/* thanks Mike! */
ksort($INFO);
ksort($MISS);
ksort($_SERVER);
 
echo "Received These Variables:\n";
print_r($INFO);
 
echo "Missed These Variables:\n";
print_r($MISS);
 
echo "ALL Variables:\n";
print_r($_SERVER);
?>

Time to Get Crazy

Just create the above php file on your site as /test/index.php or whatever, then create /test/.htaccess which should contain the below .htaccess file snippet. Now just request /test/index.php and be amazed! If you're looking for more general help check out this excellent mod_rewrite cheat sheet.

Ok, so I've prepared the .htaccess code you can use to view the values of all these variables. Just add it to a .htaccess file and make a request. For this test I created an index.php file that printed out all the $_SERVER variables, and made requests to it.

RewriteEngine On
RewriteBase /
RewriteRule .* - [E=INFO_API_VERSION:%{API_VERSION},NE]
RewriteRule .* - [E=INFO_AUTH_TYPE:%{AUTH_TYPE},NE]
RewriteRule .* - [E=INFO_CONTENT_LENGTH:%{CONTENT_LENGTH},NE]
RewriteRule .* - [E=INFO_CONTENT_TYPE:%{CONTENT_TYPE},NE]
RewriteRule .* - [E=INFO_DOCUMENT_ROOT:%{DOCUMENT_ROOT},NE]
RewriteRule .* - [E=INFO_GATEWAY_INTERFACE:%{GATEWAY_INTERFACE},NE]
RewriteRule .* - [E=INFO_HTTPS:%{HTTPS},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT:%{HTTP_ACCEPT},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_CHARSET:%{HTTP_ACCEPT_CHARSET},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_ENCODING:%{HTTP_ACCEPT_ENCODING},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_LANGUAGE:%{HTTP_ACCEPT_LANGUAGE},NE]
RewriteRule .* - [E=INFO_HTTP_CACHE_CONTROL:%{HTTP_CACHE_CONTROL},NE]
RewriteRule .* - [E=INFO_HTTP_CONNECTION:%{HTTP_CONNECTION},NE]
RewriteRule .* - [E=INFO_HTTP_COOKIE:%{HTTP_COOKIE},NE]
RewriteRule .* - [E=INFO_HTTP_FORWARDED:%{HTTP_FORWARDED},NE]
RewriteRule .* - [E=INFO_HTTP_HOST:%{HTTP_HOST},NE]
RewriteRule .* - [E=INFO_HTTP_KEEP_ALIVE:%{HTTP_KEEP_ALIVE},NE]
RewriteRule .* - [E=INFO_HTTP_MOD_SECURITY_MESSAGE:%{HTTP_MOD_SECURITY_MESSAGE},NE]
RewriteRule .* - [E=INFO_HTTP_PROXY_CONNECTION:%{HTTP_PROXY_CONNECTION},NE]
RewriteRule .* - [E=INFO_HTTP_REFERER:%{HTTP_REFERER},NE]
RewriteRule .* - [E=INFO_HTTP_USER_AGENT:%{HTTP_USER_AGENT},NE]
RewriteRule .* - [E=INFO_IS_SUBREQ:%{IS_SUBREQ},NE]
RewriteRule .* - [E=INFO_ORIG_PATH_INFO:%{ORIG_PATH_INFO},NE]
RewriteRule .* - [E=INFO_ORIG_PATH_TRANSLATED:%{ORIG_PATH_TRANSLATED},NE]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_FILENAME:%{ORIG_SCRIPT_FILENAME},NE]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_NAME:%{ORIG_SCRIPT_NAME},NE]
RewriteRule .* - [E=INFO_PATH:%{PATH},NE]
RewriteRule .* - [E=INFO_PATH_INFO:%{PATH_INFO},NE]
RewriteRule .* - [E=INFO_PHP_SELF:%{PHP_SELF},NE]
RewriteRule .* - [E=INFO_QUERY_STRING:%{QUERY_STRING},NE]
RewriteRule .* - [E=INFO_REDIRECT_QUERY_STRING:%{REDIRECT_QUERY_STRING},NE]
RewriteRule .* - [E=INFO_REDIRECT_REMOTE_USER:%{REDIRECT_REMOTE_USER},NE]
RewriteRule .* - [E=INFO_REDIRECT_STATUS:%{REDIRECT_STATUS},NE]
RewriteRule .* - [E=INFO_REDIRECT_URL:%{REDIRECT_URL},NE]
RewriteRule .* - [E=INFO_REMOTE_ADDR:%{REMOTE_ADDR},NE]
RewriteRule .* - [E=INFO_REMOTE_HOST:%{REMOTE_HOST},NE]
RewriteRule .* - [E=INFO_REMOTE_IDENT:%{REMOTE_IDENT},NE]
RewriteRule .* - [E=INFO_REMOTE_PORT:%{REMOTE_PORT},NE]
RewriteRule .* - [E=INFO_REMOTE_USER:%{REMOTE_USER},NE]
RewriteRule .* - [E=INFO_REQUEST_FILENAME:%{REQUEST_FILENAME},NE]
RewriteRule .* - [E=INFO_REQUEST_METHOD:%{REQUEST_METHOD},NE]
RewriteRule .* - [E=INFO_REQUEST_TIME:%{REQUEST_TIME},NE]
RewriteRule .* - [E=INFO_REQUEST_URI:%{REQUEST_URI},NE]
RewriteRule .* - [E=INFO_SCRIPT_FILENAME:%{SCRIPT_FILENAME},NE]
RewriteRule .* - [E=INFO_SCRIPT_GROUP:%{SCRIPT_GROUP},NE]
RewriteRule .* - [E=INFO_SCRIPT_NAME:%{SCRIPT_NAME},NE]
RewriteRule .* - [E=INFO_SCRIPT_URI:%{SCRIPT_URI},NE]
RewriteRule .* - [E=INFO_SCRIPT_URL:%{SCRIPT_URL},NE]
RewriteRule .* - [E=INFO_SCRIPT_USER:%{SCRIPT_USER},NE]
RewriteRule .* - [E=INFO_SERVER_ADDR:%{SERVER_ADDR},NE]
RewriteRule .* - [E=INFO_SERVER_ADMIN:%{SERVER_ADMIN},NE]
RewriteRule .* - [E=INFO_SERVER_NAME:%{SERVER_NAME},NE]
RewriteRule .* - [E=INFO_SERVER_PORT:%{SERVER_PORT},NE]
RewriteRule .* - [E=INFO_SERVER_PROTOCOL:%{SERVER_PROTOCOL},NE]
RewriteRule .* - [E=INFO_SERVER_SIGNATURE:%{SERVER_SIGNATURE},NE]
RewriteRule .* - [E=INFO_SERVER_SOFTWARE:%{SERVER_SOFTWARE},NE]
RewriteRule .* - [E=INFO_THE_REQUEST:%{THE_REQUEST},NE]
RewriteRule .* - [E=INFO_TIME:%{TIME},NE]
RewriteRule .* - [E=INFO_TIME_DAY:%{TIME_DAY},NE]
RewriteRule .* - [E=INFO_TIME_HOUR:%{TIME_HOUR},NE]
RewriteRule .* - [E=INFO_TIME_MIN:%{TIME_MIN},NE]
RewriteRule .* - [E=INFO_TIME_MON:%{TIME_MON},NE]
RewriteRule .* - [E=INFO_TIME_SEC:%{TIME_SEC},NE]
RewriteRule .* - [E=INFO_TIME_WDAY:%{TIME_WDAY},NE]
RewriteRule .* - [E=INFO_TIME_YEAR:%{TIME_YEAR},NE]
RewriteRule .* - [E=INFO_TZ:%{TZ},NE]
RewriteRule .* - [E=INFO_UNIQUE_ID:%{UNIQUE_ID},NE]
 
RequestHeader set INFO_API_VERSION "%{INFO_API_VERSION}e"
RequestHeader set INFO_AUTH_TYPE "%{INFO_AUTH_TYPE}e"
RequestHeader set INFO_CONTENT_LENGTH "%{INFO_CONTENT_LENGTH}e"
RequestHeader set INFO_CONTENT_TYPE "%{INFO_CONTENT_TYPE}e"
RequestHeader set INFO_DOCUMENT_ROOT "%{INFO_DOCUMENT_ROOT}e"
RequestHeader set INFO_GATEWAY_INTERFACE "%{INFO_GATEWAY_INTERFACE}e"
RequestHeader set INFO_HTTPS "%{INFO_HTTPS}e"
RequestHeader set INFO_HTTP_ACCEPT "%{INFO_HTTP_ACCEPT}e"
RequestHeader set INFO_HTTP_ACCEPT_CHARSET "%{INFO_HTTP_ACCEPT_CHARSET}e"
RequestHeader set INFO_HTTP_ACCEPT_ENCODING "%{INFO_HTTP_ACCEPT_ENCODING}e"
RequestHeader set INFO_HTTP_ACCEPT_LANGUAGE "%{INFO_HTTP_ACCEPT_LANGUAGE}e"
RequestHeader set INFO_HTTP_CACHE_CONTROL "%{INFO_HTTP_CACHE_CONTROL}e"
RequestHeader set INFO_HTTP_CONNECTION "%{INFO_HTTP_CONNECTION}e"
RequestHeader set INFO_HTTP_COOKIE "%{INFO_HTTP_COOKIE}e"
RequestHeader set INFO_HTTP_FORWARDED "%{INFO_HTTP_FORWARDED}e"
RequestHeader set INFO_HTTP_HOST "%{INFO_HTTP_HOST}e"
RequestHeader set INFO_HTTP_KEEP_ALIVE "%{INFO_HTTP_KEEP_ALIVE}e"
RequestHeader set INFO_HTTP_MOD_SECURITY_MESSAGE "%{INFO_HTTP_MOD_SECURITY_MESSAGE}e"
RequestHeader set INFO_HTTP_PROXY_CONNECTION "%{INFO_HTTP_PROXY_CONNECTION}e"
RequestHeader set INFO_HTTP_REFERER "%{INFO_HTTP_REFERER}e"
RequestHeader set INFO_HTTP_USER_AGENT "%{INFO_HTTP_USER_AGENT}e"
RequestHeader set INFO_IS_SUBREQ "%{INFO_IS_SUBREQ}e"
RequestHeader set INFO_ORIG_PATH_INFO "%{INFO_ORIG_PATH_INFO}e"
RequestHeader set INFO_ORIG_PATH_TRANSLATED "%{INFO_ORIG_PATH_TRANSLATED}e"
RequestHeader set INFO_ORIG_SCRIPT_FILENAME "%{INFO_ORIG_SCRIPT_FILENAME}e"
RequestHeader set INFO_ORIG_SCRIPT_NAME "%{INFO_ORIG_SCRIPT_NAME}e"
RequestHeader set INFO_PATH "%{INFO_PATH}e"
RequestHeader set INFO_PATH_INFO "%{INFO_PATH_INFO}e"
RequestHeader set INFO_PHP_SELF "%{INFO_PHP_SELF}e"
RequestHeader set INFO_QUERY_STRING "%{INFO_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_QUERY_STRING "%{INFO_REDIRECT_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_REMOTE_USER "%{INFO_REDIRECT_REMOTE_USER}e"
RequestHeader set INFO_REDIRECT_STATUS "%{INFO_REDIRECT_STATUS}e"
RequestHeader set INFO_REDIRECT_URL "%{INFO_REDIRECT_URL}e"
RequestHeader set INFO_REMOTE_ADDR "%{INFO_REMOTE_ADDR}e"
RequestHeader set INFO_REMOTE_HOST "%{INFO_REMOTE_HOST}e"
RequestHeader set INFO_REMOTE_IDENT "%{INFO_REMOTE_IDENT}e"
RequestHeader set INFO_REMOTE_PORT "%{INFO_REMOTE_PORT}e"
RequestHeader set INFO_REMOTE_USER "%{INFO_REMOTE_USER}e"
RequestHeader set INFO_REQUEST_FILENAME "%{INFO_REQUEST_FILENAME}e"
RequestHeader set INFO_REQUEST_METHOD "%{INFO_REQUEST_METHOD}e"
RequestHeader set INFO_REQUEST_TIME "%{INFO_REQUEST_TIME}e"
RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"
RequestHeader set INFO_SCRIPT_FILENAME "%{INFO_SCRIPT_FILENAME}e"
RequestHeader set INFO_SCRIPT_GROUP "%{INFO_SCRIPT_GROUP}e"
RequestHeader set INFO_SCRIPT_NAME "%{INFO_SCRIPT_NAME}e"
RequestHeader set INFO_SCRIPT_URI "%{INFO_SCRIPT_URI}e"
RequestHeader set INFO_SCRIPT_URL "%{INFO_SCRIPT_URL}e"
RequestHeader set INFO_SCRIPT_USER "%{INFO_SCRIPT_USER}e"
RequestHeader set INFO_SERVER_ADDR "%{INFO_SERVER_ADDR}e"
RequestHeader set INFO_SERVER_ADMIN "%{INFO_SERVER_ADMIN}e"
RequestHeader set INFO_SERVER_NAME "%{INFO_SERVER_NAME}e"
RequestHeader set INFO_SERVER_PORT "%{INFO_SERVER_PORT}e"
RequestHeader set INFO_SERVER_PROTOCOL "%{INFO_SERVER_PROTOCOL}e"
RequestHeader set INFO_SERVER_SIGNATURE "%{INFO_SERVER_SIGNATURE}e"
RequestHeader set INFO_SERVER_SOFTWARE "%{INFO_SERVER_SOFTWARE}e"
RequestHeader set INFO_THE_REQUEST "%{INFO_THE_REQUEST}e"
RequestHeader set INFO_TIME "%{INFO_TIME}e"
RequestHeader set INFO_TIME_DAY "%{INFO_TIME_DAY}e"
RequestHeader set INFO_TIME_HOUR "%{INFO_TIME_HOUR}e"
RequestHeader set INFO_TIME_MIN "%{INFO_TIME_MIN}e"
RequestHeader set INFO_TIME_MON "%{INFO_TIME_MON}e"
RequestHeader set INFO_TIME_SEC "%{INFO_TIME_SEC}e"
RequestHeader set INFO_TIME_WDAY "%{INFO_TIME_WDAY}e"
RequestHeader set INFO_TIME_YEAR "%{INFO_TIME_YEAR}e"
RequestHeader set INFO_TZ "%{INFO_TZ}e"
RequestHeader set INFO_UNIQUE_ID "%{INFO_UNIQUE_ID}e"

Mod_Rewrite Variables Decoded!

[API_VERSION] => 20020903:12
[AUTH_TYPE] => Digest
[DOCUMENT_ROOT] => /home/user/www_root/askapache.com
[HTTPS] => off
[HTTP_ACCEPT] => text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
[HTTP_COOKIE] => PHPSESSID=752ee6d56e15f305233e30045987e5ce568c034; __qca=1176541225-59967328-5223185;
[HTTP_HOST] => www.askapache.com
[HTTP_REFERER] => http://www.askapache.com/protest/index.php?askapache=awesomeness&you=rock
[HTTP_USER_AGENT] => Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16
[IS_SUBREQ] => false
[QUERY_STRING] => e=404
[REMOTE_ADDR] => 22.162.144.211
[REMOTE_HOST] => 22.162.144.211
[REMOTE_PORT] => 4511
[REMOTE_USER] => administrator
[REQUEST_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[REQUEST_METHOD] => GET
[REQUEST_URI] => /protest/index.php
[SCRIPT_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[SCRIPT_GROUP] => daemonu
[SCRIPT_USER] => askapache
[SERVER_ADDR] => 208.113.134.190
[SERVER_ADMIN] => webmaster@askapache.com
[SERVER_NAME] => www.askapache.com
[SERVER_PORT] => 80
[SERVER_PROTOCOL] => HTTP/1.1
[SERVER_SOFTWARE] => Apache/2.0.61 (Unix) PHP/4.4.7 mod_ssl/2.0.61 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2
[THE_REQUEST] => GET /protest/adf HTTP/1.1
[TIME] => 20080820014309
[TIME_DAY] => 20
[TIME_HOUR] => 01
[TIME_MIN] => 43
[TIME_MON] => 08
[TIME_SEC] => 09
[TIME_WDAY] => 3
[TIME_YEAR] => 2008

Request using HTTPS

[API_VERSION] => 20020903:12
[AUTH_TYPE] => Digest
[DOCUMENT_ROOT] => /home/user/www_root/askapache.com
[HTTPS] => on
[HTTP_ACCEPT] => text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
[HTTP_COOKIE] => PHPSESSID=752ee6d56e15f305233e30045987e5ce568c034; __qca=1176541225-59967328-5223185;
[HTTP_HOST] => www.askapache.com
[HTTP_REFERER] => http://www.askapache.com/protest/index.php?askapache=awesomeness&you=rock
[HTTP_USER_AGENT] => Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16
[IS_SUBREQ] => false
[QUERY_STRING] => hi=you&whats=&you
[REMOTE_ADDR] => 22.162.144.211
[REMOTE_HOST] => 22.162.144.211
[REMOTE_PORT] => 4605
[REMOTE_USER] => administrator
[REQUEST_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[REQUEST_METHOD] => GET
[REQUEST_URI] => /protest/index.php
[SCRIPT_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[SCRIPT_GROUP] => daemonu
[SCRIPT_USER] => askapache
[SERVER_ADDR] => 208.113.134.190
[SERVER_ADMIN] => webmaster@askapache.com
[SERVER_NAME] => www.askapache.com
[SERVER_PORT] => 443
[SERVER_PROTOCOL] => HTTP/1.1
[SERVER_SOFTWARE] => Apache/2.0.61 (Unix) PHP/4.4.7 mod_ssl/2.0.61 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2
[THE_REQUEST] => GET /protest/index.php?hi=you&whats=&you HTTP/1.1
[TIME] => 20080820015016
[TIME_DAY] => 20
[TIME_HOUR] => 01
[TIME_MIN] => 50
[TIME_MON] => 08
[TIME_SEC] => 16
[TIME_WDAY] => 3
[TIME_YEAR] => 2008

Emulating ErrorDocuments with Mod_Rewrite

The ErrorDocument directive is helpful because an errordocument is called differently then a normal file, and it contains special variables to help an admin debug.

I've wanted to use a RewriteCond + a RewriteRule to cause an Apache ErrorDocument to be displayed for a long time... I finally figured it out. Simply use the HTTP STATUS CODE trick in combination with a simple RewriteRule to trigger an Apache ErrorDocument.

This code emulates the internal 404 process Apache goes through.. If the file is not found it requests the /test/trigger-error/404 internally which triggers the 404 ErrorDocument.

ErrorDocument 404 /test/errordocument/404.html
Redirect 404 /test/trigger-error/404
 
RewriteEngine On
RewriteBase /
RewriteCond %{ENV:REDIRECT_STATUS} !=404
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .* /test/trigger-error/404 [L]

Big Deal.. you might say... well consider that this works with any status code, and using this method you now have the power to trigger any errordocument page based on any kind of rewritecond. I'll be writing about some practical uses for this powerful method in the coming weeks, but heres a good example now so you can see how it can be used.

This bit of code Triggers the 505 HTTP Version Not Supported When a request is made to the server with a protocol other than 1.1.

ErrorDocument 505 /test/errordocument/505.html
Redirect 505 /test/trigger-error/505
 
RewriteEngine On
RewriteBase /
 
RewriteCond %{ENV:REDIRECT_STATUS} !=505
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /.*\ HTTP/(0\.9|1\.0|1\.1) [NC]
RewriteRule .* /test/trigger-error/505 [L]

YES! I realize I didn't explain that very well, I didn't realize it was that complicated.. I wanted to go into how to use these advanced tricks and methods to achieve some really cool stuff, but explaining just this little bit took me awhile and I'm out of page space!

So play around with this and I'll post back some of the untapped sicknesses you can give a website with such powerful methods at your disposal.

Ralf S. Engelschall

/*
 *  URL Rewriting Module
 *
 *  This module uses a rule-based rewriting engine (based on a
 *  regular-expression parser) to rewrite requested URLs on the fly.
 *
 *  It supports an unlimited number of additional rule conditions (which can
 *  operate on a lot of variables, even on HTTP headers) for granular
 *  matching and even external database lookups (either via plain text
 *  tables, DBM hash files or even external processes) for advanced URL
 *  substitution.
 *
 *  It operates on the full URLs (including the PATH_INFO part) both in
 *  per-server context (httpd.conf) and per-dir context (.htaccess) and even
 *  can generate QUERY_STRING parts on result.   The rewriting result finally
 *  can lead to internal subprocessing, external request redirection or even
 *  to internal proxy throughput.
 *
 *  This module was originally written in April 1996 and
 *  gifted exclusively to the The Apache Software Foundation in July 1997 by
 *
 *      Ralf S. Engelschall
 *      rse engelschall.com
 *      www.engelschall.com
 */

Crazy Advanced Mod_Rewrite Debug Tutorial originally appeared on AskApache.com

.htaccess is a very ancient configuration file that controls the Web Server running your website, and is one of the most powerful configuration files you will ever come across. Htaccess has the ability to control access of the WWW's HyperText Transfer Protocol (HTTP) using Password Protection, 301 Redirects, rewrites, and much much more. This is because this configuration file was coded in the earliest days of the web (HTTP), for one of the first Web Servers ever! Eventually these Web Servers (configured with htaccess) became known as the World Wide Web, and eventually grew into the Internet we use today.

This is not an introduction to .htaccess… This is the evolution of the best of the best .htaccess on the planet. Its changed over the years with more and better .htaccess tricks using real .htaccess example code. I add my favorite htaccess-related links and sites, results from my htaccess research, and frequently update this article.

You've come to the right place if you are looking to acquire mad skills for using .htaccess files.

Originally (2003) this guide was known in certain hacker circles and hidden corners of the net as an ultimate .htaccess due to the powerful htaccess tricks and tips to bypass security on a webhost, and also because many of the tricks and examples were pretty impressive back then in that group.

AskApache Htaccess Journey

Skip this - still under edit

I discovered these tips and tricks mostly while working as a network security penetration specialist hired to find security holes in web hosting environments. Shared hosting is the most common and cheapest form of web-hosting where multiple customers are placed on a single machine and "share" the resources (CPU/RAM/SPACE). The machines are configured to basically ONLY do HTTP and FTP. No shells or any interactive logins, no ssh, just FTP access. That is when I started examining htaccess files in great detail and learned about the incredible untapped power of htaccess. For 99% of the worlds best Apache admins, they don't use .htaccess much, if AT ALL. It's much easier, safer, and faster to configure Apache using the httpd.conf file instead. However, this file is almost never readable on shared-hosts, and I've never seen it writable. So the only avenue left for those on shared-hosting was and is the .htaccess file, and holy freaking fiber-optics.. it's almost as powerful as httpd.conf itself!

Most all .htaccess code works in the httpd.conf file, but not all httpd.conf code works in .htaccess files, around 50%. So all the best Apache admins and programmers never used .htaccess files. There was no incentive for those with access to httpd.conf to use htaccess, and the gap grew. It's common to see "computer gurus" on forums and mailing lists rail against all uses and users of .htaccess files, smugly announcing the well known problems with .htaccess files compared with httpd.conf - I wonder if these "gurus" know the history of the htaccess file, like it's use in the earliest versions of the HTTP Server- NCSA's HTTPd, which BTW, became known as Apache HTTP. So you could easily say that htaccess files predates Apache itself.

Once I discovered what .htaccess files could do towards helping me enumerate and exploit security vulnerabilities even on big shared-hosts I focused all my research into .htaccess files, meaning I was reading the venerable Apache HTTP Source code 24/7! I compiled every released version of the Apache Web Server, ever, even NCSA's, and focused on enumerating the most powerful htaccess directives. Good times! Because my focus was on protocol/file/network vulnerabilites instead of web dev I built up a nice toolbox of htaccess tricks to do unusual things. When I switched over to webdev in 2005 I started using htaccess for websites, not research. I documented most of my favorites and rewrote the htaccess guide for webdevelopers. After some great encouragement on various forums and nets I decided to start a blog to share my work with everyone, AskApache.com was registered, I published my guide, and it was quickly plagiarized and scraped all over the net. Information is freedom, and freedom is information, so this blog has the least restrictive copyright for you. Feel free to modify, copy, republish, sell, or use anything on this site ;)

Htaccess - Evolved

The Hyper Text Transfer Protocol (HTTP) was initiated at the CERN in Geneve (Switzerland), where it emerged (together with the HTML presentation language) from the need to exchange scientific information on a computer network in a simple manner. The first public HTTP implementation only allowed for plain text information, and almost instantaneously became a replacement of the GOPHER service. One of the first text-based browsers was LYNX which still exists today; a graphical HTTP client appeared very quickly with the name NCSA Mosaic. Mosaic was a popular browser back in 1994. Soon the need for a more rich multimedia experience was born, and the markup language provided support for a growing multitude of media types.

Htaccess file know-how will do several things for you:

• Make your website noticeably faster.
• Allow you to debug your server with ease.
• Make your life easier and more rewarding.
• Allow you to work faster and more productively.

What Is .htaccess

Specifically, .htaccess is the default file name of a special configuration file that provides a number of directives (commands) for controlling and configuring the Apache Web Server, and also to control and configure modules that can be built into the Apache installation, or included at run-time like mod_rewrite (for htaccess rewrite), mod_alias (for htaccess redirects), and mod_ssl (for controlling SSL connections).

Htaccess allows for decentralized management of Web Server configurations which makes life very easy for web hosting companies and especially their savvy consumers. They set up and run "server farms" where many hundreds and thousands of web hosting customers are all put on the same Apache Server. This type of hosting is called "virtual hosting" and without .htaccess files would mean that every customer must use the same exact settings as everyone else on their segment. So that is why any half-decent web host allows/enables (DreamHost, Powweb, MediaTemple, GoDaddy) .htaccess files, though few people are aware of it. Let's just say that if I was a customer on your server-farm, and .htaccess files were enabled, my websites would be a LOT faster than yours, as these configuration files allow you to fully take advantage of and utilize the resources allotted to you by your host. If even 1/10 of the sites on a server-farm took advantage of what they are paying for, the providers would go out of business.

One of the design goals for this server was to maintain external compatibility with the NCSA 1.3 server --- that is, to read the same configuration files, to process all the directives therein correctly, and in general to be a drop-in replacement for NCSA. On the other hand, another design goal was to move as much of the server's functionality into modules which have as little as possible to do with the monolithic server core. The only way to reconcile these goals is to move the handling of most commands from the central server into the modules.

However, just giving the modules command tables is not enough to divorce them completely from the server core. The server has to remember the commands in order to act on them later. That involves maintaining data which is private to the modules, and which can be either per-server, or per-directory. Most things are per-directory, including in particular access control and authorization information, but also information on how to determine file types from suffixes, which can be modified by AddType and DefaultType directives, and so forth. In general, the governing philosophy is that anything which can be made configurable by directory should be; per-server information is generally used in the standard set of modules for information like Aliases and Redirects which come into play before the request is tied to a particular place in the underlying file system.

Another requirement for emulating the NCSA server is being able to handle the per-directory configuration files, generally called .htaccess files, though even in the NCSA server they can contain directives which have nothing at all to do with access control. Accordingly, after URI -> filename translation, but before performing any other phase, the server walks down the directory hierarchy of the underlying filesystem, following the translated pathname, to read any .htaccess files which might be present. The information which is read in then has to be merged with the applicable information from the server's own config files (either from the <directory> sections in access.conf, or from defaults in srm.conf, which actually behaves for most purposes almost exactly like <directory />).

Finally, after having served a request which involved reading .htaccess files, we need to discard the storage allocated for handling them. That is solved the same way it is solved wherever else similar problems come up, by tying those structures to the per-transaction resource pool.

Creating Htaccess Files

Htaccess files use the default filename ".htaccess" but any unix-style file name can be specified from the main server config using the AccessFileName directive. The file isn't .htaccess.txt, its literally just named .htaccess.

In a Windows Environment like the one I use for work, you can change how Windows opens and views .htaccess files by modifying the Folder Options in explorer. As you can see, on my computer files ending in .htaccess are recognized as having the HTACCESS extension and are handled/opened by Adobe Dreamweaver CS4.

Htaccess Scope

Unlike the main server configuration files like httpd.conf, Htaccess files are read on every request therefore changes in these files take immediate effect. Apache searches all directories and subdirectories that are htaccess-enabled for an .htaccess file which results in performance loss due to file accesses. I've never noticed a performance loss but OTOH, I know how to use them. If you do have access to your main server configuration file, you should of course use that instead, and lucky for you ALL the .htaccess tricks and examples can be used there as well (just not vice versa).

Htaccess File Syntax

Htaccess files follow the same syntax as the main Apache configuration files, for powerusers here's an apache.vim for VI. The one main difference is the context of the directive, which means whether or not that directive is ALLOWED to be used inside of an .htaccess file. Htaccess files are incredibly powerful, and can also be very dangerous as some directives allowed in the main configuration files would allow users/customers to completely bypass security/bandwidth-limits/resource-limits/file-permissions, etc.. About 1/4 of all Apache directives cannot be used inside an .htaccess file (also known as a per-directory context config). The Apache Developers are well-regarded throughout the world as being among some of the best programmers, ever. To enable a disallowed directive inside a .htaccess file would require modifying the source code and re-compiling the server (which they allow and encourage if you are the owner/admin). Here's a taste of that famous Apache source code that builds the directives allowed in .htaccess file context, the key that tells whether its enabled in .htaccess context is the DIR_CMD_PERMS and then the OR_FILEINFO, which means a directive is enabled dependent on the AllowOverride directive that is only allowed in the main config. First Apache 1.3.0, then Apache 2.2.10

mod_autoindex

AddIcon, add_icon, BY_PATH, DIR_CMD_PERMS, an icon URL followed by one or more filenames
AddIconByType, add_icon, BY_TYPE, DIR_CMD_PERMS, an icon URL followed by one or more MIME types
AddIconByEncoding, add_icon, BY_ENCODING, DIR_CMD_PERMS, an icon URL followed by one or more content encodings
AddAlt, add_alt, BY_PATH, DIR_CMD_PERMS, alternate descriptive text followed by one or more filenames
AddAltByType, add_alt, BY_TYPE, DIR_CMD_PERMS, alternate descriptive text followed by one or more MIME types
AddAltByEncoding, add_alt, BY_ENCODING, DIR_CMD_PERMS, alternate descriptive text followed by one or more content encodings
IndexOptions, add_opts, DIR_CMD_PERMS, RAW_ARGS, one or more index options
IndexIgnore, add_ignore, DIR_CMD_PERMS, ITERATE, one or more file extensions
AddDescription, add_desc, BY_PATH, DIR_CMD_PERMS, Descriptive text followed by one or more filenames
HeaderName, add_header, DIR_CMD_PERMS, TAKE1, a filename
ReadmeName, add_readme, DIR_CMD_PERMS, TAKE1, a filename
FancyIndexing, fancy_indexing, DIR_CMD_PERMS, FLAG, Limited to 'on' or 'off' (superseded by IndexOptions FancyIndexing)
DefaultIcon, ap_set_string_slot, (void *) XtOffsetOf(autoindex_config_rec, default_icon), DIR_CMD_PERMS, TAKE1, an icon URL

mod_rewrite

// mod_rewrite
RewriteEngine, cmd_rewriteengine, OR_FILEINFO, On or Off to enable or disable (default)
RewriteOptions, cmd_rewriteoptions, OR_FILEINFO, List of option strings to set
RewriteBase, cmd_rewritebase, OR_FILEINFO, the base URL of the per-directory context
RewriteCond, cmd_rewritecond, OR_FILEINFO, an input string and a to be applied regexp-pattern
RewriteRule, cmd_rewriterule, OR_FILEINFO, an URL-applied regexp-pattern and a substitution URL
RewriteMap, cmd_rewritemap, RSRC_CONF, a mapname and a filename
RewriteLock, cmd_rewritelock, RSRC_CONF, the filename of a lockfile used for inter-process synchronization
RewriteLog, cmd_rewritelog, RSRC_CONF, the filename of the rewriting logfile
RewriteLogLevel, cmd_rewriteloglevel, RSRC_CONF, the level of the rewriting logfile verbosity (0=none, 1=std, .., 9=max)
RewriteLog, fake_rewritelog, RSRC_CONF, [DISABLED] the filename of the rewriting logfile
RewriteLogLevel, fake_rewritelog, RSRC_CONF, [DISABLED] the level of the rewriting logfile verbosity

Htaccess Directives

Don't ask why, but I personally downloaded each major/beta release of the Apache HTTPD source code from version 1.3.0 to version 2.2.10 (all 63 Apache versions!), then I configured and compiled each version for a custom HTTPD installation built from source. This allowed me to find every directive allowed in .htaccess files for each particular version, which has never been done before, or since. YES! I think that is so cool..

An .htaccess directive is basically a command that is specific to a module or builtin to the core that performs a specific task or sets a specific setting for how Apache serves your WebSite. Directives placed in Htaccess files apply to the directory they are in, and all sub-directories. Here's the 3 top links (official Apache Docs) you will repeatedly use, bookmark/print/save them.

1. Terms Used to Describe Directives
2. Official List of Apache Directives
3. Directive Quick-Reference -- with Context

Litespeed Htaccess support

Unlike other lightweight web servers, Apache compatible per-directory configuration overridden is fully supported by LiteSpeed Web Server. With .htacess you can change configurations for any directory under document root on-the-fly, which in most cases is a mandatory feature in shared hosting environment. It is worth noting that enabling .htaccess support in LiteSpeed Web Server will not degrade server's performance, comparing to Apache's 40% drop in performance.

Main Server Config Examples

Now lets take a look at some htaccess examples to get a feel for the syntax and some general ideas at the capabilities. Some of the best examples for .htaccess files are included with Apache for main server config files, so lets take a quick look at a couple of them on our way down to the actual .htaccess examples further down the page (this site has thousands, take your time). As you can see, the basic syntax is a line starting with # is a comment, everything else are directives followed by the directive argument.

httpd-multilang-errordoc.conf: The configuration below implements multi-language error documents through content-negotiation

Options IncludesNoExec
AddOutputFilter Includes html
AddHandler type-map var
LanguagePriority en cs de es fr it ja ko nl pl pt-br ro sv tr
ForceLanguagePriority Prefer Fallback
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
ErrorDocument 403 /error/HTTP_FORBIDDEN.html.var
ErrorDocument 404 /error/HTTP_NOT_FOUND.html.var

httpd-manual.conf: Provide local access to the server documentation on your server

SetEnvIf Request_URI ^/manual/(de|en|es|fr|ja|ko|pt-br|ru|tr)/ prefer-language=$1
RedirectMatch 301 ^/manual(?:/(de|en|es|fr|ja|ko|pt-br|ru|tr)){2, }(/.*)?$ /manual/$1$2
LanguagePriority en de es fr ja ko pt-br ru tr
ForceLanguagePriority Prefer Fallback

httpd-languages.conf: Settings for hosting different languages.

DefaultLanguage en
AddLanguage ca .ca
# Just list the languages in decreasing order of preference.
LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv tr zh-CN zh-TW
# Commonly used filename extensions to character sets.
AddCharset us-ascii.ascii .us-ascii
AddCharset ISO-8859-1  .iso8859-1  .latin1

httpd-autoindex.conf: Directives controlling the display of server-generated directory listings.

# IndexOptions: Controls the appearance of server-generated directory listings.
IndexOptions FancyIndexing HTMLTable VersionSort
# AddIcon* directives tell the server which icon to show for different files or filename extensions.
AddIconByEncoding (CMP, /icons/compressed.gif) x-compress x-gzip
AddIconByType (TXT, /icons/text.gif) text/*
AddIcon /icons/folder.gif ^^DIRECTORY^^
# DefaultIcon is which icon to show for files which do not have an icon explicitly set.
DefaultIcon /icons/unknown.gif
# AddDescription allows you to place a short description after a file in server-generated indexes.
AddDescription "GZIP compressed document" .gz
# ReadmeName is the name of the README file the server will look for by default, and append to directory listings.
ReadmeName README.html
# HeaderName is the name of a file which should be prepended to directory indexes.
HeaderName HEADER.html

Here are the rest of them if you wanna take a look. (httpd-mpm.conf, httpd-default.conf, httpd-ssl.conf, httpd-info.conf, httpd-vhosts.conf, httpd-dav.conf)

Example .htaccess Files

Here are some samples and examples taken from different .htaccess files I've used over the years. Specific solutions are farther down on this page and throughout the site.

# Set the Time Zone of your Server
SetEnv TZ America/Indianapolis
# ServerAdmin:  This address appears on some server-generated pages, such as error documents.
SetEnv SERVER_ADMIN webmaster@askapache.com
# Possible values for the Options directive are "None", "All", or any combination of:
#  Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
Options -ExecCGI -MultiViews -Includes -Indexes FollowSymLinks
# DirectoryIndex: sets the file that Apache will serve if a directory is requested.
DirectoryIndex index.html index.php /index.php
#
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#
Action php5-cgi /bin/php.cgi
#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
AddHandler php-cgi .php .inc
# Commonly used filename extensions to character sets.
AddDefaultCharset UTF-8
# AddType allows you to add to or override the MIME configuration
AddType 'application/rdf+xml; charset=UTF-8' .rdf
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml.gz
AddType 'text/html; charset=UTF-8' .html
AddType 'text/html; charset=UTF-8' .html.gz
AddType application/octet-stream .rar .chm .bz2 .tgz .msi .pdf .exe
AddType application/vnd.ms-excel .csv
AddType application/x-httpd-php-source .phps
AddType application/x-pilot .prc .pdb
AddType application/x-shockwave-flash .swf
AddType application/xrds+xml .xrdf
AddType text/plain .ini .sh .bsh .bash .awk .nawk .gawk .csh .var .c .in .h .asc .md5 .sha .sha1
AddType video/x-flv .flv
# AddEncoding allows you to have certain browsers uncompress information on the fly. Note: Not all browsers support this.
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
# DefaultType: the default MIME type the server will use for a document.
DefaultType text/html
#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature Off

## MAIN DEFAULTS
Options +ExecCGI -Indexes
DirectoryIndex index.html index.htm index.php
DefaultLanguage en-US
AddDefaultCharset UTF-8
ServerSignature Off
## ENVIRONMENT VARIABLES
SetEnv PHPRC /webroot/includes
SetEnv TZ America/Indianapolis
 
SetEnv SERVER_ADMIN webmaster@askapache.com
## MIME TYPES
AddType video/x-flv .flv
AddType application/x-shockwave-flash .swf
AddType image/x-icon .ico
## FORCE FILE TO DOWNLOAD INSTEAD OF APPEAR IN BROWSER
# http://www.htaccesselite.com/addtype-addhandler-action-vf6.html
AddType application/octet-stream .mov .mp3 .zip
## ERRORDOCUMENTS
# http://askapache.com/htaccess/apache-status-code-headers-errordocument.html
ErrorDocument 400 /e400/
ErrorDocument 401 /e401/
ErrorDocument 402 /e402/
ErrorDocument 403 /e403/
ErrorDocument 404 /e404/
#
# Handlers be builtin, included in a module, or added with Action directive
# default-handler: default, handles static content (core)
#   send-as-is: Send file with HTTP headers (mod_asis)
#   cgi-script: treat file as CGI script (mod_cgi)
#    imap-file: Parse as an imagemap rule file (mod_imap)
#   server-info: Get server config info (mod_info)
#  server-status: Get server status report (mod_status)
#    type-map: type map file for content negotiation (mod_negotiation)
#  fastcgi-script: treat file as fastcgi script (mod_fastcgi)
#
# http://www.askapache.com/php/custom-phpini-tips-and-tricks.html
## PARSE AS CGI
AddHandler cgi-script .cgi .pl .spl
## RUN PHP AS APACHE MODULE
AddHandler application/x-httpd-php .php .htm
## RUN PHP AS CGI
AddHandler php-cgi .php .htm
## CGI PHP WRAPPER FOR CUSTOM PHP.INI
AddHandler phpini-cgi .php .htm
Action phpini-cgi /cgi-bin/php5-custom-ini.cgi
## FAST-CGI SETUP WITH PHP-CGI WRAPPER FOR CUSTOM PHP.INI
AddHandler fastcgi-script .fcgi
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5-wrapper.fcgi
## CUSTOM PHP CGI BINARY SETUP
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php.cgi
## PROCESS SPECIFIC FILETYPES WITH CGI-SCRIPT
Action image/gif /cgi-bin/img-create.cgi
## CREATE CUSTOM HANDLER FOR SPECIFIC FILE EXTENSIONS
AddHandler custom-processor .ssp
Action custom-processor /cgi-bin/myprocessor.cgi
### HEADER CACHING
# http://www.askapache.com/htaccess/speed-up-sites-with-htaccess-caching.html
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico)$">
Header set Cache-Control "max-age=2592000"
</FilesMatch>
<FilesMatch "\.(js|css|pdf|swf)$">
Header set Cache-Control "max-age=604800"
</FilesMatch>
<FilesMatch "\.(html|htm|txt)$">
Header set Cache-Control "max-age=600"
</FilesMatch>
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
Header unset Cache-Control
</FilesMatch>
## ALTERNATE EXPIRES CACHING
# htaccesselite.com/d/use-htaccess-to-speed-up-your-site-discussion-vt67.html
ExpiresActive On
ExpiresDefault A604800
ExpiresByType image/x-icon A2592000
ExpiresByType application/x-javascript A2592000
ExpiresByType text/css A2592000
ExpiresByType text/html A300
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
ExpiresActive Off
</FilesMatch>
## META HTTP-EQUIV REPLACEMENTS
<FilesMatch "\.(html|htm|php)$">
Header set imagetoolbar "no"
</FilesMatch>

Here are some default MOD_REWRITE code examples.

## REWRITE DEFAULTS
RewriteEngine On
RewriteBase /
## REQUIRE SUBDOMAIN
RewriteCond %{HTTP_HOST} !^$
RewriteCond %{HTTP_HOST} !^subdomain\.askapache\.com$ [NC]
RewriteRule ^/(.*)$ http://subdomain.askapache.com/$1 [L,R=301]
## SEO REWRITES
RewriteRule ^(.*)/ve/(.*)$ $1/voluntary-employee/$2 [L,R=301]
RewriteRule ^(.*)/hsa/(.*)$ $1/health-saving-account/$2 [L,R=301]
## WORDPRESS
RewriteCond %{REQUEST_FILENAME} !-f  # Existing File
RewriteCond %{REQUEST_FILENAME} !-d  # Existing Directory
RewriteRule . /index.php [L]
## ALTERNATIVE ANTI-HOTLINKING
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(subdomain\.)?askapache.com/.*$ [NC]
RewriteRule ^.*\.(bmp|tif|gif|jpg|jpeg|jpe|png)$ - [F]
## REDIRECT HOTLINKERS
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(subdomain\.)?askapache.com/.*$ [NC]
RewriteRule ^.*\.(bmp|tif|gif|jpg|jpeg|jpe|png)$ http://google.com [R]
## DENY REQUEST BASED ON REQUEST METHOD
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS|HEAD)$ [NC]
RewriteRule ^.*$ - [F]
## REDIRECT UPLOADS
RewriteCond %{REQUEST_METHOD} ^(PUT|POST)$ [NC]
RewriteRule ^(.*)$ /cgi-bin/form-upload-processor.cgi?p=$1 [L,QSA]
## REQUIRE SSL EVEN WHEN MOD_SSL IS NOT LOADED
RewriteCond %{HTTPS} !=on [NC]
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [R,L]
### ALTERNATATIVE TO USING ERRORDOCUMENT
# http://www.htaccesselite.com/d/htaccess-errordocument-examples-vt11.html
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^.*$ /error.php [L]
## SEO REDIRECTS
Redirect 301 /2006/oldfile.html http://subdomain.askapache.com/newfile.html
RedirectMatch 301 /o/(.*)$ http://subdomain.askapache.com/s/dl/$1

Examples of protecting your files and securing with password protection.

#
# Require (user|group|valid-user) (username|groupname)
#
## BASIC PASSWORD PROTECTION
AuthType basic
AuthName "prompt"
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user
## ALLOW FROM IP OR VALID PASSWORD
Require valid-user
Allow from 192.168.1.23
Satisfy Any
## PROTECT FILES
<FilesMatch "\.(htaccess|htpasswd|ini|phps|fla|psd|log|sh)$">
Order Allow,Deny
Deny from all
</FilesMatch>
## PREVENT HOTLINKING
SetEnvIfNoCase Referer "^http://subdomain.askapache.com/" good
SetEnvIfNoCase Referer "^$" good
<FilesMatch "\.(png|jpg|jpeg|gif|bmp|swf|flv)$">
Order Deny,Allow
Deny from all
Allow from env=good
ErrorDocument 403 http://www.google.com/intl/en_ALL/images/logo.gif
ErrorDocument 403 /images/you_bad_hotlinker.gif
</FilesMatch>
## LIMIT UPLOAD FILE SIZE TO PROTECT AGAINST DOS ATTACK
#bytes, 0-2147483647(2GB)
LimitRequestBody 10240000
## MOST SECURE WAY TO REQUIRE SSL
# http://www.askapache.com/htaccess/apache-ssl-in-htaccess-examples.html
SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "askapache.com"
ErrorDocument 403 https://askapache.com
## COMBINED DEVELOPER HTACCESS CODE-USE THIS
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|js|css|pdf|swf|html|htm|txt)$">
Header set Cache-Control "max-age=5"
</FilesMatch>
AuthType basic
AuthName "Ooops! Temporarily Under Construction..."
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user      # password prompt for everyone else
Order Deny,Allow
Deny from all
Allow from 192.168.64.5   # Your, the developers IP address
Allow from w3.org      # css/xhtml check jigsaw.w3.org/css-validator/
Allow from googlebot.com   # Allows google to crawl your pages
Satisfy Any        # no password required if host/ip is Allowed
## DONT HAVE TO EMPTY CACHE OR RELOAD TO SEE CHANGES
ExpiresDefault A5 #If using mod_expires
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|js|css|pdf|swf|html|htm|txt)$">
Header set Cache-Control "max-age=5"
</FilesMatch>
## ALLOW ACCESS WITH PASSWORD OR NO PASSWORD FOR SPECIFIC IP/HOSTS
AuthType basic
AuthName "Ooops! Temporarily Under Construction..."
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user      # password prompt for everyone else
Order Deny,Allow
Deny from all
Allow from 192.168.64.5   # Your, the developers IP address
Allow from w3.org      # css/xhtml check jigsaw.w3.org/css-validator/
Allow from googlebot.com   # Allows google to crawl your pages
Satisfy Any        # no password required if host/ip is Allowed

Example .htaccess Code Snippets

Here are some specific examples, this is the most popular section of this page. Updated frequently.

Redirect Everyone Except IP address to alternate page

ErrorDocument 403 http://www.yahoo.com/
Order deny,allow
Deny from all
Allow from 208.113.134.190

When developing sites

This lets google crawl the page, lets me access without a password, and lets my client access the page WITH a password. It also allows for XHTML and CSS validation! (w3.org)

AuthName "Under Development"
AuthUserFile /home/sitename.com/.htpasswd
AuthType basic
Require valid-user
Order deny,allow
Deny from all
Allow from 208.113.134.190 w3.org htmlhelp.com googlebot.com
Satisfy Any

Fix double-login prompt

Redirect non-https requests to https server and ensure that .htpasswd authorization can only be entered across HTTPS

SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "askapache.com"
ErrorDocument 403 https://askapache.com

Set Timezone of the Server (GMT)

SetEnv TZ America/Indianapolis

Administrator Email for ErrorDocument

SetEnv SERVER_ADMIN webmaster@google.com

ServerSignature for ErrorDocument

ServerSignature off | on | email

Charset and Language headers

Article: Setting Charset in htaccess, and article by Richard Ishida

AddDefaultCharset UTF-8
DefaultLanguage en-US

Disallow Script Execution

Options -ExecCGI
AddHandler cgi-script .php .pl .py .jsp .asp .htm .shtml .sh .cgi

Deny Request Methods

RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|OPTIONS|POST|PUT)
RewriteRule .* - [F]

Force "File Save As" Prompt

AddType application/octet-stream .avi .mpg .mov .pdf .xls .mp4

Show CGI Source Code

RemoveHandler cgi-script .pl .py .cgi
AddType text/plain .pl .py .cgi

Serve all .pdf files on your site using .htaccess and mod_rewrite with the php script.

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+)\.pdf$  /cgi-bin/pdf.php?file=$1 [L,NC,QSA]

Rewrite to www

RewriteCond %{REQUEST_URI} !^/(robots\.txt|favicon\.ico|sitemap\.xml)$
RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC]
RewriteRule ^(.*)$ http://www.askapache.com/$1 [R=301,L]

Rewrite to www dynamically

RewriteCond %{REQUEST_URI} !^/robots\.txt$ [NC]
RewriteCond %{HTTP_HOST} !^www\.[a-z-]+\.[a-z]{2,6} [NC]
RewriteCond %{HTTP_HOST} ([a-z-]+\.[a-z]{2,6})$   [NC]
RewriteRule ^/(.*)$ http://%1/$1 [R=301,L]

301 Redirect Old File

Redirect 301 /old/file.html http://www.askapache.com/new/file.html

301 Redirect Entire Directory

RedirectMatch 301 /blog(.*) http://www.askapache.com/$1

Protecting your php.cgi

<FilesMatch "^php5?\.(ini|cgi)$">
Order Deny,Allow
Deny from All
Allow from env=REDIRECT_STATUS
</FilesMatch>

Set Cookie based on Request

This code sends the Set-Cookie header to create a cookie on the client with the value of a matching item in 2nd parantheses.

RewriteEngine On
RewriteBase /
RewriteRule ^(.*)(de|es|fr|it|ja|ru|en)/$ - [co=lang:$2:.askapache.com:7200:/]

Set Cookie with env variable

Header set Set-Cookie "language=%{lang}e; path=/;" env=lang

Custom ErrorDocuments

ErrorDocument 100 /100_CONTINUE
ErrorDocument 101 /101_SWITCHING_PROTOCOLS
ErrorDocument 102 /102_PROCESSING
ErrorDocument 200 /200_OK
ErrorDocument 201 /201_CREATED
ErrorDocument 202 /202_ACCEPTED
ErrorDocument 203 /203_NON_AUTHORITATIVE
ErrorDocument 204 /204_NO_CONTENT
ErrorDocument 205 /205_RESET_CONTENT
ErrorDocument 206 /206_PARTIAL_CONTENT
ErrorDocument 207 /207_MULTI_STATUS
ErrorDocument 300 /300_MULTIPLE_CHOICES
ErrorDocument 301 /301_MOVED_PERMANENTLY
ErrorDocument 302 /302_MOVED_TEMPORARILY
ErrorDocument 303 /303_SEE_OTHER
ErrorDocument 304 /304_NOT_MODIFIED
ErrorDocument 305 /305_USE_PROXY
ErrorDocument 307 /307_TEMPORARY_REDIRECT
ErrorDocument 400 /400_BAD_REQUEST
ErrorDocument 401 /401_UNAUTHORIZED
ErrorDocument 402 /402_PAYMENT_REQUIRED
ErrorDocument 403 /403_FORBIDDEN
ErrorDocument 404 /404_NOT_FOUND
 
ErrorDocument 405 /405_METHOD_NOT_ALLOWED
ErrorDocument 406 /406_NOT_ACCEPTABLE
ErrorDocument 407 /407_PROXY_AUTHENTICATION_REQUIRED
ErrorDocument 408 /408_REQUEST_TIME_OUT
ErrorDocument 409 /409_CONFLICT
ErrorDocument 410 /410_GONE
ErrorDocument 411 /411_LENGTH_REQUIRED
ErrorDocument 412 /412_PRECONDITION_FAILED
ErrorDocument 413 /413_REQUEST_ENTITY_TOO_LARGE
ErrorDocument 414 /414_REQUEST_URI_TOO_LARGE
ErrorDocument 415 /415_UNSUPPORTED_MEDIA_TYPE
ErrorDocument 416 /416_RANGE_NOT_SATISFIABLE
ErrorDocument 417 /417_EXPECTATION_FAILED
ErrorDocument 422 /422_UNPROCESSABLE_ENTITY
ErrorDocument 423 /423_LOCKED
ErrorDocument 424 /424_FAILED_DEPENDENCY
ErrorDocument 426 /426_UPGRADE_REQUIRED
ErrorDocument 500 /500_INTERNAL_SERVER_ERROR
ErrorDocument 501 /501_NOT_IMPLEMENTED
ErrorDocument 502 /502_BAD_GATEWAY
ErrorDocument 503 /503_SERVICE_UNAVAILABLE
ErrorDocument 504 /504_GATEWAY_TIME_OUT
ErrorDocument 505 /505_VERSION_NOT_SUPPORTED
ErrorDocument 506 /506_VARIANT_ALSO_VARIES
ErrorDocument 507 /507_INSUFFICIENT_STORAGE
ErrorDocument 510 /510_NOT_EXTENDED

Implementing a Caching Scheme with .htaccess

# year
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf|mp3|mp4)$">
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
</FilesMatch>
#2 hours
<FilesMatch "\.(html|htm|xml|txt|xsl)$">
Header set Cache-Control "max-age=7200, must-revalidate"
</FilesMatch>
<FilesMatch "\.(js|css)$">
SetOutputFilter DEFLATE
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</FilesMatch>

Password Protect single file

<Files login.php>
AuthName "Prompt"
AuthType Basic
AuthUserFile /home/askapache.com/.htpasswd
Require valid-user
</Files>

Password Protect multiple files

<FilesMatch "^(private|phpinfo)\.*$">
AuthName "Development"
AuthUserFile /.htpasswd
AuthType basic
Require valid-user
</FilesMatch>

Send Custom Headers

Header set P3P "policyref=\"http://www.askapache.com/w3c/p3p.xml\""
Header set X-Pingback "http://www.askapache.com/xmlrpc.php"
Header set Content-Language "en-US"
Header set Vary "Accept-Encoding"

Blocking based on User-Agent Header

SetEnvIfNoCase ^User-Agent$ .*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures) HTTP_SAFE_BADBOT
SetEnvIfNoCase ^User-Agent$ .*(libwww-perl|aesop_com_spiderman) HTTP_SAFE_BADBOT
Deny from env=HTTP_SAFE_BADBOT

Blocking with RewriteCond

RewriteCond %{HTTP_USER_AGENT} ^.*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]
RewriteRule . - [F,L]

.htaccess for mod_php

SetEnv PHPRC /location/todir/containing/phpinifile

.htaccess for php as cgi

AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5.cgi

Shell wrapper for custom php.ini

#!/bin/sh
export PHP_FCGI_CHILDREN=3
exec php5.cgi -c /abs/php5/php.ini

Add values from HTTP Headers

SetEnvIfNoCase ^If-Modified-Since$ "(.+)" HTTP_IF_MODIFIED_SINCE=$1
SetEnvIfNoCase ^If-None-Match$ "(.+)" HTTP_IF_NONE_MATCH=$1
SetEnvIfNoCase ^Cache-Control$ "(.+)" HTTP_CACHE_CONTROL=$1
SetEnvIfNoCase ^Connection$ "(.+)" HTTP_CONNECTION=$1
SetEnvIfNoCase ^Keep-Alive$ "(.+)" HTTP_KEEP_ALIVE=$1
SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
SetEnvIfNoCase ^Cookie$ "(.+)" HTTP_MY_COOKIE=$1

Stop hotlinking

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?askapache.com/.*$ [NC]
RewriteRule \.(gif|jpg|swf|flv|png)$ http://www.askapache.com/feed.gif [R=302,L]

Turn logging off for IP

SecFilterSelective REMOTE_ADDR "208\.113\.183\.103" "nolog,noauditlog,pass"

Turn logging on for IP

SecFilterSelective REMOTE_ADDR "!^208\.113\.183\.103" "nolog,noauditlog,pass"
SecFilterSelective REMOTE_ADDR "208\.113\.183\.103" "log,auditlog,pass"

Advanced Mod_Rewrite Examples

Here are some specific examples taken mostly from my WordPress Password Protection plugin, that also does alot more than password protection as you can see from the following mod_rewrite examples. These are a few of the mod_rewrite uses that BlogSecurity declared pushed the boundaries of Mod_Rewrite! Some of these snippets are quite exotic and unlike anything you may have seen before, also only for those who understand them as they can kill a website pretty quick.

Directory Protection

Enable the DirectoryIndex Protection, preventing directory index listings and defaulting. [Disable]

Options -Indexes
DirectoryIndex index.html index.php /index.php

Password Protect wp-login.php

Requires a valid user/pass to access the login page[401]

<Files wp-login.php>
Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user
</Files>

Password Protect wp-admin

Requires a valid user/pass to access any non-static (css, js, images) file in this directory.[401]

Options -ExecCGI -Indexes +FollowSymLinks -Includes
DirectoryIndex index.php /index.php
Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$">
Allow from All
</FilesMatch>
<FilesMatch "(async-upload)\.php$">
<IfModule mod_security.c>
SecFilterEngine Off
</IfModule>
Allow from All
</FilesMatch>

Protect wp-content

Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [401]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-content/.*$ [NC]
RewriteCond %{REQUEST_FILENAME} !^.+flexible-upload-wp25js.php$
RewriteCond %{REQUEST_FILENAME} ^.+\.(php|html|htm|txt)$
RewriteRule .* - [F,NS,L]

Protect wp-includes

Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-includes/.*$ [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /wp-includes/js/.+/.+\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} ^.+\.php$
RewriteRule .* - [F,NS,L]

Common Exploits

Block common exploit requests with 403 Forbidden. These can help alot, may break some plugins. [403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ ///.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\=?(http|ftp|ssl|https):/.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\?.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(asp|ini|dll).*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(htpasswd|htaccess|aahtpasswd).*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

Stop Hotlinking

Denies any request for static files (images, css, etc) if referrer is not local site or empty. [403]

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_REFERER} !^http://www.askapache.com.*$ [NC]
RewriteRule \.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$ - [F,NS,L]

Safe Request Methods

Denies any request not using GET,PROPFIND,POST,OPTIONS,PUT,HEAD[403]

RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|POST|PROPFIND|OPTIONS|PUT)$ [NC]
RewriteRule .* - [F,NS,L]

Forbid Proxies

Denies any POST Request using a Proxy Server. Can still access site, but not comment. See Perishable Press [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:VIA}%{HTTP:FORWARDED}%{HTTP:USERAGENT_VIA}%{HTTP:X_FORWARDED_FOR}%{HTTP:PROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION}%{HTTP:HTTP_PC_REMOTE_ADDR}%{HTTP:HTTP_CLIENT_IP} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

Real wp-comments-post.php

Denies any POST attempt made to a non-existing wp-comments-post.php[403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

HTTP PROTOCOL

Denies any badly formed HTTP PROTOCOL in the request, 0.9, 1.0, and 1.1 only[403]

RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ .+\ HTTP/(0\.9|1\.0|1\.1) [NC]
RewriteRule .* - [F,NS,L]

SPECIFY CHARACTERS

Denies any request for a url containing characters other than "a-zA-Z0-9.+/-?=&" - REALLY helps but may break your site depending on your links. [403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ [a-zA-Z0-9\.\+_/\-\?\=\&]+\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

BAD Content Length

Denies any POST request that doesnt have a Content-Length Header[403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Length} ^$
RewriteCond %{REQUEST_URI} !^/(wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

BAD Content Type

Denies any POST request with a content type other than application/x-www-form-urlencoded|multipart/form-data[403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Type} !^(application/x-www-form-urlencoded|multipart/form-data.*(boundary.*)?)$ [NC]
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

Missing HTTP_HOST

Denies requests that dont contain a HTTP HOST Header.[403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_HOST} ^$
RewriteRule .* - [F,NS,L]

Bogus Graphics Exploit

Denies obvious exploit using bogus graphics[403]

RewriteCond %{HTTP:Content-Disposition} \.php [NC]
RewriteCond %{HTTP:Content-Type} image/.+ [NC]
RewriteRule .* - [F,NS,L]

No UserAgent, Not POST

Denies POST requests by blank user-agents. May prevent a small number of visitors from POSTING. [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

No Referer, No Comment

Denies any comment attempt with a blank HTTP_REFERER field, highly indicative of spam. May prevent some visitors from POSTING. [403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteCond %{HTTP_REFERER} ^-?$
RewriteRule .* - [F,NS,L]

Trackback Spam

Denies obvious trackback spam. See Holy Shmoly! [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^.*(opera|mozilla|firefox|msie|safari).*$ [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.+/trackback/?\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

Map all URIs except those corresponding to existing files to a handler

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteRule . /script.php

Map any request to a handler

In the case where all URIs should be sent to the same place (including potentially requests for static content) the method to use depends on the type of the handler. For php scripts, use: For other handlers such as php scripts, use:

RewriteEngine On
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php

And for CGI scripts:

ScriptAliasMatch .* /var/www/script.cgi

Map URIs corresponding to existing files to a handler instead

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php

If the existing files you wish to have handled by your script have a common set of file extensions distinct from that of the hander, you can bypass mod_rewrite and use instead mod_actions. Let's say you want all .html and .tpl files to be dealt with by your script:

Action foo-action /script.php
AddHandler foo-action html tpl

Deny access if var=val contains the string foo.

RewriteCond %{QUERY_STRING} foo
RewriteRule ^/url - [F]

Removing the Query String

RewriteRule ^/url /url?

Adding to the Query String

Keep the existing query string using the Query String Append flag, but add var=val to the end.

RewriteRule ^/url /url?var=val [QSA]

Rewriting For Certain Query Strings

Rewrite URLs like http://askapache.com/url1?var=val to http://askapache.com/url2?var=val but don't rewrite if val isn't present.

RewriteCond %{QUERY_STRING} val
RewriteRule ^/url1 /url2

Modifying the Query String

Change any single instance of val in the query string to other_val when accessing /path. Note that %1 and %2 are back-references to the matched part of the regular expression in the previous RewriteCond.

RewriteCond %{QUERY_STRING} ^(.*)val(.*)$
RewriteRule /path /path?%1other_val%2

Best .htaccess Articles

.htaccess for Webmasters

• Process certain requests for files using a cgi script
• Process Requests with certain Request Methods
• Make any file be a certain filetype
• Use IfModule directive for robust code

Mod_Rewrite URL Rewriting

Undocumented techniques and methods will allow you to utilize mod_rewrite at an "expert level" by showing you how to unlock its secrets.

• Check for a key in QUERY_STRING
• Block access to files during certain hours of the day
• Rewrite underscores to hyphens for SEO URL
• Redirecting WordPress Feeds to Feedburner

301 Redirects without mod_rewrite

• Redirect single url
• Redirect to new Domain

Secure PHP with .htaccess

If you have a php.cgi or php.ini file in your /cgi-bin/ directory or other pub directory, try requesting them from your web browser. If your php.ini shows up or worse you are able to execute your php cgi, you'll need to secure it ASAP. This shows several ways to secure these files, and other interpreters like perl, fastCGI, bash, csh, etc.

.htaccess Cookie Manipulation

Fresh .htaccess code for you! Check out the Cookie Manipulation and environment variable usage with mod_rewrite! I also included a couple Mod_Security .htaccess examples. Enjoy!

• Mod_Rewrite .htaccess Examples
• Cookie Manipulation and Tests with mod_rewrite
• Setting Environment Variables
• Using the Environment Variable
• Mod_Security .htaccess Examples

.htaccess Caching

• Speed Up Sites with htaccess Caching
• htaccess time cheat sheet

Password Protection and Authentication

• Require password for single file
• Example .htaccess file for password protection

Control HTTP Headers

• Prevent Caching 100%
• Remove IE imagetoolbar without meta tag
• Add Privacy (P3P) Header to your site
• Add language and charset headers without meta tags

Blocking Spam and bad Bots

Want to block a bad robot or web scraper using .htaccess files? Here are 2 methods that illustrate blocking 436 various user-agents. You can block them using either SetEnvIf methods, or by using Rewrite Blocks.

PHP htaccess tips

By using some cool .htaccess tricks we can control PHP to be run as a cgi or a module. If php is run as a cgi then we need to compile it ourselves or use .htaccess to force php to use a local php.ini file. If it is running as a module then we can use various directives supplied by that modules in .htaccess

• When php run as CGI
• Use a custom php.ini with mod_php or php as a cgi
• When php run as Apache Module (mod_php)
• When cgi php is run with wrapper (FastCGI)

HTTP to HTTPS Redirects with mod_rewrite

This is freaking sweet if you use SSL I promise you! Basically instead of having to check for HTTPS using a RewriteCond %{HTTPS} =on for every redirect that can be either HTTP or HTTPS, I set an environment variable once with the value "http" or "https" if HTTP or HTTPS is being used for that request, and use that env variable in the RewriteRule.

SSL in .htaccess

• Redirect non-https requests to https server
• Rewrite non-https to HTTPS without mod_ssl!
• Redirect everything served on port 80 to HTTPS URI

SetEnvIf and SetEnvIfNoCase in .htaccess

• Unique mod_setenvif Variables
• Populates HTTP_MY_ Variables with mod_setenvif variable values
• Allows only if HOST Header is present in request
• Add values from HTTP Headers

Site Security with .htaccess

chmod .htpasswd files 640, chmod .htaccess 644, php files 600, and chmod files that you really dont want people to see as 400. (NEVER chmod 777, try 766)

• CHMOD your files
• Prevent access to .htaccess and .htpasswd files
• Show Source Code instead of executing
• Securing directories: Remove ability to execute scripts
• .htaccess ErrorDocuments

.htaccess Security with MOD_SECURITY

• mod_security + mod_rewrite
• Block Spam by examining POST form fields
• Disabling mod_security conditionally per IP
• Disabling mod_security with .htaccess Authorization
• Block WordPress Spam Forever!
• Force Any Connections to be Paused a set number of ms
• ModSecurity Debugging and Logging
• Turn Off/On Logging JUST for your IP Address
• Mod_Security Directives for DreamHost
• Example httpd.conf mod_security rule files

Merging Notes

The order of merging is:

1. <Directory> (except regular expressions) and .htaccess done simultaneously (with .htaccess, if allowed, overriding <Directory>)
2. <DirectoryMatch> (and <Directory ~>)
3. <Files> and <FilesMatch> done simultaneously
4. <Location> and <LocationMatch> done simultaneously

Below is an artificial example to show the order of merging. Assuming they all apply to the request, the directives in this example will be applied in the order:

A > B > C > D > E

<Location />
E
</Location>
<Files askapache.txt>
D
</Files>
<VirtualHost *>
<Directory /a/b>
B
</Directory>
</VirtualHost>
<DirectoryMatch "^.*b$">
C
</DirectoryMatch>
<Directory /a/b>
A
</Directory>

My Favorite .htaccess Links

These are just some of my favorite .htaccess resources. I'm really into doing your own hacking to get knowledge and these links are all great resources in that respect. I'm really interested in new or unusual htaccess solutions or htaccess hacks using .htaccess files, so let me know if you find one.

NCSA HTTPd Tutorials

Robert Hansen
Here's a great Hardening HTAccess part 1, part 2, part 3 article that goes into detail about some of the rarer security applications for .htaccess files.

SAMAXES
Some very detailed and helpful .htaccess articles, such as the ".htaccess - gzip and cache your site for faster loading and bandwidth saving."

PerishablePress
Stupid .htaccess tricks is probably the best explanation online for many of the best .htaccess solutions, including many from this page. Unlike me they are fantastic writers, even for technical stuff they are very readable, so its a good blog to kick back on and read. They also have a fantastic article detailing how to block/deny specific requests using mod_rewrite.

BlogSecurity
Mostly a site for... blog security (which is really any web-app security) this blog has a few really impressive articles full of solid information for Hardening WordPress with .htaccess among more advanced topics that can be challenging but effective. This is a good site to subscribe to their feed, they publish plugin exploits and wordpress core vulnerabilities quite a bit.

Check-These
Oldschool security/unix dude with some incredibly detailed mod_rewrite tutorials, helped me the most when I first got into this, and a great guy too. See: Basic Mod_Rewrite Guide, and Advanced Mod_Rewrite Tutorial

Reaper-X
Alot of .htaccess tutorials and code. See: Hardening WordPress with Mod Rewrite and htaccess

jdMorgan
jdMorgan is the Moderator of the Apache Forum at WebmasterWorld, a great place for answers. In my experience he can answer any tough question pertaining to advanced .htaccess usage, haven't seen him stumped yet.

The W3C
Setting Charset in .htaccess is very informative.

Holy Shmoly!
A great blogger with analysis of attacks and spam. See: More ways to stop spammers and unwanted traffic.

Apache Week
A partnership with Red Hat back in the 90's that produced some excellent documentation.

Corz
Here's a resource that I consider to have some of the most creative and ingenious ideas for .htaccess files, although the author is somewhat of a character ;) Its a trip trying to navigate around the site, a fun trip. Its like nothing I've ever seen. There are only a few articles on the site, but the htaccess articles are very original and well-worth a look. See: htaccess tricks and tips.

Htaccess Directives

This is an AskApache.com exclusive you won't find this anywhere else.

Directory, DirectoryMatch, Files, FilesMatch, IfDefine, IfVersion, IfModule, Limit, LimitExcept, Location, LocationMatch, Proxy, ProxyMatch, VirtualHost, AcceptMutex, AcceptPathInfo, AccessFileName, Action, AddCharset, AddDefaultCharset, AddDescription, AddEncoding, AddHandler, AddInputFilter, AddLanguage, AddOutputFilter, AddOutputFilterByType, AddType, Alias, AliasMatch, AllowCONNECT, AllowOverride, Anonymous, Anonymous_Authoritative, Anonymous_LogEmail, Anonymous_MustGiveEmail, Anonymous_NoUserId, Anonymous_VerifyEmail, AuthAuthoritative, AuthDBMAuthoritative, AuthDBMGroupFile, AuthDBMType, AuthDBMUserFile, AuthDigestAlgorithm, AuthDigestDomain, AuthDigestFile, AuthDigestGroupFile, AuthDigestNcCheck, AuthDigestNonceFormat, AuthDigestNonceLifetime, AuthDigestQop, AuthDigestShmemSize, AuthGroupFile, AuthName, AuthType, AuthUserFile, BS2000Account, BrowserMatch, BrowserMatchNoCase, CacheNegotiatedDocs, CharsetDefault, CharsetOptions, CharsetSourceEnc, CheckSpelling, ContentDigest, CookieDomain, CookieExpires, CookieName, CookieStyle, CookieTracking, CoreDumpDirectory, DAV, DAVDepthInfinity, DAVMinTimeout, DefaultIcon, DefaultLanguage, DefaultType, DocumentRoot, ErrorDocument, ErrorLog, ExtFilterDefine, ExtFilterOptions, FancyIndexing, FileETag, ForceLanguagePriority, ForceType, GprofDir, Header, HeaderName, HostnameLookups, IdentityCheck, ImapBase, ImapDefault, ImapMenu, Include, IndexIgnore, LanguagePriority, LimitRequestBody, LimitRequestFields, LimitRequestFieldsize, LimitRequestLine, LimitXMLRequestBody, LockFile, LogLevel, MaxRequestsPerChild, MultiviewsMatch, NameVirtualHost, NoProxy, Options, PassEnv, PidFile, Port, ProxyBlock, ProxyDomain, ProxyErrorOverride, ProxyIOBufferSize, ProxyMaxForwards, ProxyPass, ProxyPassReverse, ProxyPreserveHost, ProxyReceiveBufferSize, ProxyRemote, ProxyRemoteMatch, ProxyRequests, ProxyTimeout, ProxyVia, RLimitCPU, RLimitMEM, RLimitNPROC, ReadmeName, Redirect, RedirectMatch, RedirectPermanent, RedirectTemp, RemoveCharset, RemoveEncoding, RemoveHandler, RemoveInputFilter, RemoveLanguage, RemoveOutputFilter, RemoveType, RequestHeader, Require, RewriteCond, RewriteRule, SSIEndTag, SSIErrorMsg, SSIStartTag, SSITimeFormat, SSIUndefinedEcho, Satisfy, ScoreBoardFile, Script, ScriptAlias, ScriptAliasMatch, ScriptInterpreterSource, ServerAdmin, ServerAlias, ServerName, ServerPath, ServerRoot, ServerSignature, ServerTokens, SetEnv, SetEnvIf, SetEnvIfNoCase, SetHandler, SetInputFilter, SetOutputFilter, Timeout, TypesConfig, UnsetEnv, UseCanonicalName, XBitHack, allow, deny, order, CGIMapExtension, EnableMMAP, ISAPIAppendLogToErrors, ISAPIAppendLogToQuery, ISAPICacheFile, ISAPIFakeAsync, ISAPILogNotSupported, ISAPIReadAheadBuffer, SSLLog, SSLLogLevel, MaxMemFree, ModMimeUsePathInfo, EnableSendfile, ProxyBadHeader, AllowEncodedSlashes, LimitInternalRecursion, EnableExceptionHook, TraceEnable, ProxyFtpDirCharset, AuthBasicAuthoritative, AuthBasicProvider, AuthDefaultAuthoritative, AuthDigestProvider, AuthLDAPAuthzEnabled, AuthLDAPBindDN, AuthLDAPBindPassword, AuthLDAPCharsetConfig, AuthLDAPCompareDNOnServer, AuthLDAPDereferenceAliases, AuthLDAPGroupAttribute, AuthLDAPGroupAttributeIsDN, AuthLDAPRemoteUserIsDN, AuthLDAPURL, AuthzDBMAuthoritative, AuthzDBMType, AuthzDefaultAuthoritative, AuthzGroupFileAuthoritative, AuthzLDAPAuthoritative, AuthzOwnerAuthoritative, AuthzUserAuthoritative, BalancerMember, DAVGenericLockDB, FilterChain, FilterDeclare, FilterProtocol, FilterProvider, FilterTrace, IdentityCheckTimeout, IndexStyleSheet, ProxyPassReverseCookieDomain, ProxyPassReverseCookiePath, ProxySet, ProxyStatus, ThreadStackSize, AcceptFilter, Protocol, AuthDBDUserPWQuery, AuthDBDUserRealmQuery, UseCanonicalPhysicalPort, CheckCaseOnly, AuthLDAPRemoteUserAttribute, ProxyPassMatch, SSIAccessEnable, Substitute, ProxyPassInterpolateEnv

Htaccess Modules

Here are most of the modules that come with Apache. Each one can have new commands that can be used in .htaccess file scopes.

mod_actions, mod_alias, mod_asis, mod_auth_basic, mod_auth_digest, mod_authn_anon, mod_authn_dbd, mod_authn_dbm, mod_authn_default, mod_authn_file, mod_authz_dbm, mod_authz_default, mod_authz_groupfile, mod_authz_host, mod_authz_owner, mod_authz_user, mod_autoindex, mod_cache, mod_cern_meta, mod_cgi, mod_dav, mod_dav_fs, mod_dbd, mod_deflate, mod_dir, mod_disk_cache, mod_dumpio, mod_env, mod_expires, mod_ext_filter, mod_file_cache, mod_filter, mod_headers, mod_ident, mod_imagemap, mod_include, mod_info, mod_log_config, mod_log_forensic, mod_logio, mod_mem_cache, mod_mime, mod_mime_magic, mod_negotiation, mod_proxy, mod_proxy_ajp, mod_proxy_balancer, mod_proxy_connect, mod_proxy_ftp, mod_proxy_http, mod_rewrite, mod_setenvif, mod_speling, mod_ssl, mod_status, mod_substitute, mod_unique_id, mod_userdir, mod_usertrack, mod_version, mod_vhost_alias

Htaccess Software

Apache HTTP Server comes with the following programs.

httpd

Apache hypertext transfer protocol server

apachectl

Apache HTTP server control interface

ab

Apache HTTP server benchmarking tool

apxs

APache eXtenSion tool

dbmmanage

Create and update user authentication files in DBM format for basic authentication

fcgistarter

Start a FastCGI program

htcacheclean

Clean up the disk cache

htdigest

Create and update user authentication files for digest authentication

htdbm

Manipulate DBM password databases.

htpasswd

Create and update user authentication files for basic authentication

httxt2dbm

Create dbm files for use with RewriteMap

logresolve

Resolve hostnames for IP-addresses in Apache logfiles

log_server_status

Periodically log the server's status

rotatelogs

Rotate Apache logs without having to kill the server

split-logfile

Split a multi-vhost logfile into per-host logfiles

suexec

Switch User For Exec

Technical Look at .htaccess

Source: Apache API notes

Per-directory configuration structures

Let's look out how all of this plays out in mod_mime.c, which defines the file typing handler which emulates the NCSA server's behavior of determining file types from suffixes. What we'll be looking at, here, is the code which implements the AddType and AddEncoding commands. These commands can appear in .htaccess files, so they must be handled in the module's private per-directory data, which in fact, consists of two separate tables for MIME types and encoding information, and is declared as follows:

table *forced_types;      /* Additional AddTyped stuff */
table *encoding_types;    /* Added with AddEncoding... */
mime_dir_config;

When the server is reading a configuration file, or <Directory> section, which includes one of the MIME module's commands, it needs to create a mime_dir_config structure, so those commands have something to act on. It does this by invoking the function it finds in the module's `create per-dir config slot', with two arguments: the name of the directory to which this configuration information applies (or NULL for srm.conf), and a pointer to a resource pool in which the allocation should happen.

(If we are reading a .htaccess file, that resource pool is the per-request resource pool for the request; otherwise it is a resource pool which is used for configuration data, and cleared on restarts. Either way, it is important for the structure being created to vanish when the pool is cleared, by registering a cleanup on the pool if necessary).

For the MIME module, the per-dir config creation function just ap_pallocs the structure above, and a creates a couple of tables to fill it. That looks like this:

void *create_mime_dir_config (pool *p, char *dummy)
mime_dir_config *new = (mime_dir_config *) ap_palloc (p, sizeof(mime_dir_config));
 
new->forced_types = ap_make_table (p, 4);
new->encoding_types = ap_make_table (p, 4);

Now, suppose we've just read in a .htaccess file. We already have the per-directory configuration structure for the next directory up in the hierarchy. If the .htaccess file we just read in didn't have any AddType or AddEncoding commands, its per-directory config structure for the MIME module is still valid, and we can just use it. Otherwise, we need to merge the two structures somehow.

To do that, the server invokes the module's per-directory config merge function, if one is present. That function takes three arguments: the two structures being merged, and a resource pool in which to allocate the result. For the MIME module, all that needs to be done is overlay the tables from the new per-directory config structure with those from the parent:

void *merge_mime_dir_configs (pool *p, void *parent_dirv, void *subdirv)
mime_dir_config *parent_dir = (mime_dir_config *)parent_dirv;
mime_dir_config *subdir = (mime_dir_config *)subdirv;
mime_dir_config *new =  (mime_dir_config *)ap_palloc (p, sizeof(mime_dir_config));
new->forced_types = ap_overlay_tables (p, subdir->forced_types, parent_dir->forced_types);
new->encoding_types = ap_overlay_tables (p, subdir->encoding_types, parent_dir->encoding_types);

As a note --- if there is no per-directory merge function present, the server will just use the subdirectory's configuration info, and ignore the parent's. For some modules, that works just fine (e.g., for the includes module, whose per-directory configuration information consists solely of the state of the XBITHACK), and for those modules, you can just not declare one, and leave the corresponding structure slot in the module itself NULL.

Command handling

Now that we have these structures, we need to be able to figure out how to fill them. That involves processing the actual AddType and AddEncoding commands. To find commands, the server looks in the module's command table. That table contains information on how many arguments the commands take, and in what formats, where it is permitted, and so forth. That information is sufficient to allow the server to invoke most command-handling functions with pre-parsed arguments. Without further ado, let's look at the AddType command handler, which looks like this (the AddEncoding command looks basically the same, and won't be shown here):

char *add_type(cmd_parms *cmd, mime_dir_config *m, char *ct, char *ext)
if (*ext == '.') ++ext;
ap_table_set (m->forced_types, ext, ct);

This command handler is unusually simple. As you can see, it takes four arguments, two of which are pre-parsed arguments, the third being the per-directory configuration structure for the module in question, and the fourth being a pointer to a cmd_parms structure. That structure contains a bunch of arguments which are frequently of use to some, but not all, commands, including a resource pool (from which memory can be allocated, and to which cleanups should be tied), and the (virtual) server being configured, from which the module's per-server configuration data can be obtained if required.

Another way in which this particular command handler is unusually simple is that there are no error conditions which it can encounter. If there were, it could return an error message instead of NULL; this causes an error to be printed out on the server's stderr, followed by a quick exit, if it is in the main config files; for a .htaccess file, the syntax error is logged in the server error log (along with an indication of where it came from), and the request is bounced with a server error response (HTTP error status, code 500).

The MIME module's command table has entries for these commands, which look like this:

command_rec mime_cmds[] =
{ "AddType", add_type, NULL, OR_FILEINFO, TAKE2, "a mime type followed by a file extension" },
{ "AddEncoding", add_encoding, NULL, OR_FILEINFO, TAKE2, "an encoding (e.g., gzip), followed by a file extension" },

The entries in these tables are:

• The name of the command
• The function which handles it a (void *) pointer, which is passed in the cmd_parms structure to the command handler --- this is useful in case many similar commands are handled by the same function.
• A bit mask indicating where the command may appear. There are mask bits corresponding to each AllowOverride option, and an additional mask bit, RSRC_CONF, indicating that the command may appear in the server's own config files, but not in any .htaccess file.
• A flag indicating how many arguments the command handler wants pre-parsed, and how they should be passed in. TAKE2 indicates two pre-parsed arguments. Other options are TAKE1, which indicates one pre-parsed argument, FLAG, which indicates that the argument should be On or Off, and is passed in as a boolean flag, RAW_ARGS, which causes the server to give the command the raw, unparsed arguments (everything but the command name itself). There is also ITERATE, which means that the handler looks the same as TAKE1, but that if multiple arguments are present, it should be called multiple times, and finally ITERATE2, which indicates that the command handler looks like a TAKE2, but if more arguments are present, then it should be called multiple times, holding the first argument constant.
• Finally, we have a string which describes the arguments that should be present. If the arguments in the actual config file are not as required, this string will be used to help give a more specific error message. (You can safely leave this NULL).

Finally, having set this all up, we have to use it. This is ultimately done in the module's handlers, specifically for its file-typing handler, which looks more or less like this; note that the per-directory configuration structure is extracted from the request_rec's per-directory configuration vector by using the ap_get_module_config function.

Side notes --- per-server configuration, virtual servers, etc.

The basic ideas behind per-server module configuration are basically the same as those for per-directory configuration; there is a creation function and a merge function, the latter being invoked where a virtual server has partially overridden the base server configuration, and a combined structure must be computed. (As with per-directory configuration, the default if no merge function is specified, and a module is configured in some virtual server, is that the base configuration is simply ignored).

The only substantial difference is that when a command needs to configure the per-server private module data, it needs to go to the cmd_parms data to get at it. Here's an example, from the alias module, which also indicates how a syntax error can be returned (note that the per-directory configuration argument to the command handler is declared as a dummy, since the module doesn't actually have per-directory config data):

Continue Reading Page 2

THE Ultimate Htaccess originally appeared on AskApache.com

In part I: SEO Secrets of AskApache.com we talked about content and building a website to be your "SEO Base". This article discusses some advanced SEO concepts to get a site indexed, move your best pages higher in search results, and controlling the pagerank/seo-juice of your site. but relatively easy ways to control and tweak WHAT urls on your site are indexed, and HOW. I've heard some people refer to this as "controlling pagerank flow" and "controlling pagerank juice", basically we want our best pages to rank higher in the search engine results.

Big Picture: Going from no website to AskApache.com in less than a year can be accomplished by anyone with unique content and a resolve to avoid any shortcuts and take it one step at a time.

We want what Google wants, to provide the most relevant content for someone who is doing a search. Basically, you want every url on your site that has unique content to be included in the index. In other words, you have to think like a search engine!

The Goal of Google

Here's what I mean, Google approaches search with the overwhelming goal of bringing the content to a searcher that is the most-likely to be what that searcher is searching for. Another way of looking at it is something I read on Google...

Google's goal is to get you off of their site as fast as possible by providing you with exactly what you are looking for.

Think about SEO like this

If you search for htaccess tutorial for seo on Google, would you be more likely to visit a tutorial about using .htaccess for seo or a category page for htaccess articles? AskApache.com has both of those urls included in the index, but the article ranks higher than the category page, as it very well should.

Get Your URLS Indexed

Before I explain how I am able to help Google and other search engines rank my article pages higher than my category pages, we need to get the urls in the index or nothing will show up. There are many well-discussed methods for getting included in the index, so I'll just list a few that I use.

1. Provide High-Quality original content, people will link to it
2. Get a sitemap and use Google Webmaster Tools
3. Publish an RSS/Atom Feed and ping the net when you publish a post
4. Use robots.txt and robots meta tags

Your URLs in the index

Here's how to find out which of your pages are indexed.

• Indexed pages in your site: site:
• Pages that link to your site's front page: link:
• The current cache of your site: cache:
• Information about your site: info:
• Pages that are similar to your site: related:

Break It Down

Yo homeslice! I didn't mean break dance.. I mean lets simplify AskApache in the context of getting our urls indexed high/low. Here's the stats: 1 Homepage, 206 Articles, 19 Pages, 31 Categories

1 Homepage

This page is generally the highest ranking page in the index, it should contain links to your best urls, and provide easy navigation

206 Specific Topic Articles

These are the article's (like this one) of AskApache.com and are the main source of search engine traffic. You want each url (if its a good article) to be ranked as high as possible. Some keys are to really make each article specific to a topic by using best-practice (X)HTML.

19 Static Pages

Most of these are pages like the online-tools hosted on this site, or other basic pages like about, contact us, site-map, etc.. Some of these you may want to rank very high ( like the /about/ page ) and some you may not want to even be included in the index.

31 Specific Topic Category Pages

These are tricky because they are generally just lists of articles from each category, which isn't specific enough to get much seach-engine-traffic, but is very useful to site visitors. I beefed up my category pages by adding additional information about the category topic in addition to excerpts of the articles.

Higher Pagerank = Higher Up in Search Results

So Googlebot and other search engine robots have these crazy complicated algorithms (many patented) that SEO Industry types may get caught up in and try to technically analyze them. I'm sure you've seen/read/heard the complicated advice that will always be pushed by many... advice like:

• analyze the number of words in the description/title/1st paragraph/etc.
• Make sure your "keyword" is sprinkled throughout the text every 10-30th word..
• Other equally unexciting technical analysis

Now if you've had success with that then props to you, success is success, but I personally choose to completely ignore all that. The number 1 thing that the top search engines advise is to design your page for a Human Visitor, not a computer. The golden rule for me is how I would rank the page, not how some algorithm would.

Designing for a Human Visitor

This is a major factor in your site being at the top vs. nowhere. You design your HTML to be as minimal as possible (see source code for my homepage) and contain ONLY the neccessary elements. Above all, use semantically sound XHTML markup. (view source of W3C)

External CSS/Javascript

Get your javascript and CSS out of your HTML and use external files (like this site) ALWAYS! You should start with just the HTML, no css, no colors, no javascript, and THEN you add the .css and then you add the javascript.

Site Accessibility

Say your browser didn't have a mouse, didn't support images, css, javascript, or even colors! Your HTML should be structured such that your page is still easily readable and easy to navigate. You can use lynx, links, and many other terminal-based browsers to test for this... please see the Web Accessibility Initiative (WAI) for detailed info.

Web accessibility refers to the practice of making websites usable by people of all abilities and disabilities. When sites are correctly designed, developed and edited, all users can have equal access to information and functionality. For example, when a site is coded with semantically meaningful HTML, with textual equivalents provided for images and with links named meaningfully, this helps blind users using text-to-speech software and/or text-to-Braille hardware.

Controlling a URL's Pagerank

A few tools and techniques are available for controlling the "juice" or "pagerank" of your urls.

1. Robots.txt
2. Robots Meta Tags
3. Links

Robots.txt

I've done quite a bit of research and experimentation with robots.txt files, which is a file located in the root of your website at http://www.askapache.com/robots.txt that is downloaded by all legitimate search engine spiders/bots and used as a Blacklist to prevent certain urls from being indexed. Here are a few of the articles on this site, which you may skip if you like as they don't illustrate the big-picture that I am going to discuss now.

• Control Flow of Pagerank with robots.txt and NoFollow, NoIndex
• WordPress robots.txt
• SEO with Robots.txt
• Google AdSense using robots.txt
• WordPress robots.txt file optimized for SEO and Google

How To Use Robots.txt

Even though robots.txt files are for whitelisting and blacklisting urls, I have found that they should only be used as an extreme form of blacklisting. When you Disallow a url in your robots.txt file, that means most search engine bots won't even LOOK at the url. As you can see in the below example, I only disallow urls that shouldn't ever be LOOKED at. The real powertool is the robots meta tag.

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content
 
Sitemap: http://www.askapache.com/sitemap.xml

Robots Meta Tag

Ok I'm really trying to simplify, because what you should understand is the big-picture. Every page can have a robots meta tag in the header, and this robots meta tag can tell the search-engine to index/not-index AND follow/not-follow. Here are some examples:

<meta name="robots" content="index" />
<meta name="robots" content="noindex" />
<meta name="robots" content="noindex,follow" />
<meta name="robots" content="index,nofollow" />

content="index" / noindex

index means the search engine is free to index, archive, cache, and follow the page whereas noindex means DO NOT include this page in the search engine results.

content="follow" / nofollow

follow means the search engine is free to LOOK at the page and follow the links on the page whereas nofollow means DO NOT follow the links on the page.

WordPress Auto-Robots meta tag code

Just add this to any plugin file and it will add the right robots meta tag to your site.. tweak to taste.

function askapache_robots_header(){
 global $wpdb;
 
 $robot = '<meta name="robots" content="noindex,follow,nocache,noarchive" />';
 
 if ( is_paged() || is_search() || is_404() || is_author() || is_tag() )
   $robot = '<meta name="robots" content="noindex,follow" />';
 elseif ( is_home() || is_front_page() || is_single() )
   $robot = '<meta name="robots" content="follow,index" />';
 elseif ( is_category() || is_page() )
   $robot = '<meta name="robots" content="follow" />';
 
 echo $robot . "\n";
}
add_action( 'wp_head', 'askapache_robots_header' );

Links

External and Internal Links are the crux of SEO. It's important to start FIRST on your Internal Links and linking structure... Once you are satisfied that the correct pages are indexed and ranked appropriately, then you can begin to look at external links.

• The fewer links on a page, generally the better.
• If every page of your site points to the same url on your site, pagerank goes up from the number of internal links.
• Use of the rel, title, alt attribute semantically is very helpful. (next, prev, index)
• Put your best links higher up in the XHTML, and put helpful/solid links at the end.
• You can add rel="nofollow" to links that you dont want followed.

The web has gotten to be so full of malicious/non-helpful SEO activity that I recommend developing your content NOT external links. If you want to do this right and provide great content that makes search engine users happy and makes the web better, then explore this blog and develop content until the next article in this series, where I'll show you how to make your site explode.

Stay tuned for Part III, which will dive deeper into the pipeworks of AskApache.com

SEO Secrets of AskApache Part 2 originally appeared on AskApache.com

First let me say that I don't read SEO stuff, I don't participate in the SEO community, I only have an interest in regards to the technology used by the search engines. All websites are hosted on servers, mostly Apache, and that is the primary topic of this blog. During the past year of this blog, my research into non-seo areas has turned up some very valuable SEO techniques.. All of them legal, ethical, and genuinely good for the Internet at large.

Update: SEO Secrets part II: Advanced Indexing and Pagerank Control

Some Background

I started this blog in late 2006, my first foray into blogging, and I've been extremely successful at achieving top ten google rankings and maintaining on average 15K unique visitors/day (per google analytics) 85% of which come from search engine traffic.

NOTE: I take it for granted that anyone reading AskApache is an expert of some skill, if you aren't I apologize, I can't waste time on the easy stuff.

Prerequisite SEO

There are literally hundreds of thousands of SEO articles on the net, 99.9% of which are absolute garbage. Especially in the sense that they just repeat the same 10 year old stuff. However, to do any kind of advanced SEO like I am going to discuss in this article, I am assuming that you, the intelligent reader, has already read those and has a basic understanding of SEO fundamentals like meta tags, titles, keywords, etc.

First, Great Content

The foremost and most important step in achieving any kind of traffic is to produce great content. I'm sure you've heard that a million times, but let me break it down how I perceive it. Before I even started to mess with SEO for AskApache.com I began by writing articles. At that point I didn't have a clue what my blog was going to be about or even if I was going to be doing it after a week.

What is Content

For me, being a top-paid professional web developer, I spend about 80% of my time doing research. I think that is a bit uncommon, but its a throwback from the 10 years I spent in the network/computer security field, where research is 99% of the job, a story for another time perhaps.

So the research I was doing at that time was about best-practice standards-based web design, mainly XHTML Strict, CSS, and unobtrusive javascript. Each of those subjects has become near and dear to my heart, and each should also be mandatory learning for anyone interested in SEO. The best advice I can give towards that end is checking out the CSS, Javascript, and XHTML Strict source code for this page and site. And of course the holy W3.org.

In addition to striving to master those 3 subjects, I was also and always will be researching web programming languages like PHP, Ajax, Ruby, and Server Technology like Apache. Although I should note that my research into Apache and server technologies is more of a hobby than a job requirement, also a throwback to my days in the security industry and of course my love for open source software.

My Content

So basically I was spending 25% of my time at work actually working, and the other 75% of the time I would research how to do something better, faster, the best. Incredibly, I discovered or re-discovered a ton of tips, tricks, and methods to aid me in my work. I was learning so much valuable information that I joined a couple of forums to discuss them and get feedback on making them even better. Soon I realized that I was one of a small few who actually post content to a forum instead of just questions, so I decided to write my tutorials down on a blog, and AskApache was born.

So that is why this blog is comprised of almost 100% tutorials, and why almost all of them are completely original works you won't find elsewhere. That's how I create content, but you might do something different. Whatever it is that you do for content, just make sure you are providing VALUE with everything you do. Not to everyone, just stuff that you would consider to have value if you were reading it.

Second, Great Site

Ok so I had 10 or so great articles that I knew would provide value for many web developers, but so what? Nobody cares you know.. That's when I decided to take a closer look at the software that was running my new blog, WordPress, and I've been hacking the code ever since on my never-ending quest to be the best and know the most advanced web development. You'll see why in a couple paragraphs.

I Mean, a Really Great Site

By great, I mean you need to make it incredibly user-friendly. Every design and development decision you make should be about the visitor. THATS the number one key to success on the net, regardless of endeavor. Here is a list of things you definately need to have before you do SEO, I'm not listing obvious stuff like descriptions, titles, and good writing.

1. Intuitive and circular, your website should be a spider-web of urls.
2. Easy to read, plenty of white-space, design is your decision but I like minimalistic.
3. Super fast rendering. You need effective caching and optimization.
4. A very helpful 404 error page, hopefully never seen.

Focus in on your URL's

Many sites that use a CMS of some kind, be it Drupal or WordPress, have hundreds or thousands of URL's even if they only have 10 actual posts/articles.

Removing Duplicate Content

You've all heard this before, but almost no-one has taken it to the level I am going to discuss. Bear with me.

Removing duplicate content is actually a very straightforward process if you know what you are doing, and if you don't, well that's why I'm going to quickly explain how to really do a good job.

Locate Duplicate Content and URLS

People misunderstand that you should just not repeat the same paragraph in a different article, that is partially true, but the main impact this has on your site is if you can access the same article from more than a single URL.

I hope you realize you MUST use pretty urls like my site and not codey looking ones with question marks. You can find any potential duplicate urls on wordpress with the rewriterules plugin. Also look at Google's webmaster tools to look for any duplicate urls, and you can use xenus link sleuth tool as well.

Remove Duplicate Urls with .htaccess

Once you've found duplicate urls, you need to instruct google and other search engine robots to be redirected to the correct url. By doing a 301 redirect you tell the search engines NOT to index the bad url, only the good one. Below are some of the .htaccess code I use on this site to accomplish this technique, this is gold I myself use so pay attention. It works.

301 Redirects with mod_rewrite

First lets start with one everyone should know, and the most common, to www or not to www?

RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC]
RewriteRule ^(.*)$ http://www.askapache.com/$1 [R=301,L]

Its a highly rare individual who has seen this one, which forces requests for .html/ to .html

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /[^\.]+\.html/\ HTTP/ [NC]
RewriteRule ^(.*)\.html/$ http://www.askapache.com/$1.html [R=301,L]

301 Redirects without mod_rewrite

For the rest of the duplicate urls that you find, I like to use Redirect and RedirectMatch.

This redirects requests that start with an & or /&amp or /( to my homepage.

RedirectMatch 301 ^/&(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/&amp(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/([\(]+)(.*)$ http://www.askapache.com/

This redirects requests with //whatev to /whatev

RedirectMatch 301 ^//(.*)$ http://www.askapache.com/$1

But this is just a brief look at what you will have to spend some time on. There are detailed guides to doing this with mod_rewrite and using Redirect on my blog. Its time now for some real SEO tips. The heart of the matter, as it were.

Wrap It Up

So I realize that was brief, so I want to really stress 2 things or you won't take away much from part 1.

Locate Duplicate URLS

This is truly one of the most important things in my personal experience. I personally take this as extreme as I can, I regularly grep my access files, mod_security and error log files looking for bad URLS. I am always checking them out to see if someone has a bad link to me somewhere, or if someone just typed it in wrong. If its a bad link on a site, I will very politely attempt contacting the webmaster about it until they fix it.

Even I, with my many colorful years of Internet travel, was caught off-guard by the variety and creativity and the sheer number of urls people are using to link to my site. I found that often bad links would be published because my URL was just too long, so I shortened the URL's. Now of course bad links can't really even touch my site with all my 301's in place.

Besides grepping your server's logs, the 2nd best place to locate duplicate urls or just plain wrong urls is by using Googles free webmaster tools. They keep track of all the bad urls linking to your site and allow you to download this data in a .csv spreadsheet format. The first time I checked into this I found over 1,000 bad links, after a couple months with my RewriteRules and 301 Redirects, I've narrowed the list down to under 50 most months. That is a powerful reason to use 301 Redirects, as we'll really get into in part 2.

301 Redirect Bad URLS

Finding the bad urls takes some time, a couple hours even, and then the whole reason you do that is to be able to create 301 Redirects for all of those bad urls to good urls.

One reason that I wasn't even aware of until several months ago is that when Googlebot locates a bad URL for your site, it tries to access it, and if you haven't planned for this in advance, your page most likely will return a 200 OK status header, or if you are lucky a 404 Not Found error, both of which really hurt you.

Basically, a 200 response will produce duplicate content in 99.9% of the time, and 404 responses will whisper to Google's algorithms that you don't know what you are doing and your site isn't authoritative. 200 means google will index your site, 404 means google won't index your site, but it also won't give up trying for awhile, which takes away from your real urls.

What a 301 Response tells Google

301 Responses were practically invented for user-agents/clients/web-crawling robots like google. They instruct the client, whether that be a persons browser or a googlebot, that the resource/page that they are looking for is actually at a different URL. This is an authoritative response that makes googlebot and other search engines ecstatic because now they can give up on the 200 and 404 responses that didn't really give them an answer either way.

On the other hand, a great 404 can and should be just as powerful as a 301, but hardly anyone uses them in the correct way according to HTTP 1.1 or 1.0 Specifications. We'll tear that subject apart further down the road.

I'll leave this topic for now with one last idea, 301 Redirects when implemented and used correctly, actually redirect the page rank and search engine ranking for itself to the redirected / correct url. That means if you have 1000 unique links pointing to your article, and all of them are incorrect in some way, if you can 301 redirect all of those bad links to your correct link you now have 1000 new good links! It has to be done right and in a classy way though of course.

Now that you have content and a great site, its time to SEO like a mofo.

SEO Secrets of AskApache.com originally appeared on AskApache.com