Advanced Htaccess – SSI, ErrorDocuments, DirectoryIndexing SEO
Advanced Htaccess – SSI, ErrorDocuments, DirectoryIndexing SEO
« Password Protection Plugin StatusHtaccess SEO Trends by Google »
SSI, Server Side Includes, can be very useful to webmasters and visitors alike. They are easily set up in an .htaccess file and require little to no maintaining.. the term “Set it and Forget it” applies here. Before we look at how to implement SSI, and dig into the Apache module that makes them possible, lets look at few uses for SSI so you can quickly determine if you would like to read further.
Improved Directory Listings and SEO
One way that I use SSI is for improved Directory Listing. Anyone here like SEO? Check out my WordPress uploads directory to get an idea. Basically I can customize the header and footer of each directory.. Pretty sweet, thanks Apache!
In your /uploads/.htaccess
# turn on auto-indexing and turn off SSI's ability to exec Options None Options SymLinksIfOwnerMatch Indexes IncludesNOEXEC # we need to make sure files are displayed when requested, not executed or parsed AddType text/plain .ini .sh .bsh .bash .csh .var .asc .md5 .sha .sha1 .cgi .pl .php .inc .asp .exe .bin DefaultType text/plain # turn on auto-indexing, with askapache-optimized options IndexOptions FancyIndexing SuppressColumnSorting SuppressHTMLPreamble IconHeight=22 IconWidth=20 IndexOptions IgnoreClient NameWidth=40 DescriptionWidth=* XHTML FoldersFirst # don't show these files and folders IndexIgnore .htaccess .ht* *_notes *.log feed inc HEADER.html FOOTER.html feed*.gif # the SSI files used for the header and footer HeaderName /ssi/HEADER.html ReadmeName /ssi/FOOTER.html # used to determine the time and for SSI output SetEnv TZ America/Indianapolis SetEnv SERVER_ADMIN webmaster@askapache.com
In your /ssi/.htaccess
# makes files ending in .html be filtered through the INCLUDES filter before being sent to client AddOutputFilter Includes html
My HEADER.html
<!--#set var="PAGETITLE" value="-- z.askapache.com" --> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html> <head> <title><!--#echo encoding="none" var="REQUEST_URI" --> <!--#echo encoding="none" var="PAGETITLE" --></title> <meta http-equiv="content-type" content="text/html; charset=UTF-8" /> <link rel="stylesheet" href="http://z.askapache.com/c/error.css" /> <link rev="made" href="mailto:<!--#echo encoding="url" var="SERVER_ADMIN" -->" /> </head> <body> <h1><a href="http://www.askapache.com/" title="AskApache Home" accesskey="1"> <img src="http://www.askapache.com/nlogo.jpg" height="75" alt="AskApache" /> </a> <!--#echo encoding="url" var="REQUEST_URI" --> <!--#echo encoding="none" var="PAGETITLE" --></h1> <hr />
My FOOTER.html
<p>Find the information you are looking for on the <a href="http://www.askapache.com/">AskApache Home page</a>
or the <a href="http://www.askapache.com/search<!--#echo encoding="url" var="REQUEST_URI" -->">
AskApache search page</a>.</p>
<hr />
<address>
<small>$Id:<!--#echo encoding="none" var="UNIQUE_ID" --> E:<!--#echo encoding="none" var="REDIRECT_STATUS" -->,v 1.30
<!--#config timefmt="%c" --><!--#echo var="DATE_LOCAL" --></small><br />
<small><!--#echo var="SERVER_SOFTWARE" --></small><br /><br />
z.askapache.com -- AskApache | <a href="http://www.askapache.com/about/">Webmaster</a>
| Copyright © 2009 AskApache<br />
</address>
<!--#if expr="$REMOTE_ADDR = 10.10.10.10" -->
<pre ><!--#printenv --></ pre>
<!--#endif -->
<script src="http://z.askapache.com/j/apache-0780.js" type="text/javascript"></script>
<script type="text/javascript">var pageTracker = _gat._getTracker("UA-7"+"321"+"53-38");
pageTracker._initData();pageTracker._trackPageview();</script>
</body>
</html>
Enhanced Error Pages
If you are using WordPress, I’m sure you are using my AskApache Google 404 Plugin, but whatever your error pages may be, using SSI you can make them better. My 404 Error Page is handled by WordPress and PHP, but all of my other error pages (codes 4xx to 5xx) are handled using SSI. You can check out any of them: /show-error-400, /show-error-403, /show-error-503, etc..
Notice the email note which has the subject prefilled? Thats one of the main uses for SSI, you can add forms to your errordocuments and get notified of problems which mean you can fix them.
Add this to your /.htaccess for each ErrorDocument you make.
ErrorDocument 503 /errordocs/503.html
My /errordocs/.htaccess
# turn on symlinks for rewrites and turn off SSI's ability to exec Options None Options SymLinksIfOwnerMatch IncludesNOEXEC # makes files ending in .html be filtered through the INCLUDES filter before being sent to client AddOutputFilter Includes html # this internal apache variable prevents your errordocs from allowing keep-alive connections SetEnv nokeepalive # used to determine the time and for SSI output SetEnv TZ America/Indianapolis SetEnv SERVER_ADMIN webmaster@askapache.com
My /errordocs/503.html
<!--#set var="TITLE" value="Service Temporarily Unavailable" --> <!--#include virtual="/errordocs/TOP.html" --> <p>The server is temporarily unable to service your request due to <strong>maintenance downtime</strong> or super-crazy-extreme capacity problems. Please try again later... Or <a href="mailto:<!--#echo encoding="url" var="SERVER_ADMIN" -->">send me an email</a> and let me know about it..</p> <!--#include virtual="/errordocs/BOTTOM.html" -->
My /errordocs/TOP.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html> <head> <title><!--#echo encoding="none" var="REQUEST_URI" --> <!--#echo encoding="none" var="TITLE" --></title> <meta http-equiv="content-type" content="text/html; charset=UTF-8" /> <link rel="stylesheet" href="http://z.askapache.com/c/error.css" /> <link rev="made" href="mailto:<!--#echo encoding="url" var="SERVER_ADMIN" -->" /> </head> <body> <h1><a href="http://www.askapache.com/" title="AskApache Home" accesskey="1"><img src="http://www.askapache.com/nlogo.jpg" height="75" alt="AskApache" /> </a> <!--#echo encoding="url" var="REQUEST_URI" --> <!--#echo encoding="none" var="TITLE" --></h1> <hr />
My /errordocs/BOTTOM.html
<p>If this should not be an error please <a href="mailto:webmaster@askapache.com?subject=ID#<!--#echo encoding="none" var="UNIQUE_ID" -->">email</a> me right away.</p> <!--#if expr="$HTTP_REFERER" --> <p>You came from <!--#echo var="HTTP_REFERER"--></p> <!--#endif --> <p>If you still have a question, please try to find the information you are looking for on the <a href="http://www.askapache.com/">AskApache Home page</a> or the <a href="http://www.askapache.com/search/">AskApache search page</a>.</p> <hr /> <address> <a href="http://www.askapache.com/about/">Webmaster</a> $Id:<!--#echo encoding="none" var="UNIQUE_ID" --> Error-<!--#echo encoding="none" var="REDIRECT_STATUS" -->,v 1.30 <!--#config timefmt="%c" --><!--#echo var="DATE_LOCAL" --><br /> <!--#echo var="SERVER_SOFTWARE" --> </address> <!--#if expr="$REMOTE_ADDR = 10.10.10.10" --> <pre ><!--#printenv --></ pre> <!--#endif --> </body> </html>
Dealing with Spam and Website Attacks
Say you are using some nice .htaccess rewrite rules to block offending bots, web scrapers, and other nefarious net characters. Instead of just sending a 403 Denied, you could send them to be handled by an SSI document that could do any number of things.. From adding the offending bot’s IP address to the .htaccess Deny List (blacklisting), emailing you with an alert, emailing the IP Block Owner, executing a denial-of-service response ala guardian (script will return an artificially high Content-Length, and will then spoon-feed content bytes back to the client at a rate of one byte per second, for single-threaded or fixed-threadpool clients, this will hang all of their requests and render the attack inoperable), or just output a frightening looking message which usually does the trick if the bot is humanoid.
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\=?(http|ftp|ssl|https):/.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\?.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(asp|ini|dll).*\ HTTP/ [NC]
RewriteRule .* - [F]
ErrorDocument 403 /errordocs/f-off.html
Example /errordocs/f-off.html View it Live (Not Responsible for mental or physical harm caused by fright)
<html><head><title>WARNING <!--#echo encoding="none" var="REMOTE_ADDR" --> - SECURITY TEAM ALERTED</title></head><body> <h2>ATTENTION <!--#echo encoding="none" var="REMOTE_ADDR" --></h2> <p>You have been flagged by our system as a potential threat.</p> <p>This request has been logged and the Security Team has been notified.<br> Repeated attempts from IP address <strong><!--#echo encoding="none" var="REMOTE_ADDR" --></strong> will result <br> in the IP automatically being <strong>blacklisted</strong> on this server<br> and sent to blacklists around the world. Additionally, the Security Team<br> WILL alert the IP Address Block Owner and/or contact the authorities.<br><br></p> <h3>LOGGED INFORMATION FROM THIS REQUEST</h3> < pre> REMOTE_ADDR : <!--#echo encoding="none" var="REMOTE_ADDR" --> REMOTE_PORT : <!--#echo encoding="none" var="REMOTE_PORT" --> REQUEST_METHOD: <!--#echo encoding="none" var="REQUEST_METHOD" --><!--#if expr="$HTTP_REFERER" --> REFERER : <!--#echo encoding="url" var="HTTP_REFERER" --><!--#endif --> REQUEST_URI : <!--#echo encoding="url" var="REQUEST_URI" --><!--#if expr="$HTTP_USER_AGENT" --> USER_AGENT :</ pre> < pre style="white-space:normal;margin-left:150px;max-width:80%;"> <!--#echo encoding="none" var="HTTP_USER_AGENT" --> </ pre> < pre><!--#endif --><!--#if expr="$HTTP_COOKIE" --> COOKIES :</ pre> < pre style="white-space:normal;margin-left:150px;max-width:80%;"> <!--#echo encoding="none" var="HTTP_COOKIE" --> </ pre> < pre><!--#endif --><!--#if expr="$QUERY_STRING" --> QUERY_STRING : <!--#echo encoding="url" var="QUERY_STRING" --><!--#endif --> </ pre></body></html>
Server Side Includes Detailed Info
Ok now that we have the real-world usage out of the way, lets dig in a bit to the actual module mod_include, which if you want you can view the source code here.
Enabling Server-Side Includes
Server Side Includes are implemented by the INCLUDES filter. For backwards compatibility, the server-parsed handler also activates the INCLUDES filter. As well, Apache will activate the INCLUDES filter for any document with mime type text/x-server-parsed-html or text/x-server-parsed-html3 (and the resulting output will have the mime type text/html). If documents containing server-side include directives are given the extension .shtml, the following directives will make Apache parse them and assign the resulting document the mime type of text/html:
AddType text/html .shtml AddOutputFilter INCLUDES .shtml
The following directive must be given for the directories containing the shtml files (typically in a
Options +Includes
Server-Side Include Directives
These are the Directives allowed in .htaccess files that are handled by mod_include. Note that other modules may add additional directives, for instance the exec SSI Directive is supplied by mod_cgi. This is how mod_cgi registers with mod_include to provide processing of the exec directive. This is the code required to handle the “exec” SSI directive.
cgi_pfn_reg_with_ssi = APR_RETRIEVE_OPTIONAL_FN(ap_register_include_handler);
cgi_pfn_reg_with_ssi("exec", handle_exec);
static const char * const aszPre[] = { "mod_include.c", NULL };
config- Controls various aspects of the parsing.
<!--#config [timefmt="..."] [sizefmt="..."] [errmsg="..."] --> <!--#config [echomsg="..."] --> <!--#config errmsg="[It appears that you don't know how to use SSI]" -->
echomsg- (since 2.1) The value is a message that is sent back to the client if the
<a href="#element.echo">echo</a>element attempts to echo an undefined variable. This overrides any<a href="#ssiundefinedecho">SSIUndefinedEcho</a>directives. errmsg- The value is a message that is sent back to the client if an error occurs while parsing the document. This overrides any
<a href="#ssierrormsg">SSIErrorMsg</a>directives. sizefmt- The value sets the format to be used which displaying the size of a file. Valid values are
bytesfor a count in bytes, orabbrevfor a count in Kb or Mb as appropriate, for example a size of 1024 bytes will be printed as “1K”. timefmt- The value is a string to be used by the
strftime(3)library routine when printing dates.
echo- Prints one of the include variables. If the variable is unset, the result is determined by the SSIUndefinedEcho directive.
<!--#echo [encoding="none|url|entity"] var="..." [encoding="none|url|entity"] var="..." ... -->
var- The value is the name of the variable to print.
encoding-
Specifies how Apache should encode special characters contained in the variable before outputting them. If set to
none, no encoding will be done. If set tourl, then URL encoding (also known as %-encoding; this is appropriate for use within URLs in links, etc.) will be performed. The default is set toentity, resulting in entity encoding.
exec- Execute external programs
<!--#exec cgi="/cgi-bin/s.cgi" --> <!--#exec cmd="ls" --> <!--#include virtual="/cgi-bin/s.cgi?argument=value" --> <!--#exec cmd="perl /cgi-bin/s.pl args" -->
include- Include a file
<!--#include virtual|file="..." [virtual|file="..."] ... -->
printenv- Print all available variables
<!--#printenv -->
set- Set a value of a variable.
<!--#set var="..." value="..." ... --> <!--#set var="modified" value="$LAST_MODIFIED" --> <!--#set var="name" value="AskApache" --> <!--#set var="date" value="${DATE_LOCAL}_${DATE_GMT}" --> flastmod- Prints the last modification date of the specified file, subject to the timefmt format specification.
<!--#flastmod virtual|file="..." [virtual|file="..."] ... -->
fsize- Prints the size of the specified file, subject to the sizefmt format specification.
<!--#fsize virtual|file="..." [virtual|file="..."] ... -->
if- The if element works like an if statement in a programming language. The test condition is evaluated and if the result is true, then the text until the next elif, else or endif element is included in the output stream.
<!--#if expr="..." --> <!--#if expr="${REMOTE_USER} && ${HTTP_USER_AGENT}" --><!--#if expr="test_condition" --> <!--#elif expr="test_condition" --> <!--#else --> <!--#endif -->
elif- Used to put text into the output stream if the original test_condition was false.
<!--#elif expr="..." -->
else- Used to put text into the output stream if the original test_condition was false.
<!--#else -->
endif- Ends the if element and is required.
<!--#endif -->
.htaccess directives
- XBitHack
- SSIErrorMsg
- SSITimeFormat
- SSIStartTag
- SSIEndTag
- SSIUndefinedEcho
- SSIAccessEnable
mod_include Default SSI Values
START_SEQUENCE<!--#– The starting tag for mod_include to recognize and parse as SSI.END_SEQUENCE-->- The ending tag for mod_include to recognize and parse as SSI.ERROR_MSG[an error occurred while processing this directive]- On Errors parsing SSI.TIME_FORMAT%A, %d-%b-%Y %H:%M:%S %Z- Default Time format for DATEUNDEFINED_ECHO(none)- When echoing an undefined variable.
SSI Variables
DATE_GMT=Sun Mar 8 22:58:56 2009 DATE_LOCAL=Sun Mar 8 15:58:56 2009 DOCUMENT_NAME=FOOTER.html DOCUMENT_ROOT=/root-srv/protected/askapache.com/sec DOCUMENT_URI=/includes/FOOTER.html GATEWAY_INTERFACE=CGI/1.1 HTTP_ACCEPT=text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 HTTP_ACCEPT_CHARSET=ISO-8859-1,utf-8;q=0.7,*;q=0.7 HTTP_ACCEPT_ENCODING=gzip,deflate HTTP_ACCEPT_LANGUAGE=en-us,en;q=0.5 HTTP_CACHE_CONTROL=max-age=0 HTTP_CONNECTION=keep-alive HTTP_COOKIE=__qca=12298910-686528-46510; __utmb=50625.1.0.11311 HTTP_HOST=www.askapache.com HTTP_KEEP_ALIVE=300 HTTP_REFERER=http://www.askapache.com/htaccess/htaccess.html HTTP_USER_AGENT=Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 (.NET CLR 3.5.30729) LAST_MODIFIED=Sun Mar 8 14:53:50 2009 PATH=/bin:/usr/bin:/sbin:/usr/sbin QUERY_STRING= REMOTE_ADDR=24.123.215.60 REMOTE_PORT=4785 REQUEST_METHOD=GET REQUEST_URI=/htaccess/ SCRIPT_FILENAME=/root-srv/protected/askapache.com/sec/includes/FOOTER.html SCRIPT_NAME=/includes/FOOTER.html SCRIPT_URI=http://www.askapache.com/htaccess/ SCRIPT_URL=/htaccess/ SERVER_ADDR=64.111.114.111 SERVER_ADMIN=webmaster@askapache.com SERVER_NAME=www.askapache.com SERVER_PORT=80 SERVER_PROTOCOL=INCLUDED SERVER_SIGNATURE= SERVER_SOFTWARE=Apache/2.0.61 (Unix) PHP/4.4.7 mod_ssl/2.0.63 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2 SVN/1.4.2 UNIQUE_ID=dnbtH0Bvcm8A2ZHqcAAAAM USER_NAME=
More SSI Information
« Password Protection Plugin Status
Htaccess SEO Trends by Google »
Similar Reads
- Using TIME_HOUR and TIME_MIN for htaccess RewriteCond
- Apache Variable Fun in htaccess
- New Version of Password Protection Plugin
- .htaccess – Wikipedia
- Make phpBB SEO friendly with htaccess
- Security Enhancing with htaccess
- Replacing %23 with # in incoming links
- Manipulating HTTP Headers with htaccess
- PHP htaccess tips and tricks
- Security with Apache htaccess Tutorial
- Apache HTTPD Google CSE
- Speed Tips: Remove Last-Modified Header
- Rewrite underscores to hyphens for SEO URL
- Apache HTTPD and Module API Versions
- Speed Tips: Add Future Expires Headers
- Securing php.ini and php.cgi with .htaccess
Tags: errordocument, htaccess, htaccess rewrite, mod_include, Server Side Includes, SSI
My Picks
- THE Mod_Rewrite Cheatsheet
- AskApache .bash_profile
- Elite Log Viewing
- THE Ultimate Htaccess
- Intense Windows Speed
- DNS: Round Robin Speed
- Fsockopen: Nuts
Related Articles
- Using TIME_HOUR and TIME_MIN for htac
- Apache Variable Fun in htaccess
- New Version of Password Protection Pl
- .htaccess – Wikipedia
- Make phpBB SEO friendly with htaccess
- Security Enhancing with htaccess
- Replacing %23 with # in incoming link
- Manipulating HTTP Headers with htacce
- PHP htaccess tips and tricks
- Security with Apache htaccess Tutoria
- Apache HTTPD Google CSE
- Speed Tips: Remove Last-Modified Head
- Rewrite underscores to hyphens for SE
- Apache HTTPD and Module API Versions
- Speed Tips: Add Future Expires Header
- Securing php.ini and php.cgi with .ht
Newest Posts
- Update: Best Free Online Banking
- Firefox Add-ons for Web Developers
- Optimizing Servers and Processes for Speed with ionice, nice, ulimit
Random
Donate
Please consider donating to support active development of the free software and articles here.
Good Causes
The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect. Tim Berners-Lee
Reader Comments
It's very simple - you read the protocol and write the code. -Bill Joy
HTML | DCMI | GRDDL | XOXO | XDMP | XFN | DOM | XML | XHTML 1.1 Strict | CSS 2.1 | W3C | TLDP | WAI | DISA | ICSI | GIAC | SANS RR | GHOST | DEFCON | NIST | DHS CYBER | NIST | .:: Phrack Magazine ::.
↑ TOPExcept where otherwise noted, content on this site is licensed under a Creative Commons Attribution 3.0 License, just credit with a link.
This site is not supported or endorsed by The Apache Software Foundation (ASF). All software and documentation produced by The ASF is licensed. "Apache" is a trademark of The ASF. HTTPD based on NCSA HTTPd
what is with the strike through? its make it very hard to read ur blog :s