AskApache » Apache http://www.askapache.com Advanced Web Development Fri, 25 Jun 2010 00:55:59 +0000 en hourly 1 PHP Session File Hacks http://www.askapache.com/php/php-session-hack.html http://www.askapache.com/php/php-session-hack.html#comments Fri, 25 Jun 2010 00:00:09 +0000 AskApache http://www.askapache.com/?p=1019 What they say about kung-fu is true..

It can be attained by anyone through hard work over time. You can become as good as the amount of work you put in. Here's a short look at a basic technique that I use. Simply reverse engineering the source code and taking notes along the way...

static void php_session_send_cookie(TSRMLS_D)
  if (SG(headers_sent)) {
          if (output_start_filename) {
                  php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cookie - headers already sent by (output started at %s:%d)",
                          output_start_filename, output_start_lineno);
          } else {
                  php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cookie - headers already sent");
          }
          return;
  }
 
  /* URL encode session_name and id because they might be user supplied */
  e_session_name = php_url_encode(PS(session_name), strlen(PS(session_name)), NULL);
]]> If you want to learn best tricks and tips, there’s only one way to do it… at least only one way that I know of. Here are some notes I created while learning about the intricacies of php sessions, it’s all in the code.

[Session]
; Handler used to store/retrieve data.
session.save_handler = files

Argument passed to save_handler. In the case of files, this is the path where data files are stored. As of PHP 4.0.1, you can define the path as:

session.save_path = "N;/path"

where N is an integer. Instead of storing all the session files in
/path, what this will do is use subdirectories N-levels deep, and
store the session data in those directories. This is useful if you
or your OS have problems with lots of files in one directory, and is
a more efficient layout for servers that handle lots of sessions.

;
; NOTE 1: PHP will not create this directory structure automatically.
;         You can use the script in the ext/session dir for that purpose.
; NOTE 2: See the section on garbage collection below if you choose to
;         use subdirectories for session storage
;
; The file storage module creates files using mode 600 by default.
; You can change that by using
;
;     session.save_path = "N;MODE;/path"
;
; where MODE is the octal representation of the mode. Note that this
; does not overwrite the process's umask.
;session.save_path = "/tmp"

session.c

/* {{{ PHP_INI
 */
PHP_INI_BEGIN()
        STD_PHP_INI_BOOLEAN("session.bug_compat_42",    "1",         PHP_INI_ALL, OnUpdateBool,   bug_compat,         php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.bug_compat_warn",  "1",         PHP_INI_ALL, OnUpdateBool,   bug_compat_warn,    php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.save_path",          "",          PHP_INI_ALL, OnUpdateSaveDir,save_path,          php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.name",               "PHPSESSID", PHP_INI_ALL, OnUpdateString, session_name,       php_ps_globals,    ps_globals)
        PHP_INI_ENTRY("session.save_handler",           "files",     PHP_INI_ALL, OnUpdateSaveHandler)
        STD_PHP_INI_BOOLEAN("session.auto_start",       "0",         PHP_INI_ALL, OnUpdateBool,   auto_start,         php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.gc_probability",     "1",         PHP_INI_ALL, OnUpdateLong,    gc_probability,     php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.gc_divisor",         "100",       PHP_INI_ALL, OnUpdateLong,    gc_divisor,        php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.gc_maxlifetime",     "1440",      PHP_INI_ALL, OnUpdateLong,    gc_maxlifetime,     php_ps_globals,    ps_globals)
        PHP_INI_ENTRY("session.serialize_handler",      "php",       PHP_INI_ALL, OnUpdateSerializer)
        STD_PHP_INI_ENTRY("session.cookie_lifetime",    "0",         PHP_INI_ALL, OnUpdateLong,    cookie_lifetime,    php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cookie_path",        "/",         PHP_INI_ALL, OnUpdateString, cookie_path,        php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cookie_domain",      "",          PHP_INI_ALL, OnUpdateString, cookie_domain,      php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.cookie_secure",    "",          PHP_INI_ALL, OnUpdateBool,   cookie_secure,      php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.cookie_httponly",  "",          PHP_INI_ALL, OnUpdateBool,   cookie_httponly,    php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.use_cookies",      "1",         PHP_INI_ALL, OnUpdateBool,   use_cookies,        php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.use_only_cookies", "0",         PHP_INI_ALL, OnUpdateBool,   use_only_cookies,   php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.referer_check",      "",          PHP_INI_ALL, OnUpdateString, extern_referer_chk, php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.entropy_file",       "",          PHP_INI_ALL, OnUpdateString, entropy_file,       php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.entropy_length",     "0",         PHP_INI_ALL, OnUpdateLong,    entropy_length,     php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cache_limiter",      "nocache",   PHP_INI_ALL, OnUpdateString, cache_limiter,      php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cache_expire",       "180",       PHP_INI_ALL, OnUpdateLong,    cache_expire,       php_ps_globals,    ps_globals)
        PHP_INI_ENTRY("session.use_trans_sid",          "0",         PHP_INI_ALL, OnUpdateTransSid)
        STD_PHP_INI_ENTRY("session.hash_function",      "0",         PHP_INI_ALL, OnUpdateLong,    hash_func,          php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.hash_bits_per_character",      "4",         PHP_INI_ALL, OnUpdateLong,    hash_bits_per_character,          php_ps_globals,    ps_globals)
 
        /* Commented out until future discussion */
        /* PHP_INI_ENTRY("session.encode_sources", "globals,track", PHP_INI_ALL, NULL) */
PHP_INI_END()
/* }}} */

Session Errors

The session id contains illegal characters, valid characters are a-z, A-Z, 0-9 and '-,'
fcntl(%d, F_SETFD, FD_CLOEXEC) failed: %s (%d)
open(%s, O_RDWR) failed: %s (%d)
ps_files_cleanup_dir: opendir(%s) failed: %s (%d)
read failed: %s (%d)
read returned less bytes than requested
write failed: %s (%d)
write wrote less bytes than requested
mm_malloc failed, avail %d, err %s
cannot allocate new data segment
Skipping numeric key %ld.
A session is active. You cannot change the session module's ini settings at this time.
Cannot find save handler %s
Cannot find serialization handler %s
Unknown session.serialize_handler. Failed to encode session object.
Cannot encode non-existent session.
Unknown session.serialize_handler. Failed to decode session object.
Failed to decode session object. Session has been destroyed.
Invalid session hash function
The ini setting hash_bits_per_character is out of range (should be 4, 5, or 6) - using 4 for now
No storage module chosen - failed to initialize session.
Failed to initialize storage module: %s (path: %s)
The session bug compatibility code will not
Your script possibly relies on a session side-effect which existed until PHP 4.2.3. Please be advised that the session extension does not consider global variables as a source of data, unless register_globals is enabled. You can disable this functionality and this warning by setting session.bug_compat_42 or session.bug_compat_warn to off, respectively.
Failed to write session data (%s). Please
Cannot send session cache limiter - headers already sent (output started at %s:%d)
Cannot send session cache limiter - headers already sent
Cannot send session cookie - headers already sent by (output started at %s:%d)
Cannot send session cookie - headers already sent
Cannot find save handler %s
Cannot find unknown save handler
purged %d expired session objects
Trying to destroy uninitialized session
Session object destruction failed
Cannot find named PHP session module (%s)
Argument %d is not a valid callback
Cannot regenerate session id - headers already sent
Session object destruction failed
PS_GC_FUNC(files)
{
        PS_FILES_DATA;
 
        /* we don't perform any cleanup, if dirdepth is larger than 0.
           we return SUCCESS, since all cleanup should be handled by
           an external entity (i.e. find -ctime x | xargs rm) */
 
        if (data->dirdepth == 0) {
                *nrdels = ps_files_cleanup_dir(data->basedir, maxlifetime TSRMLS_CC);
        }
 
        return SUCCESS;
}

mod_files.c

/* If you change the logic here, please also update the error message in
 * ps_files_open() appropriately */
static int ps_files_valid_key(const char *key)
{
        size_t len;
        const char *p;
        char c;
        int ret = 1;
 
        for (p = key; (c = *p); p++) {
                /* valid characters are a..z,A..Z,0..9 */
                if (!((c >= 'a' && c <= 'z')
                                || (c >= 'A' && c <= 'Z')
                                || (c >= '0' && c <= '9')
                                || c == ','
                                || c == '-')) {
                        ret = 0;
                        break;
                }
        }
 
        len = p - key;
 
        if (len == 0) {
                ret = 0;
        }
 
        return ret;
}
static int ps_files_cleanup_dir(const char *dirname, int maxlifetime TSRMLS_DC)
{
        DIR *dir;
        char dentry[sizeof(struct dirent) + MAXPATHLEN];
        struct dirent *entry = (struct dirent *) &dentry;
        struct stat sbuf;
        char buf[MAXPATHLEN];
        time_t now;
        int nrdels = 0;
        size_t dirname_len;
 
        dir = opendir(dirname);
        if (!dir) {
                php_error_docref(NULL TSRMLS_CC, E_NOTICE, "ps_files_cleanup_dir: opendir(%s) failed: %s (%d)", dirname, strerror(errno), errno);
                return (0);
        }
 
        time(&now);
 
        dirname_len = strlen(dirname);
 
        /* Prepare buffer (dirname never changes) */
        memcpy(buf, dirname, dirname_len);
        buf[dirname_len] = PHP_DIR_SEPARATOR;
 
        while (php_readdir_r(dir, (struct dirent *) dentry, &entry) == 0 && entry) {
                /* does the file start with our prefix? */
                if (!strncmp(entry->d_name, FILE_PREFIX, sizeof(FILE_PREFIX) - 1)) {
                        size_t entry_len = strlen(entry->d_name);
 
                        /* does it fit into our buffer? */
                        if (entry_len + dirname_len + 2 < MAXPATHLEN) {
                                /* create the full path.. */
                                memcpy(buf + dirname_len + 1, entry->d_name, entry_len);
 
                                /* NUL terminate it and */
                                buf[dirname_len + entry_len + 1] = '\0';
 
                                /* check whether its last access was more than maxlifet ago */
                                if (VCWD_STAT(buf, &sbuf) == 0 &&
#ifdef NETWARE
                                                (now - sbuf.st_mtime.tv_sec) > maxlifetime) {
#else
                                                (now - sbuf.st_mtime) > maxlifetime) {
#endif
                                        VCWD_UNLINK(buf);
                                        nrdels++;
                                }
                        }
                }
        }
 
        closedir(dir);
 
        return (nrdels);
}

ext/session/mod_files.c

#define PS_FILES_DATA ps_files *data = PS_GET_MOD_DATA()
 
PS_OPEN_FUNC(files)
{
        ps_files *data;
        const char *p, *last;
        const char *argv[3];
        int argc = 0;
        size_t dirdepth = 0;
        int filemode = 0600;
 
        if (*save_path == '\0') {
                /* if save path is an empty string, determine the temporary dir */
                save_path = php_get_temporary_directory();
 
                if (strcmp(save_path, "/tmp")) {
                        if (PG(safe_mode) && (!php_checkuid(save_path, NULL, CHECKUID_CHECK_FILE_AND_DIR))) {
                                return FAILURE;
                        }
                        if (php_check_open_basedir(save_path TSRMLS_CC)) {
                                return FAILURE;
                        }
                }
        }
 
        /* split up input parameter */
        last = save_path;
        p = strchr(save_path, ';');
        while (p) {
                argv[argc++] = last;
                last = ++p;
                p = strchr(p, ';');
                if (argc > 2) break;
        }
        argv[argc++] = last;
 
        if (argc > 1) {
                errno = 0;
                dirdepth = (size_t) strtol(argv[0], NULL, 10);
                if (errno == ERANGE) {
                        php_error(E_WARNING, "The first parameter in session.save_path is invalid");
                        return FAILURE;
                }
        }
 
        if (argc > 2) {
                errno = 0;
                filemode = strtol(argv[1], NULL, 8);
                if (errno == ERANGE || filemode < 0 || filemode > 07777) {
                        php_error(E_WARNING, "The second parameter in session.save_path is invalid");
                        return FAILURE;
                }
        }
        save_path = argv[argc - 1];
 
        data = emalloc(sizeof(*data));
        memset(data, 0, sizeof(*data));
 
        data->fd = -1;
        data->dirdepth = dirdepth;
        data->filemode = filemode;
        data->basedir_len = strlen(save_path);
        data->basedir = estrndup(save_path, data->basedir_len);
 
        PS_SET_MOD_DATA(data);
 
        return SUCCESS;
}
[PHP 5.2.0 session.save_path safe_mode and open_basedir bypass]
 
Author: Maksymilian Arciemowicz (SecurityReason)
Date:
- - Written: 02.10.2006
- - Public: 08.12.2006
SecurityAlert Id: 43
CVE: CVE-2006-6383
SecurityRisk: High
Affected Software: PHP 5.2.0
Advisory URL: http://securityreason.com/achievement_securityalert/43
Vendor: http://www.php.net
 
- --- 0.Description ---
PHP is an HTML-embedded scripting language. Much of its syntax is borrowed from
C, Java and Perl with a couple of unique PHP-specific features thrown in. The
goal of the language is to allow web developers to write dynamically generated
pages quickly.
 
A nice introduction to PHP by Stig Sather Bakken can be found at
http://www.zend.com/zend/art/intro.php on the Zend website. Also, much  of the
PHP Conference Material is freely available.
 
Session support in PHP consists of a way to preserve certain data across
subsequent accesses. This enables you to build more customized applications and
increase the appeal of your web site.
 
A visitor accessing your web site is assigned a unique id, the so-called
session id. This is either stored in a cookie on the user side or is propagated
in the URL.
 
session.save_path defines the argument which is passed to the save handler. If
you choose the default files handler, this is the path where the files are
created. Defaults to /tmp. See also session_save_path().
 
There is an optional N argument to this directive that determines the number of
directory levels your session files will be spread around in. For example,
setting to '5;/tmp' may end up creating a session file and location like
/tmp/4/b/1/e/3/sess_4b1e384ad74619bd212e236e52a5a174If . In order to use N you
must create all of these directories before use. A small shell script exists in
ext/session to do this, it's called mod_files.sh. Also note that if N is used
and greater than 0 then automatic garbage collection will not be performed, see
a copy of php.ini for further information. Also, if you use N, be sure to
surround session.save_path in "quotes" because the separator (;) is also used
for comments in php.ini.
 
- --- 1. session.save_path safe mode and open basedir bypass ---
session.save_path can be set in ini_set(), session_save_path() function. In
session.save_path there must be path where you will save yours tmp file. But
syntax for session.save_path can be:
 
[/PATH]
 
OR
 
[N;/PATH]
 
N - can be a string.
 
EXAMPLES:
 
1. session_save_path("/DIR/WHERE/YOU/HAVE/ACCESS")
2. session_save_path("5;/DIR/WHERE/YOU/HAVE/ACCESS")
 
and
 
3.
session_save_path("/DIR/WHERE/YOU/DONT/HAVE/ACCESS\0;/DIR/WHERE/YOU/HAVE/ACCESS")
CACHE_LIMITER_FUNC(public)
{
        char buf[MAX_STR + 1];
        struct timeval tv;
        time_t now;
 
        gettimeofday(&tv, NULL);
        now = tv.tv_sec + PS(cache_expire) * 60;
#define EXPIRES "Expires: "
        memcpy(buf, EXPIRES, sizeof(EXPIRES) - 1);
        strcpy_gmt(buf + sizeof(EXPIRES) - 1, &now);
        ADD_HEADER(buf);
 
        snprintf(buf, sizeof(buf) , "Cache-Control: public, max-age=%ld", PS(cache_expire) * 60); /* SAFE */
        ADD_HEADER(buf);
 
        last_modified(TSRMLS_C);
}
 
CACHE_LIMITER_FUNC(private_no_expire)
{
        char buf[MAX_STR + 1];
 
        snprintf(buf, sizeof(buf), "Cache-Control: private, max-age=%ld, pre-check=%ld", PS(cache_expire) * 60, PS(cache_expire) * 60); /* SAFE */
        ADD_HEADER(buf);
 
        last_modified(TSRMLS_C);
}
 
CACHE_LIMITER_FUNC(private)
{
        ADD_HEADER("Expires: Thu, 19 Nov 1981 08:52:00 GMT");
        CACHE_LIMITER(private_no_expire)(TSRMLS_C);
}
 
CACHE_LIMITER_FUNC(nocache)
{
        ADD_HEADER("Expires: Thu, 19 Nov 1981 08:52:00 GMT");
        /* For HTTP/1.1 conforming clients and the rest (MSIE 5) */
        ADD_HEADER("Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0");
        /* For HTTP/1.0 conforming clients */
        ADD_HEADER("Pragma: no-cache");
}
 
static php_session_cache_limiter_t php_session_cache_limiters[] = {
        CACHE_LIMITER_ENTRY(public)
        CACHE_LIMITER_ENTRY(private)
        CACHE_LIMITER_ENTRY(private_no_expire)
        CACHE_LIMITER_ENTRY(nocache)
        {0}
};
 
static int php_session_cache_limiter(TSRMLS_D)
{
        php_session_cache_limiter_t *lim;
 
        if (PS(cache_limiter)[0] == '\0') return 0;
 
        if (SG(headers_sent)) {
                char *output_start_filename = php_get_output_start_filename(TSRMLS_C);
                int output_start_lineno = php_get_output_start_lineno(TSRMLS_C);
 
                if (output_start_filename) {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cache limiter - headers already sent (output started at %s:%d)",
                                output_start_filename, output_start_lineno);
                } else {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cache limiter - headers already sent");
                }
                return -2;
        }
 
        for (lim = php_session_cache_limiters; lim->name; lim++) {
                if (!strcasecmp(lim->name, PS(cache_limiter))) {
                        lim->func(TSRMLS_C);
                        return 0;
                }
        }
 
        return -1;
}
static void php_session_send_cookie(TSRMLS_D)
{
        smart_str ncookie = {0};
        char *date_fmt = NULL;
        char *e_session_name, *e_id;
 
        if (SG(headers_sent)) {
                char *output_start_filename = php_get_output_start_filename(TSRMLS_C);
                int output_start_lineno = php_get_output_start_lineno(TSRMLS_C);
 
                if (output_start_filename) {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cookie - headers already sent by (output started at %s:%d)",
                                output_start_filename, output_start_lineno);
                } else {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cookie - headers already sent");
                }
                return;
        }
 
        /* URL encode session_name and id because they might be user supplied */
        e_session_name = php_url_encode(PS(session_name), strlen(PS(session_name)), NULL);
        e_id = php_url_encode(PS(id), strlen(PS(id)), NULL);
 
        smart_str_appends(&ncookie, COOKIE_SET_COOKIE);
        smart_str_appends(&ncookie, e_session_name);
        smart_str_appendc(&ncookie, '=');
        smart_str_appends(&ncookie, e_id);
 
        efree(e_session_name);
        efree(e_id);
 
        if (PS(cookie_lifetime) > 0) {
                struct timeval tv;
                time_t t;
 
                gettimeofday(&tv, NULL);
                t = tv.tv_sec + PS(cookie_lifetime);
 
                if (t > 0) {
                        date_fmt = php_std_date(t TSRMLS_CC);
                        smart_str_appends(&ncookie, COOKIE_EXPIRES);
                        smart_str_appends(&ncookie, date_fmt);
                        efree(date_fmt);
                }
        }
 
        if (PS(cookie_path)[0]) {
                smart_str_appends(&ncookie, COOKIE_PATH);
                smart_str_appends(&ncookie, PS(cookie_path));
        }
 
        if (PS(cookie_domain)[0]) {
                smart_str_appends(&ncookie, COOKIE_DOMAIN);
                smart_str_appends(&ncookie, PS(cookie_domain));
        }
 
        if (PS(cookie_secure)) {
                smart_str_appends(&ncookie, COOKIE_SECURE);
        }
 
        if (PS(cookie_httponly)) {
                smart_str_appends(&ncookie, COOKIE_HTTPONLY);
        }
 
        smart_str_0(&ncookie);
 
        /*      'replace' must be 0 here, else a previous Set-Cookie
                header, probably sent with setcookie() will be replaced! */
        sapi_add_header_ex(ncookie.c, ncookie.len, 0, 0 TSRMLS_CC);
}

PHP Session File Hacks originally appeared on AskApache.com

]]> http://www.askapache.com/php/php-session-hack.html/feed/ 0 Crazy POWERFUL Bash Prompt http://www.askapache.com/linux-unix/bash-power-prompt.html http://www.askapache.com/linux-unix/bash-power-prompt.html#comments Tue, 25 May 2010 19:02:34 +0000 AskApache http://www.askapache.com/?p=4069 bash power prompt PS1This amazing bash linux prompt does more than meets the eye. If you want to know how to become really good with technology, linux is the secret sauce behind the AskApache articles. Open Source is elixir of the web. Thanks to everyone who helped me for the past 20 years. I use linux/bsd because homey don't play, so this is geared to be as productive a prompt as I can make it.

Don't have much time.. or just don't care? Not a problem, here are the 3 lines to copy and paste - you can just paste them right in your shell to test it, or add to a startup script.

export AA_P="export PVE=\"\\033[m\\033[38;5;2m\"\$(( \`sed -n \"s/MemFree:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\` / 1024 ))\"\\033[38;5;22m/\"\$((\`sed -n \"s/MemTotal:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\`/ 1024 ))MB\"\\t\\033[m\\033[38;5;55m\$(< /proc/loadavg)\\033[m\";echo -en \"\""
export PROMPT_COMMAND="history -a;((\$SECONDS % 10==0 ))&&eval \"\$AA_P\";echo -en \"\$PVE\";"
export PS1="\\[\\e[m\\n\\e[1;30m\\][\$\$:\$PPID \\j:\\!\\[\\e[1;30m\\]]\\[\\e[0;36m\\] \\T \\d \\[\\e[1;30m\\][\\[\\e[1;34m\\]\\u@\\H\\[\\e[1;30m\\]:\\[\\e[0;37m\\]\${SSH_TTY} \\[\\e[0;32m\\]+\${SHLVL}\\[\\e[1;30m\\]] \\[\\e[1;37m\\]\\w\\[\\e[0;37m\\] \\n(\$SHLVL:\\!)\\\$ " && eval $AA_P
]]> bash power prompt screenshotFrom setting the window title to show the last run command (filtered), saving your history and keeping multi-session history intact, resetting the color/cursor/highlighting of errant color-emitting commands, these 2 prompt examples do more than meet the eye and are extremely fast.

Don’t have much time or just don’t care? No problem, this is a simple copy and paste… here’s one to start (all 1 line, just cut and paste right into the shell, once you’ve made sure its safe).

PS1="\n\[\033[1;30m\][$$:$PPID - \j:\!\[\033[1;30m\]]\[\033[0;36m\] \T \
\[\033[1;30m\][\[\033[1;34m\]\u@\H\[\033[1;30m\]:\[\033[0;37m\]${SSH_TTY:-o} \
\[\033[0;32m\]+${SHLVL}\[\033[1;30m\]] \[\033[1;37m\]\w\[\033[0;37m\] \n\$ "

toprc

Or for POWER PROMPT

PROMPT_COMMAND='history -a;echo -en "\033[m\033[38;5;2m"$(( `sed -n "s/MemFree:[\t ]\+\([0-9]\+\) kB/\1/p" /proc/meminfo`/1024))"\033[38;5;22m/"$((`sed -n "s/MemTotal:[\t ]\+\([0-9]\+\) kB/\1/Ip" /proc/meminfo`/1024 ))MB"\t\033[m\033[38;5;55m$(< /proc/loadavg)\033[m"'
PS1='\[\e[m\n\e[1;30m\][$$:$PPID \j:\!\[\e[1;30m\]]\[\e[0;36m\] \T \d \[\e[1;30m\][\[\e[1;34m\]\u@\H\[\e[1;30m\]:\[\e[0;37m\]${SSH_TTY} \[\e[0;32m\]+${SHLVL}\[\e[1;30m\]] \[\e[1;37m\]\w\[\e[0;37m\] \n($SHLVL:\!)\$ '

Or for Extreme Power Prompt

Updated! 5/25/2010, This is what I use at the moment.. It’s the coolest code I’ve ever seen to do this. I rewrote the above (and below) bash prompts to this format because it is much more robust, and believe it or not its way faster for your machine. Yes the code below is correct, it looks a little weird to export vars from within an exported var statement.. but this is correct. If you just learn this one bit of code, you will gain a lot of shell kung fu, at least I have! Enjoy!

export AA_P="export PVE=\"\\033[m\\033[38;5;2m\"\$(( \`sed -n \"s/MemFree:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\` / 1024 ))\"\\033[38;5;22m/\"\$((\`sed -n \"s/MemTotal:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\`/ 1024 ))MB\"\\t\\033[m\\033[38;5;55m\$(< /proc/loadavg)\\033[m\";echo -en \"\""
export PROMPT_COMMAND="history -a;((\$SECONDS % 10==0 ))&&eval \"\$AA_P\";echo -en \"\$PVE\";"
export PS1="\\[\\e[m\\n\\e[1;30m\\][\$\$:\$PPID \\j:\\!\\[\\e[1;30m\\]]\\[\\e[0;36m\\] \\T \\d \\[\\e[1;30m\\][\\[\\e[1;34m\\]\\u@\\H\\[\\e[1;30m\\]:\\[\\e[0;37m\\]\${SSH_TTY} \\[\\e[0;32m\\]+\${SHLVL}\\[\\e[1;30m\\]] \\[\\e[1;37m\\]\\w\\[\\e[0;37m\\] \\n(\$SHLVL:\\!)\\\$ "
export PVE="\\033[m\\033[38;5;2m813\\033[38;5;22m/1024MB\\t\\033[m\\033[38;5;55m0.25 0.22 0.18 1/66 26820\\033[m" && eval $AA_P
[24574:16122 0:344] 05:49:07 Wed May 26 [faux@backtrack-askapache:/dev/pts/0 +1] ~
(1:344)$ export AA_P="export PVE=\"\\033[m\\033[38;5;2m\"\$(( \`sed -n \"s/MemFree:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\` / 1024 ))\"\\033[38;5;22m/\"\$((\`sed -n \"s/MemTotal:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\`/ 1024 ))MB\"\\t\\033[m\\033[38;5;55m\$(< /proc/loadavg)\\033[m\";echo -en \"\""
>>> export 'AA_P=export PVE="\033[m\033[38;5;2m"$(( `sed -n "s/MemFree:[\t ]\+\([0-9]\+\) kB/\1/p" /proc/meminfo` / 1024 ))"\033[38;5;22m/"$((`sed -n "s/MemTotal:[\t ]\+\([0-9]\+\) kB/\1/p" /proc/meminfo`/ 1024 ))MB"\t\033[m\033[38;5;55m$(< /proc/loadavg)\033[m";echo -en ""'
>>> AA_P='export PVE="\033[m\033[38;5;2m"$(( `sed -n "s/MemFree:[\t ]\+\([0-9]\+\) kB/\1/p" /proc/meminfo` / 1024 ))"\033[38;5;22m/"$((`sed -n "s/MemTotal:[\t ]\+\([0-9]\+\) kB/\1/p" /proc/meminfo`/ 1024 ))MB"\t\033[m\033[38;5;55m$(< /proc/loadavg)\033[m";echo -en ""'
[24574:16122 0:345] 05:49:09 Wed May 26 [faux@backtrack-askapache:/dev/pts/0 +1] ~
(1:345)$ export PROMPT_COMMAND="history -a;((\$SECONDS % 10==0 ))&&eval \"\$AA_P\";echo -en \"\$PVE\";"
>>> export 'PROMPT_COMMAND=history -a;(($SECONDS % 10==0 ))&&eval "$AA_P";echo -en "$PVE";'
>>> PROMPT_COMMAND='history -a;(($SECONDS % 10==0 ))&&eval "$AA_P";echo -en "$PVE";'
>>> history -a
>>> (( 66 % 10==0  ))
>>> echo -en '\033[m\033[38;5;2m813\033[38;5;22m/1024MB\t\033[m\033[38;5;55m0.35 0.24 0.19 1/68 27241\033[m'
813/1024MB      0.35 0.24 0.19 1/68 27241
[24574:16122 0:346] 05:49:09 Wed May 26 [faux@backtrack-askapache:/dev/pts/0 +1] ~
(1:346)$ export PS1="\\[\\e[m\\n\\e[1;30m\\][\$\$:\$PPID \\j:\\!\\[\\e[1;30m\\]]\\[\\e[0;36m\\] \\T \\d \\[\\e[1;30m\\][\\[\\e[1;34m\\]\\u@\\H\\[\\e[1;30m\\]:\\[\\e[0;37m\\]\${SSH_TTY} \\[\\e[0;32m\\]+\${SHLVL}\\[\\e[1;30m\\]] \\[\\e[1;37m\\]\\w\\[\\e[0;37m\\] \\n(\$SHLVL:\\!)\\\$ "
>>> export 'PS1=\[\e[m\n\e[1;30m\][$$:$PPID \j:\!\[\e[1;30m\]]\[\e[0;36m\] \T \d \[\e[1;30m\][\[\e[1;34m\]\u@\H\[\e[1;30m\]:\[\e[0;37m\]${SSH_TTY} \[\e[0;32m\]+${SHLVL}\[\e[1;30m\]] \[\e[1;37m\]\w\[\e[0;37m\] \n($SHLVL:\!)\$ '
>>> PS1='\[\e[m\n\e[1;30m\][$$:$PPID \j:\!\[\e[1;30m\]]\[\e[0;36m\] \T \d \[\e[1;30m\][\[\e[1;34m\]\u@\H\[\e[1;30m\]:\[\e[0;37m\]${SSH_TTY} \[\e[0;32m\]+${SHLVL}\[\e[1;30m\]] \[\e[1;37m\]\w\[\e[0;37m\] \n($SHLVL:\!)\$ '
>>> history -a
>>> (( 67 % 10==0  ))
>>> echo -en '\033[m\033[38;5;2m813\033[38;5;22m/1024MB\t\033[m\033[38;5;55m0.35 0.24 0.19 1/68 27241\033[m'
813/1024MB      0.35 0.24 0.19 1/68 27241
[24574:16122 0:347] 05:49:10 Wed May 26 [faux@backtrack-askapache:/dev/pts/0 +1] ~
(1:347)$ export PVE="\\033[m\\033[38;5;2m813\\033[38;5;22m/1024MB\\t\\033[m\\033[38;5;55m0.25 0.22 0.18 1/66 26820\\033[m"
>>> export 'PVE=\033[m\033[38;5;2m813\033[38;5;22m/1024MB\t\033[m\033[38;5;55m0.25 0.22 0.18 1/66 26820\033[m'
>>> PVE='\033[m\033[38;5;2m813\033[38;5;22m/1024MB\t\033[m\033[38;5;55m0.25 0.22 0.18 1/66 26820\033[m'
>>> history -a
>>> (( 67 % 10==0  ))
>>> echo -en '\033[m\033[38;5;2m813\033[38;5;22m/1024MB\t\033[m\033[38;5;55m0.25 0.22 0.18 1/66 26820\033[m'

256 Color Prompt Command

NOTE: I have been preparing a 256color terminal article for some time now, as I have tweaked, hacked, read, and experimented with more than I would ever like to admit. So stay tuned, there isn't any tutorial, book, or online howto remotely close to as good as what I'm close to posting...

Super-optimized 256color BASH Prompt

256 color prompt_command and PS1

If you have a 256-color enabled terminal try this..

    export PROMPT_COMMAND='echo -en "\033[m\033[38;5;2m"$(( `sed -n "s/MemFree:[\t ]\+\([0-9]\+\) kB/\1/p" /proc/meminfo`/1024))"\033[38;5;22m/"$((`sed -n "s/MemTotal:[\t ]\+\([0-9]\+\) kB/\1/Ip" /proc/meminfo`/1024 ))MB"\t\033[m\033[38;5;55m$(< /proc/loadavg)\033[m"'
    export PS1='\[\e[m\n\e[1;30m\][$$:$PPID \j:\!\[\e[1;30m\]]\[\e[0;36m\] \T \d \[\e[1;30m\][\[\e[1;34m\]\u@\H\[\e[1;30m\]:\[\e[0;37m\]${SSH_TTY} \[\e[0;32m\]+${SHLVL}\[\e[1;30m\]] \[\e[1;37m\]\w\[\e[0;37m\] \n($SHLVL:\!)\$ '

Which double-quoted is:

declare -x PROMPT_COMMAND="echo -en \"\\033[m\\033[38;5;2m\"\$(( \`sed -n \"s/MemFree:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\`/1024))\"\\033[38;5;22m/\"\$((\`sed -n \"s/MemTotal:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/Ip\" /proc/meminfo\`/1024 ))MB\"\\t\\033[m\\033[38;5;55m\$(< /proc/loadavg)\\033[m\""
declare -x PS1="\\[\\e[m\\n\\e[1;30m\\][\$\$:\$PPID \\j:\\!\\[\\e[1;30m\\]]\\[\\e[0;36m\\] \\T \\d \\[\\e[1;30m\\][\\[\\e[1;34m\\]\\u@\\H\\[\\e[1;30m\\]:\\[\\e[0;37m\\]\${SSH_TTY} \\[\\e[0;32m\\]+\${SHLVL}\\[\\e[1;30m\\]] \\[\\e[1;37m\\]\\w\\[\\e[0;37m\\] \\n(\$SHLVL:\\!)\\\$ "

For Commandlinefu.com users: Here ya go! toprc

That looks great, and is practicallly totally free in terms of how much cpu/work it makes the shell do for each command. So for slow terminals, or text-terminals like I use, this is great. Other than PS1, there is 1 other variable.. It’s called PROMPT_COMMAND and is execed right in front of the PS1. I show 2 ways to utilize this, the goal being to show there is no right way.. fast is fast and money is money. I use a function called aa_pc I threw together that shows me in color how much memory is on my system.. very cool.

PROMPT = ?

Here’s a screenshot of the code in my vim + screen multiplexed bash 4 environment.. It’s placed all the way at the bottom of the file and is the last thing executed by the client processing the script. Also, while it’s great to look so good, this is all 100% for efficiency since I use one shell or another every day.

Custom Power Prompt - currently mine

Custom Power Prompt - currently mine

Here’s the code I’m using today, that produced this screen-shot — I am always changing stuff so it’s a little more tough to figure out.

  function aa_pc()
  {
        local M=$( free -olm|sed "/Mem/!d; s/Mem:[^0-9]*\([0-9]*\)[^0-9]*\([0-9]*\)[^0-9]*\([0-9]*\).*/ \
        ${GREP_COLOR}\2MB${R}\/\3MB/"; ); echo -e "$M `pwd`"
  }
 
  aa_grep_color(){ export GREP_COLOR=`tput setaf $(aa_random_under 6)`; }
  aa_random_under(){ echo -en $(( $RANDOM % ${1:-$RANDOM} + 1 )); }
  uptime1(){ sed '/.*,  \([0-9]*\) users,  load average: \(.*\)/!d; s//[ \2, \1 users ]/;q';
 
  $SED -n '/i/!Q2' <<< $- && \
  {
        case ${TERM:-dummy} in xterm*|screen*|*color*)echo; ;; *) return; esac;
  }
 
  # hack for being able to source all the functions in this file from a script
  MASTER_RESET=$`echo -e "rs1\nrs2\ninitc\nis2\ncnorm\nrmso\nsgr0" | tput -S`
 
  # only for color-aware (some guessing) terminals, kick everyone else OUT
  case ${TERM:-dummy} in xterm*|screen*|*color*)echo; ;; *) return; esac;
 
  [[ -r /etc/DIR_COLORS ]] && eval `dircolors -b /etc/DIR_COLORS`
 
  # changes the window title in terminals like putty or konsole
  echo -ne "\033]0; ${USER}@${HOSTNAME}  +${SHLVL} @${SSH_TTY/\/dev\/} - `uptime1` \007"
 
  # show title, setup the colors, vars, funcs, settings
  asetup_colors && ascript_title
  export PROMPT_COMMAND='history -a;aa_pc'
  export PS1='\[\033[1;30m\][$$:$PPID - \j:\!\[\033[1;30m\]]\[\033[0;36m\] \T\
  \d \[\033[1;30m\][\[\033[1;34m\]\u@\H\[\033[1;30m\]:\[\033[0;37m\]${SSH_TTY} \
  \[\033[0;32m\]+${SHLVL}\[\033[1;30m\]] \[\033[1;37m\]\w\[\033[0;37m\] \n\$ '

bash power prompt screenshot

I’ve used many shells over the years and have learned many tweaks and tricks for a powerful prompt. It may look deceptively simple. One cool thing this prompt does is display the command line (filtered) in the putty/konsole/xterm window, so if you have several windows minimized you can easily find the one you are looking for.

There is a lot of good info about the BASH prompt around the net, but it can be complicated for a new user, and as an advanced user you will notice most of the info is very outdated and not very optimized for speed. To get this articles recommended prompt working is devastatingly simple, just export the PROMPT_COMMAND and PS1 variables that are shown below. Most “power prompts” that utilize things like PROMPT_COMMAND do so very haphazardly or very extravagantly.

I’ve been called a lot of things but not extravagant, and because I use this prompt on many different servers, platforms, and systems I made sure to not make things complicated. To test them just copy and paste the code below into your shell which will export PROMPT_COMMAND and PS1 variables and work right away. So test it out, and if you like it add it to your .bashrc or other startup file (please do a $ man bash).

Bash Power Prompt

bash power prompt PS1
From setting the window title to show the last run command (filtered), saving your history and keeping multi-session history intact, resetting the color/cursor/highlighting of errant color-emitting commands, this prompt does more than meets the eye extremely fast.

Don’t have much time? Don’t care? You may copy and paste these 2 lines into your shell and that’s all there is to it!

export PROMPT_COMMAND='export H1="`history 1|sed -e "s/^[\ 0-9]*//; s/[\d0\d31\d34\d39\d96\d127]*//g; s/\(.\{1,50\}\).*$/\1/g"`";history -a;echo -e "sgr0\ncnorm\nrmso"|tput -S'
export PS1='\n\e[1;30m[\j:\!\e[1;30m]\e[0;36m \T \d \e[1;30m[\e[1;34m\u@\H\e[1;30m:\e[0;37m`tty 2>/dev/null` \e[0;32m+${SHLVL}\e[1;30m] \e[1;37m\w\e[0;37m\[\033]0;[ ${H1}... ] \w - \u@\H +$SHLVL @`tty 2>/dev/null` - [ `uptime` ]\007\]\n\[\]\$ '

PROMPT_COMMAND

This holds commands to be run prior to displaying the prompt. Let’s break it down.

PS1 Prompt Example

First this takes the last command executed and filters it using sed to remove the initial history command number, and also to remove any quotes (single or double) and/or backticks. Also, it only saves the first 50 chars, which are then exported as the value of H1, which because it is exported is now globally accessible to the rest of the shell. This is important because we use this variable in PS1 to print the last run command as the title of our putty window (or konsole).

export H1=`history 1 | sed -e 's/^[\ 0-9]*//; s/[\d0\d31\d34\d39\d96\d127]*//g; s/\(.\{1,50\}\).*$/\1/g'`

The next command simply appends your last command to the history file. Usually this is ~/.bash_profile, or whatever $HISTFILE is. This makes sure you never lose history in case of a shell crash or ssh connection terminated. 

history -a

Finally there is this golden nugget of shell kung-fu. This outputs 3 terminal escape sequences according to the terminal in use and its capabilities.. so very robust, should work on any shell newer than 1984. By echoing the following 3 term names separated by a newline \n and using tput -S it will output the terminal sequences for all 3. sgr0 resets the colors, cnorm resets the cursor, and rmso resets the background color.

You can view the escape sequences generated by tput with strace, cat -t, etc.. For instance on my shell (xterm-256color via portaputty) the below command generates: \33(B\33[m\33[?12l\33[?25h\33[27m. This is helpful because if you aren't coding for several terminals you can hard code that in your echo's instead of using tput, saving some overhead.

echo -e 'sgr0\ncnorm\nrmso' | tput -S
# For my term this is the same as if I just did
# echo -e "\33(B\33[m\33[?12l\33[?25h\33[27m"

PS1 Detailed

PS1 is the variable that is expanded as your prompt. So if you did a $ unset PS1 then you can still run commands and everything, but your screen will be blank which is confusing to say the least.

The HTOP command in full color to manage mysql

This PS1 shows which SHLVL level the shell is in, which is how many levels deep of shells you are currently at. If you run $ bash -l from a bash shell it will create a new instance of bash as a child of the calling bash, so the SHLVL (shell level) gets incremented by 1. I usually avoid creating subshells by instead executing a new bash with exec which instead of creating a new bash as a child of the current bash, it replaces the current bash process with the new one: $ exec bash -l.

export PS1='\n\e[1;30m[\j:\!\e[1;30m]\e[0;36m \T \d \e[1;30m[\e[1;34m\u@\H\e[1;30m:\e[0;37m`tty 2>/dev/null` \e[0;32m+${SHLVL}\e[1;30m] \e[1;37m\w\e[0;37m\[\033]0;[ ${H1}... ] \w - \u@\H +$SHLVL @`tty 2>/dev/null` - [ `uptime` ]\007\]\n\[\]\$ '
 
# with nocolors
# export PS1='\n[\j:\!] \T \d [\u@\H:`tty 2>/dev/null` +${SHLVL}] \w\[\033]0;[ ${H1}... ] \w - \u@\H +$SHLVL @`tty 2>/dev/null` - [ `uptime` ]\007\]\n\[\]\$ '

TTY

This command is very useful for me because I use multiplexed terminals, about 6 at a time, so it is crucial sometimes to know which tty I am using. Since SSH is the ONLY protocol I ever use for anything, I used to depend on the SSH_TTY variable being set instead of using tty, which as a command has more overhead then just echoing a variable. But I found that when running multiplexing terminals with tmux, screen, etc.. the SSH_TTY variable stays with the initial tty created upon login. Note: 2>/dev/null redirects any errors that running the tty command might issue to the /dev/null device, which is the same thing as a black hole, it goes nowhere so is an efficient way to send data for deletion. Depending on your server and ssh settings you may not even have a tty or it may just appear that you do not. Some web hosting companies do odd things to prevent their users from using tty devices and programs. But if you get a little creative you can always find the current tty.. like $ readlink /proc/self/fd/0 will return the same thing as $ tty.

`tty 2>/dev/null`

Changing Window Title with PS1

This simple bash function changes the title of the window to the passed parameter. The key thing to note is that this works because of the special escapes on either end of the echo'd data.

function set_window_title()
{
  echo -e "\033]0; ${1:-$USER@$HOST - $SHLVL} \007";
}

Additional Info for Hackers

More reading and digging.

Strip Colors for Readability

You can use this command to strip a PS1 of all colors, which shortens the variable dramatically in some cases.

sed 's/\\[eE][[0-9]*;[0-9]*m//g' <<< $PS1
# ie export PS1=$(sed 's/\\[eE][[0-9]*;[0-9]*m//g' <<< $PS1);

Multi-Prompt Switcher Function

This function demonstrates 1 way that you can change your prompts at will. It creates an array named AAPS that you can add as many PS1 values to as you want. Each time the function is run it simply changes the prompt string variable PS1 to the value of the next item in the AAPS array. Once it reaches the last item in AAPS, it resets back to the first AAPS[0] prompt. Just an example to play with, not tested very much.

aa_multi_prompt ()
{
    declare -a AAPS;
    AAPS[0]='\n[\j:\!] \T \d [\u@\H:`tty 2>/dev/null` +${SHLVL}] \w\[\033]0;[ ${H1}... ] \w - \u@\H +$SHLVL @`tty 2>/dev/null` - [ `uptime` ]\007\]\n\[\]\$';
    AAPS[1]='\n[\j:\!] \T \d \n[\u@\H:`tty 2>/dev/null` +${SHLVL}] \w\[\033]0;[ ${H1}... ] \w - \u@\H +$SHLVL @`tty 2>/dev/null` - [ `uptime` ]\007\]\n\[\]\$';
 
    : ${PLVL:=0};
    [[ "${#AAPS[@]}" -lt "$PLVL" || "${#AAPS[@]}" -eq "$PLVL" ]] && PLVL=0;
    export PS1=${AAPS[$PLVL]} && (( PLVL++ )) && export PLVL
}

BASH 4 Multi-Prompt

Here's an example from bash-4, but it works for all bash versions.

prompt ()
{
    case "$1" in
        d)
            PS1='$(dirs) \$ '
        ;;
        n)
            PS1='\$ '
        ;;
        hsw)
            PS1='\h[$SHLVL]: \w \$ '
        ;;
        hw)
            PS1='\h: \w \$ '
        ;;
        sh)
            PS1='[$SHLVL] \h\$ '
        ;;
        sw)
            PS1='[$SHLVL] \w \$ '
        ;;
        uh)
            PS1='\u@\h\$ '
        ;;
        uhsHw)
            PS1='\u@\h[$SHLVL]:\#: \w \$ '
        ;;
        uhsw)
            PS1='\u@\h[$SHLVL]: \w \$ '
        ;;
        uhw)
            PS1='\u@\h: \w \$ '
        ;;
        uw)
            PS1='(\u) \w \$ '
        ;;
        w)
            PS1='\w \$ '
        ;;
    esac
}

Alternative to uptime:

echo "Up for $(( $(sed 's/^\([0-9]*\).*/\1/' /proc/uptime) / (60*60*24) )) days - [`cat /proc/loadavg`]"
# or
echo "Uptime: $(( `sed 's/^\([0-9]*\).*/\1/g' /proc/uptime` / 86400 )) days. Load: $(</proc/loadavg)"

Bash Prompts - More Reading

All Bash Prompt Variables

These are the environment variables provided by BASH (and most shells) and control your prompt string. While all are interesting and good to know about, PROMPT_COMMAND and PS1 are the only ones that directly modify the prompt that is displayed.

PROMPT_COMMAND
If set, the value is executed as a command prior to issuing each primary prompt.
PS1
The value of this parameter is expanded and used as the primary prompt string. The default value is "\s-\v\$ ".
PS2
The value of this parameter is expanded as with PS1 and used as the secondary prompt string. The default is "> ".
PS3
The value of this parameter is used as the prompt for the select command.
PS4
The value of this parameter is expanded as with PS1 and the value is printed before each command bash displays during an execution trace. The first character of PS4 is replicated multiple times, as necessary, to indicate multiple levels of indirection. The default is "+".

Prompt Escape Codes

When executing interactively, bash displays the primary prompt PS1 when it is ready to read a command and the secondary prompt PS2 when it needs more input to complete a command. Bash allows these prompt strings to be customized by inserting a number of backslash-escaped special characters that are decoded as follows:

  • \a - an ASCII bell character (07)
  • \d - the date in "Weekday Month Date" format (e.g., "Tue May 26")
  • \D{format} - the format is passed to strftime(3) and the result is inserted into the prompt string; an empty format results in a locale-specific time representation. The braces are required
  • \e - an ASCII escape character (033)
  • \h - the hostname up to the first '.'
  • \H - the hostname
  • \j - the number of jobs currently managed by the shell
  • \l - the basename of the shellâs terminal device name
  • \n - newline
  • \r - carriage return
  • \s - the name of the shell, the basename of $0 (the portion following the final slash)
  • \t - the current time in 24-hour HH:MM:SS format
  • \T - the current time in 12-hour HH:MM:SS format
  • \@ - the current time in 12-hour am/pm format
  • \A - the current time in 24-hour HH:MM format
  • \u - the username of the current user
  • \v - the version of bash (e.g., 2.00)
  • \V - the release of bash, version + patch level (e.g., 2.00.0)
  • \w - the current working directory, with $HOME abbreviated with a tilde
  • \W - the basename of the current working directory, with $HOME abbreviated with a tilde
  • \! - the history number of this command
  • \# - the command number of this command
  • \$ - if the effective UID is 0, a #, otherwise a $
  • \nnn - the character corresponding to the octal number nnn
  • \\ - a backslash
  • \[ - begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt
  • \] - end a sequence of non-printing characters

The command number and the history number are usually different: the history number of a command is its position in the history list, which may include commands restored from the history file, while the command number is the position in the sequence of commands executed during the current shell session. After the string is decoded, it is expanded via parameter expansion, command substitution, arithmetic expansion, and quote removal, subject to the value of the promptvars shell option, which if set, prompt strings undergo parameter expansion, command substitution, arithmetic expansion, and quote removal after being expanded as described in PROMPTING above. promptvars is enabled by default.

bash power prompt PS1

More Example Prompts

export PS1="\n\e[1;37m[\e[0;32m\u\e[0;35m@\e[0;32m\h\e[1;37m]\e[1;37m[\e[0;31m\w\e[1;37m]\n$ \e[0m"
[user@host][~/bin/tools]
$
export PS1="\n[$?]\e[1;37m[\e[0;32m\u\e[0;35m@\e[0;32m\h\e[1;37m]\e[1;37m[\e[0;31m\w\e[1;37m]($SHLVL:\!)\n\[\033[0m\]\$ "
[0][user@host][~/bin/tools](1:2130)
export PS1='\[\033[1;33m\]\u\[\033[1;37m\]@\[\033[1;32m\]\h\[\033[1;37m\]:\[\033[1;31m\]\w \[\033[1;36m\]\$ \[\033[0m\]'
user@host:~/bin/tools $
export PS1="\e[1;31m[\h]$NC \W > \[\033]0;\${TERM} [\u@\h] \w\]"
[lifesaver] tools >

Benchmarking PROMPT_COMMAND

When figuring out how resource intensive your prompt_command is (if you choose to do that sort of thing), it's nice to do it like this. I ended up using this information from the following test to create the prompt_command used in the first 256-color prompt above.

#!/bin/bash
 
function clean_results()
{
      grep ^real | cut -dm -f2 | sort | uniq -c | sort -n
}
 
for i in {0..500};
do
      time sh -c "expr `sed -nu 's/^MemFree:[\t ]\+\([0-9]\+\) kB/\1/Ip' /proc/meminfo`/1024" &>/dev/nulll;
done 2>&1 | clean_results
 
# print separating line
printf "%$((${COLUMNS:-`tput cols`} - 10))s\n" ' ' | sed -u 's/ /-/g'
 
for i in {0..500};
do
      time sh -c "echo $(( `sed -nu 's/^MemFree:[\t ]\+\([0-9]\+\) kB/\1/Ip' /proc/meminfo`/1024 ))" &>/dev/null;
done 2>&1 | clean_results
 
exit $?

Generates this output, meaning that using expr takes longer.

      1 0.011s
      1 0.014s
      2 0.009s
      2 0.012s
      5 0.008s
     12 0.004s
     47 0.007s
    215 0.005s
    216 0.006s
---------------------------------------------------------------------
      1 0.003s
      2 0.007s
     59 0.006s
    196 0.004s
    243 0.005s

Additional Links

Crazy POWERFUL Bash Prompt originally appeared on AskApache.com

]]> http://www.askapache.com/linux-unix/bash-power-prompt.html/feed/ 7 HOWTO: Uninstall CPANEL over SSH http://www.askapache.com/server-administration/uninstall-cpanel.html http://www.askapache.com/server-administration/uninstall-cpanel.html#comments Sat, 24 Apr 2010 18:55:40 +0000 AskApache http://www.askapache.com/?p=4404 The following is just a quick reference of some of the commands I used to uninstall cpanel. This is for advanced users of the shell. If you aren't that advanced and you run a single one of these commands without fully understanding it, you will probably kill your server, probably lose everything on it permanently, probably not have a website or email for weeks.. So backup all your data FIRST. Also, if you aren't 100% sure you won't run into problems, you should contact your hosts technical support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for hosts, and the last thing tech support wants is to fix a server broken by someone who doesn't know what they are doing.

]]> WARNING! This is for advanced users of the shell, this is NOT a howto or tutorial. The thing is, I googled how to uninstall cpanel and for once in my life I came up empty.. And certainly the cpanel official sites themselves don’t provide any instructions other than to say “Dont uninstall it, reinstall your entire operating system without it.”.. they sure don’t seem confident that they know exactly what their code is doing. At this point this is more of a series of notes than a guide. Ok now go have some fun!

Why Uninstall?

Why? Because I have always built my servers, php installations, perl installs, ruby, iptables, everything from source. I read the INSTALL/README docs, I read the man pages, and I read the info pages as well. I google for configuration advice, I google for tips, and I don’t need a web-based perl script messing my stuff up!

The main problems I had with cpanel, which really is a great bit of software for millions of website developers, is that it was incredibly sneaky! I used it for about 6 months and spent that entire time trying to figure out what the heck it was doing. A couple issues that I really disliked, it takes over your bind install, it takes over your apache install, it takes over your php install. And although it does let you configure some things (very few) for custom configurations and the like, I just don’t need any of that. By removing the darn thing I am saving GIGS of space on my server, tons of bandwidth, and most importantly to me I am saving CPU and processing time along with RAM and IO speed.

Anything Else

Please.. I could go on for DAYS! Another reason I have wanted to be rid of cpanel is that I like my servers to be as lean and mean as possible. This means I like as few files and processes as possible. There are many benefits to this, like it’s much easier for my integrity checking software and rootkit/antivirus software to run and drastically reduces the false positives. And there is that glaring security issue of constantly having cpanel run it’s own software to create the WHM/Cpanel web interface, which is accessible online. I haven’t researched cpanel security at all, it’s possible that it never has security problems that are published, but for me, why take the chance?

Warning – Caution!

The following is just a quick reference of some of the commands I used to uninstall cpanel. This is for advanced users of the shell. If you aren’t that advanced and you run a single one of these commands without fully understanding it, you will probably kill your server, probably lose everything on it permanently, probably not have a website or email for weeks.. So backup all your data FIRST. Also, if you aren’t 100% sure you won’t run into problems, you should contact your hosts technical support – but be prepared for some MAJOR negativity.. cpanel makes things very easy for hosts, and the last thing tech support wants is to fix a server broken by someone who doesn’t know what they are doing.

Last Warning!

Unless you understand what theses commands do and the purpose they serve, do not try any of this. These are not the exact commands I used verbatim, they are also not in order. I only put them up here because I was so amazed that google didn’t have any uninstall cpanel intructions. Hopefully it’s not a conspiracy that will get my site taken down.. ;)

That said and out of the way, it really only took me about 10 minutes to uninstall cpanel completely. But keep in mind I have been closely monitoring and debugging cpanel for 6 months, so I knew what I was doing. And finally, I do apologize for not having better instructions.. but hey, if you don’t get this then you have no business trying to figure out how to uninstall cpanel! It’s great software and shouldn’t be removed unless you are fully capable of managing email/dns/www/ftp and any/all other servers and services on your machine by hand.

Do This First

I have a few drafts I’m working on at the moment with specifics, but for now you will have to figure it out with google. Basically you want to make sure you don’t totally knock your machine offline without being able to reconnect. What I do is compile a static version of openssh and a few other security-type shell tools, and configure this binary sshd to run by using inittab, which is the file run by init (pid 1) and makes sure if it dies it is restarted. Here is my /etc/inittab to run the static sshd binary:

hh:12345:respawn:/failover/os/sbin/aassh -D -q -u0 -f /failover/os/etc/aassh

Another trick is to keep a detached screen logged in to root. That way if you mess up your sudoers or securetty or pam or whatever, you can just reattach and fix it.

Finally, you may want to setup your syslog to start earlier than usual, and set up more than normal verbosity. ( I take it to the max ). Then you should setup a 2nd server or machine somewhere to act as a syslog server. Lastly, configure your web server syslog to copy all messages to the remote syslog you set up. I use a reverse ssh tunnel to encrypt the syslog packets, but when I do something serious like reboot after uninstalling cpanel, I prepare for it by adding additional networking routes on my machine to make sure I will get some logs even if sshd cant start or even if my network addresses aren’t brought up correctly.

If that sounds easy to you, please continue. If you are saying: Wha??? Continue in read-only mode.

Also, you can’t just uninstall cpanel, I have replaced a lot of cpanel already, like building my own bind, apache, php, syslog and making sure they work and aren’t being tampered with by cpanel. Basically cpanel runs everything on your server in most cases, so you should prepare by creating your own static software to replace cpanel, and make sure it works.

Find files Accessing /var/cpanel

More than likely these will need to be killed.

lsof +w -Rg -nP +c15 -x f +D /var/cpanel
lsof +w -Rg -nP +c15 -x f +D /usr/local/cpanel

Killing cpanel

Just an example, your machine may have a lot more than these, I have been slowly taking control of my machine back from cpanel for 6 months, so it was easier for me.

for P in tailwatchd queueprocd cpanellogd exim; do pkill -9 $P; done

Commands and Shortcuts

alias NF='nice find $PWD -mount -depth ! -type d'
alias NFF='nice find $PWD -mount -depth ! -type d | xargs -IF87 file F87'
alias NA='nice find $PWD -mount -depth'
alias NAF='nice find $PWD -mount -depth | xargs -IF87 file F87'

Watch out for crontab

An example of the sneakiness (from my POV, from most it’s called builtin robustness) that cpanel does is automagically adding crontab entries that make it behave similarly to a self-propagating virus. If you don’t disable the cronjobs and kill the right processes within a short period of time, be prepared for a magic resurrection.

Here’s my awesome crontab information function, you will need to check every file, it lists the default crons on my box, and every users crontab, but it can’t account for other cron software like at and other crons.

function askapache_crontab()
{
  local GG i;
  for i in `getent passwd|cut -d ":" -f1`;
  do
    GG=$(sudo crontab -u $i -l 2>$N6 | tr -s '\n\000' | sed '/^#/d');
    [[ ${#GG} -gt 3 ]] && sleep 1 && echo -e "$i \n\n${GG}"
  done;
   sleep 4;
   ls -aLls1ch --color=always /etc/cron.{hourly,daily,weekly,monthly,d} | sed '/^total/d; /\ drwxr-xr-x/d';
}
grep -ir /var/spool cpan
#6 3 * * * /scripts/upcp
#0 1 * * * /scripts/cpbackup
#0 2 * * * /scripts/mailman_chown_archives
#35 * * * * /usr/bin/test -x /usr/local/cpanel/bin/tail-check && /usr/local/cpanel/bin/tail-check
#11,26,41,56 * * * * /usr/local/cpanel/whostmgr/bin/dnsqueue > /dev/null 2>&1
#30 */4 * * * /usr/bin/test -x /scripts/update_db_cache && /scripts/update_db_cache
#45 */8 * * * /usr/bin/test -x /usr/local/cpanel/bin/optimizefs && /usr/local/cpanel/bin/optimizefs
#*/5 * * * * /usr/local/cpanel/bin/dcpumon >/dev/null 2>&1
#25 1 * * * /usr/local/cpanel/whostmgr/docroot/cgi/cpaddons_report.pl --notify

Delete Crontabs

sudo crontab -u mailman -r

Find INIT scripts with cpanel

This is the main startup script: /usr/local/cpanel/etc/init/startup

I had no idea ruby-on-rails was being controlled by cpanel.. sneaky bugger. You can tell by all of these advanced unix commands just how difficult it would be to uninstall cpanel, its totally like the Alien!

(1:3744)# find . ! -type d -print0|xargs -0 -I'F87' grep -Hi "cpan\|tailwat\|chkser" F87
./fastmail:# Author:       cPanel, Inc. <nick@cpanel.net>
./httpd:        HTTPD=/usr/local/cpanel/bin/chroothttpd
./cpanel:# cpanel8       Start Cpanel Services
./cpanel:# Author:       cPanel, Inc. <nick@cpanel.net>
./cpanel:# description: This is the cpanel webserver and chat.
./cpanel:# processname: cpaneld
./cpanel:# pidfile: /var/run/cpanel.pid
./cpanel:[ -f /usr/local/cpanel/etc/init/startup ] || exit 0
./cpanel:       if [ -f "/var/cpanel/smtpgidonlytweak" ]; then
./cpanel:       echo -n "Starting cPanel services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpsrvd
./cpanel:       echo -n "Starting cPanel brute force detector services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcphulkd
./cpanel:    echo -n "Starting cPanel dav services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpdavd
./cpanel:               daemon /usr/local/cpanel/etc/init/startcppop
./cpanel:       echo -n "Starting cPanel Chat services: "
./cpanel:               daemon /usr/local/cpanel/entropychat/entropychat
./cpanel:               daemon /usr/local/cpanel/bin/startmelange
./cpanel:                       /usr/local/cpanel/bin/startinterchange
./cpanel:       echo -n "Starting cPanel ssl services: "
./cpanel:       daemon /usr/local/cpanel/startstunnel
./cpanel:    echo -n "Starting cPanel Queue services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startqueueprocd
./cpanel:    echo -n "Starting tailwatchd: "
./cpanel:    daemon /usr/local/cpanel/libexec/tailwatchd --start
./cpanel:       echo -n "Starting cPanel Log services: "
./cpanel:       daemon /usr/local/cpanel/cpanellogd
./cpanel:    action "Starting mailman services: " /usr/local/cpanel/etc/init/startmailman
./cpanel:    action "Stopping tailwatchd: " /usr/local/cpanel/libexec/tailwatchd --stop
./cpanel:    action "Stopping cPanel services: " /usr/local/cpanel/etc/init/stopcpsrvd
./cpanel:       action "Stopping cPanel dav services: " /usr/local/cpanel/etc/init/stopcpdavd
./cpanel:       action "Stopping cPanel queue services: " /usr/local/cpanel/etc/init/stopqueueprocd
./cpanel:       action "Stopping cPanel brute force detector services: " /usr/local/cpanel/etc/init/stopcphulkd
./cpanel:               action "Stopping pop3 services: " /usr/local/cpanel/etc/init/stopcppop
./cpanel:       echo -n "Stopping cPanel log services: "
./cpanel:       killproc cpanellogd
./cpanel:       echo -n "Stopping cPanel Chat services: "
./cpanel:       action "Stopping cPanel ssl services: " /usr/local/cpanel/etc/init/stopstunnel
./cpanel:       action "Stopping mailman services: " /usr/local/cpanel/etc/init/stopmailman
./cpanel:       if [ -e "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl" ]; then
./exim:if [ -e "/etc/chkserv.d" ]; then
./exim:        for file in `ls /etc/chkserv.d`
./exim:            if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./exim:    if [ -x "/usr/local/cpanel/etc/init/startspamd" ]; then
./exim:        /usr/local/cpanel/etc/init/startspamd
./exim:        if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --statusall
./securetmp:# Author:       cPanel, Inc. <copyright@cpanel.net>

Turn off cpanel services

You should remove the below delete command and start by just disabling the inits by turning them off. Then reboot. Then delete. If your machine won’t reboot, I told you so, Cpanel told you so, and likely your host told you so.

for S in cpanel ror securetmp fastmail exim; do R=$(command chkconfig --level 123456 $S off ||echo); R=$(command chkconfig --del $S ||echo); done

HTTPD

If you are running chrooted httpd then you’ll need to make sure you don’t delete your entire webserver on accident. Here’s a relevant part from the /etc/init.d/httpd script.

# the path to your httpd binary, including options if necessary
if [ -e "/etc/chroothttpd" ]; then
        HTTPD=/usr/local/cpanel/bin/chroothttpd
else
        HTTPD=/usr/local/apache/bin/httpd
fi

Finding files owned by cpanel

Some super cool bash commands in this post.. let’s start with one to find all the files and folders on your machine owned by cpanel. Check your /etc/passwd file for your machines specific usernames and groups. This command saves all the filenames to ~/cpanel-files-backup.txt, which is used by tar next to create a backup of all of them.

{ find / -mount -depth -maxdepth 150 \( -group cpanel -o   -group cpanel-phpmyadmin -o -group cpanel-phppgadmin   -o -group cpanelphpmyadmin   -o -group cpanelphppgadmin   -o -group cpanelhorde   -o -group cpanelroundcube \) -print; find / -mount -depth -maxdepth 150 \( -user cpanel -o   -user cpanel-phpmyadmin -o -user cpanel-phppgadmin   -o -user cpanelphpmyadmin   -o -user cpanelphppgadmin   -o -user cpanelhorde   -o -user cpanelroundcube \) -print; } > ~/cpanel-files-backup.txt

Here’s another way to search directories.

 grep --color=always -Hir cpanel /var

Create the Backup

Note that you must have the latest version of tar for this exact command, also you should backup /var/cpanel and /usr/local/cpanel and /etc and heck the whole machine why dontcha!

tar -T ~/cpanel-files-backup.txt -cvz --checkpoint=1000 --checkpoint-action="ttyout=\rHit %s checkpoint #%u" -f /cpanel-files-backup.tgz --totals

Remove Files

Once you do this your upstream without a paddle, you better make sure you know what you’re doing with this. This removes all those files.

cat ~/cpanel-files-backup.txt | xargs -I'F87' rm -vfr F87

Additionally you will want to remove /usr/local/cpanel and /var/cpanel – What I always do when running as root is alias my rm command to instead simply move the files to a .trash folder. That way if something goes bork you have a better chance at fixing it.

Find Group-Owned Files

find / -mount -depth -maxdepth 150 \
\(  -group cpanel -o \
  -group cpanel-phpmyadmin \
  -o -group cpanel-phppgadmin \
  -o -group cpanelphpmyadmin \
  -o -group cpanelphppgadmin \
  -o -group mailman \
  -o -group cpanelhorde \
  -o -group cpanelroundcube \
\) -fprintf /root/cpanel-group-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'

Find User-Owned Files

find / -mount -depth -maxdepth 150 \(
  -user cpanel \
  -o -user cpanel-phpmyadmin \
  -o -user cpanel-phppgadmin \
  -o -user cpanelphpmyadmin \
  -o -user cpanelphppgadmin \
  -o -user mailman \
  -o -user cpanelhorde \
  -o -user cpanelroundcube
\) -fprintf /root/cpanel-users-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'
       4  0755  drwxr-xr-x     cpanel:cpanel     32002:32004 /var/cpanel/userhomes/cpanel cpanel d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/tmp tmp d reiserfs
       4  0644  -rw-r--r-- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin cpanel-phpmyadmin d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/sessions sessions d reiserfs
       4  0644  -rw-r--r-- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin cpanelphppgadmin d reiserfs
       4  0750  drwxr-x--- cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/mail mail d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/sessions sessions d reiserfs
       4  0711  drwx--x--x cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube cpanelroundcube d reiserfs
       4  0644  -rw-r--r--     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches caches d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel .cpanel d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/tmp tmp d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/log log d reiserfs

Find Permissions

cat ~/cpanel-group-files.log ~/cpanel-users-files.log |tr -s '\000 \t'|cut -d' ' -f3|sort -u

Find files tailwatchd

(1:3732)# $NICE find ${1:-`pwd`} -mount -name '*tailwatch*'
/usr/local/cpanel/libexec/tailwatchd
/usr/local/cpanel/libexec/tailwatch
/usr/local/cpanel/libexec/tailwatch/tailwatchd
/usr/local/cpanel/etc/init/scripts/freebsd/tailwatchd.sh
/usr/local/cpanel/etc/init/scripts/trustix/tailwatchd
/usr/local/cpanel/etc/init/scripts/centos/tailwatchd
/usr/local/cpanel/etc/init/scripts/suse/tailwatchd
/usr/local/cpanel/etc/init/scripts/caos/tailwatchd
/usr/local/cpanel/etc/init/scripts/whitebox/tailwatchd
/usr/local/cpanel/etc/init/scripts/mandrake/tailwatchd
/usr/local/cpanel/etc/init/scripts/debian/tailwatchd
/usr/local/cpanel/etc/init/scripts/redhat/tailwatchd
/usr/local/cpanel/etc/init/scripts/fedora/tailwatchd
/usr/local/cpanel/etc/init/stoptailwatchd
/usr/local/cpanel/etc/init/starttailwatchd
/usr/local/cpanel/bin/tailwatchd
/usr/local/cpanel/logs/tailwatchd_log
/var/log/cpanel/tailwatchd_log
/var/cpanel/log_rotation/cp_tailwatchd_log.cpanellogd
/var/cpanel/tailwatch.positions
/var/run/tailwatchd.pid
/etc/chkserv.d/tailwatchd
/scripts/restartsrv_tailwatchd

Delete cpanel Users/Groups

for U in cpanel-phpmyadmin cpanel-phppgadmin cpanelphpmyadmin cpanelphppgadmin cpanelhorde cpanelroundcube machbuild; do userdel -fr $U; groupdel $U; done

Check for broken symlinks

find / -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'

Especially check /etc

$ find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'
/etc/ftpd-rsa.pem                   broken symbolic link to `/var/cpanel/ssl/ftp/ftpd-rsa.pem'
/etc/rc.d/rc1.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc1.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc1.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc3.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc6.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc5.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc2.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc2.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc2.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc4.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc4.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc4.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc0.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/authlib/authProg               broken symbolic link to `/usr/local/cpanel/bin/courier-auth'

And delete if you are sure

find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p' |cut -d' ' -f1|xargs -I'F87' rm -rvf 'F87'

Reinstall CSF

The only thing I actually used that came with cpanel is the CSF/LFD Firewall package, which is a fantastic piece of software. I had to reinstall this, and to get it working without cpanel add the following line to the csf.conf

GENERIC = "1"

Thats It

Now once you’ve cleaned up everything, you should try everything conceivable to get an error before rebooting. Like you should start and stop every service in /etc/init.d/, you should use telinit to check various runlevels (which keeps your sshd connection still live). Go all out, should take at least a full hour.

Another thing I like to do is rebuild alot of my source-built software again in case anything got messed up. I upgrade perl from cpanels 5.8.8 to 5.10, which is pretty thorough, and you know, reinstall anything else I think I might need. One of the benefits of compiling your own software is all I have to do is cd to the source directory and type make -B && ( { make test || make check || make checks || make tests; } || echo ) && sudo make install and that’s it. The tests/checks are optional of course.

If anyone actually ever reads this and does it, please share your advice here.. everybody knows we need it! Good Luck

HOWTO: Uninstall CPANEL over SSH originally appeared on AskApache.com

]]> http://www.askapache.com/server-administration/uninstall-cpanel.html/feed/ 0 Real-Life Htaccess Files from My Server http://www.askapache.com/htaccess/real-world-htaccess-files.html http://www.askapache.com/htaccess/real-world-htaccess-files.html#comments Sat, 17 Apr 2010 15:06:22 +0000 AskApache http://www.askapache.com/?p=4357 #### No https except to wp-admin - # If the request is empty ( implies fopen or normal file access by a php script ) RewriteCond %{THE_REQUEST} ^$ [OR]   # OR if the request if for wp-admin or wp-login.php RewriteCond %{REQUEST_URI} ^/(wp-admin|wp-login\.php).*$ [NC,OR]   # OR if the Referer is https RewriteCond %{HTTP_REFERER} ^https://www.askapache.com/.*$ [NC]   # THEN skip the following rule, basically all this does is force https or badhost to be redirected # BUT because of the above 3 rewritecond's, this won't break poorly written admin scripts RewriteRule .* - [S=1]   RewriteCond %{HTTPS} =on [OR] RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC] RewriteRule .* http://www.askapache.com%{REQUEST_URI} [R=301,L]   RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(wp-admin/.*|wp-login\.php.*)\ HTTP/ [NC] RewriteCond %{HTTPS} !=on RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]]]> I was going through some backups from years ago, and ran:

locate .htaccess | xargs -I'{}' cat '{}' >> master-htaccesser.txt

My site is named after reading source code because that is what helps me the most when I’m trying to learn something unusually difficult. Just like functions and aliases, it is very helpful to have cheatsheets for common commands.. not much is better than real-world examples. Unfortunately because this was compiled from hundreds of htaccess files on multiple hosts and platforms, and due to the concatenation, it’s not organized.

Normally I would not publish something like this, who knows how much unreleased tricks I forgot about.. but in order to say thanks to all those working for open-source, the FSF, and to all those who don’t steal content, and to all the incredible authors who shared with me (I twitter most of what I find, and follow my favs), here ya go..

ATTN: Please let me know if this is total junk or not, this is around 1/500th of my master-htaccesser.txt file.. and I’d be happy to post more if it helps..

Checking for Cookie

Used this to stop mp3-scrapers.. checks for a cookie ending in MP3P=02357

Options -Indexes
RewriteEngine On
RewriteCond %{HTTP_COOKIE} !^.*MP3P=([0-9]+).* [NC]
RewriteRule .* - [F,L]

Setting Environment Var if Proxied

RewriteEngine On
RewriteRule "\.(gif|png|jpg)$" "-" [ENV=proxied_image:1]
RewriteCond "%{ENV:proxied_image}" "!1"
RewriteRule "^" "-" [ENV=proxied_other:1]

nokeepalive for ErrorDocs and Abusers

Options SymLinksIfOwnerMatch IncludesNOEXEC
AddOutputFilter Includes html
SetEnv nokeepalive
Order Allow,Deny
Allow from all
# 1 YEAR
<FilesMatch "\.(js|css)$">
Header unset Pragma
FileETag None
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
Header unset Last-Modified
Header unset ETag
 
SetOutputFilter DEFLATE
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</FilesMatch>
SecFilterEngine Off
 
# 1 YEAR
<FilesMatch "\.(js|css)$">
Header unset Pragma
FileETag None
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
Header unset Last-Modified
Header unset ETag
 
SetOutputFilter DEFLATE
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</FilesMatch>

Unreleased Tests for AskApache Password Protection

# +ASKAPACHE PASSPRO 4.6.6
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +APRO SIDS
# +SID 21030002
Order Deny,Allow
Deny from All
Satisfy Any
AuthType Digest
AuthName "Protected By AskApache"
AuthDigestDomain / http://www.askapache.com/
AuthDigestFile /home/.greer/askapache/sites/askapache.com/.htpasswda3
Require valid-user
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$">
Allow from All
</FilesMatch>
<FilesMatch "(async-upload|admin-ajax)\.php$">
<IfModule mod_security.c>
SecFilterEngine Off
</IfModule>
Allow from All
</FilesMatch>
# -SID 21030002
# -APRO SIDS
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE PASSPRO 4.6.6
 
# +ASKAPACHE PASSPRO 4.6.6
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +APRO SIDS
# +SID Test
ErrorDocument 401 /wp-content/askapache/test.gif
ErrorDocument 403 /wp-content/askapache/test.gif
ErrorDocument 404 /wp-content/askapache/test.gif
ErrorDocument 500 /wp-content/askapache/test.gif
ServerSignature On
<IfModule mod_alias.c>
RedirectMatch 305 ^.*modaliastest$ http://www.askapache.com
</IfModule>
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{QUERY_STRING} modrewritetest [NC]
RewriteRule .* http://www.askapache.com [R=307,L]
</IfModule>
<Files modsec_check.gif>
<IfModule mod_security.c>
SetEnv MODSEC_ENABLE On
SecFilterEngine On
SecFilterDefaultAction "nolog,noauditlog,pass"
SecAuditEngine Off
SecFilterInheritance Off
SecFilter modsecuritytest "deny,nolog,noauditlog,status:503"
Deny from All
</IfModule>
</Files>
<Files basic_auth_test.gif>
AuthType Basic
AuthName "askapache test"
AuthUserFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-basic
Require valid-user
</Files>
<Files digest_check.gif>
AuthType Digest
AuthName "askapache test"
AuthDigestDomain /wp-content/askapache/ http://www.askapache.com/wp-content/askapache/
AuthUserFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-digest
Require none
</Files>
<Files authuserfile_test.gif>
AuthType Digest
AuthName "askapache test"
AuthDigestDomain /wp-content/askapache/ http://www.askapache.com/wp-content/askapache/
AuthUserFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-digest
Require valid-user
</Files>
<Files authdigestfile_test.gif>
AuthType Digest
AuthName "askapache test"
AuthDigestDomain /wp-content/askapache/ http://www.askapache.com/wp-content/askapache/
AuthDigestFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-digest
Require valid-user
</Files>
# -SID Test
# -APRO SIDS
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE PASSPRO 4.6.6

Warming up to the really advanced tests

Options +ExecCGI
Order Deny,Allow
Deny from All
Allow from 208.113.134.190 64.111.114.111 208.113.134.203 208.113.152.201 env=REDIRECT_STATUS
Satisfy Any
Options +FollowSymLinks
AddHandler application/x-httpd-php .php
 
RewriteEngine On
RewriteBase /
RewriteCond %{QUERY_STRING} !^tyy+$ [NC]
RewriteCond %{REMOTE_USER} ^(.+)$
RewriteRule ^(.*)$ $1/-%1 [R=302,L]
Options +ExecCGI +FollowSymLinks
DirectoryIndex index.php
ErrorDocument 404 /cgi-bin/pro/index.php
 
AuthType Digest
AuthName "AskApache Pro"
AuthDigestFile /home/askapache/sites/askapache.com/.htpasswd-pro
AuthDigestDomain /cgi-bin/pro/ http://www.askapache.com/cgi-bin/pro/ https://www.askapache.com/cgi-bin/pro/
Require user askapacheDirectoryIndex p.php
ErrorDocument 403 /cgi-bin/p/p.php
ErrorDocument 401 /cgi-bin/p/p.php
ErrorDocument 500 /cgi-bin/p/p.php
ErrorDocument 503 /cgi-bin/p/p.php
 
RewriteEngine On
RewriteBase /
RewriteRule .* - [E=INFO_API_VERSION:%{API_VERSION}]
RewriteRule .* - [E=INFO_AUTH_TYPE:%{AUTH_TYPE}]
RewriteRule .* - [E=INFO_CONTENT_LENGTH:%{CONTENT_LENGTH}]
RewriteRule .* - [E=INFO_CONTENT_TYPE:%{CONTENT_TYPE}]
RewriteRule .* - [E=INFO_DOCUMENT_ROOT:%{DOCUMENT_ROOT}]
RewriteRule .* - [E=INFO_GATEWAY_INTERFACE:%{GATEWAY_INTERFACE}]
RewriteRule .* - [E=INFO_HTTPS:%{HTTPS}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT:%{HTTP:Accept}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_LANGUAGE:%{HTTP:Accept-Language}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_ENCODING:%{HTTP:Accept-Encoding}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_CHARSET:%{HTTP:Accept-Charset}]
RewriteRule .* - [E=INFO_HTTP_CACHE_CONTROL:%{HTTP:Cache-Control}]
RewriteRule .* - [E=INFO_HTTP_CONNECTION:%{HTTP:Connection}]
RewriteRule .* - [E=INFO_HTTP_COOKIE:%{HTTP_COOKIE}]
RewriteRule .* - [E=INFO_HTTP_FORWARDED:%{HTTP_FORWARDED}]
RewriteRule .* - [E=INFO_HTTP_HOST:%{HTTP_HOST}]
RewriteRule .* - [E=INFO_HTTP_KEEP_ALIVE:%{HTTP_KEEP_ALIVE}]
RewriteRule .* - [E=INFO_HTTP_PROXY_CONNECTION:%{HTTP_PROXY_CONNECTION}]
RewriteRule .* - [E=INFO_HTTP_REFERER:%{HTTP:Referer}]
RewriteRule .* - [E=INFO_HTTP_USER_AGENT:%{HTTP_USER_AGENT}]
RewriteRule .* - [E=INFO_IS_SUBREQ:%{IS_SUBREQ}]
RewriteRule .* - [E=INFO_ORIG_PATH_INFO:%{ORIG_PATH_INFO}]
RewriteRule .* - [E=INFO_ORIG_PATH_TRANSLATED:%{ORIG_PATH_TRANSLATED}]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_FILENAME:%{ORIG_SCRIPT_FILENAME}]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_NAME:%{ORIG_SCRIPT_NAME}]
RewriteRule .* - [E=INFO_PATH:%{PATH}]
RewriteRule .* - [E=INFO_PATH_INFO:%{PATH_INFO}]
RewriteRule .* - [E=INFO_QUERY_STRING:%{QUERY_STRING}]
RewriteRule .* - [E=INFO_REDIRECT_QUERY_STRING:%{REDIRECT_QUERY_STRING}]
RewriteRule .* - [E=INFO_REDIRECT_REMOTE_USER:%{REDIRECT_REMOTE_USER}]
RewriteRule .* - [E=INFO_REDIRECT_STATUS:%{REDIRECT_STATUS}]
RewriteRule .* - [E=INFO_REDIRECT_URL:%{REDIRECT_URL}]
RewriteRule .* - [E=INFO_REMOTE_ADDR:%{REMOTE_ADDR}]
RewriteRule .* - [E=INFO_REMOTE_HOST:%{REMOTE_HOST}]
RewriteRule .* - [E=INFO_REMOTE_IDENT:%{REMOTE_IDENT}]
RewriteRule .* - [E=INFO_REMOTE_PORT:%{REMOTE_PORT}]
RewriteRule .* - [E=INFO_REMOTE_USER:%{REMOTE_USER}]
RewriteRule .* - [E=INFO_REQUEST_FILENAME:%{REQUEST_FILENAME}]
RewriteRule .* - [E=INFO_REQUEST_METHOD:%{REQUEST_METHOD}]
RewriteRule .* - [E=INFO_REQUEST_URI:%{REQUEST_URI}]
RewriteRule .* - [E=INFO_REDIRECT_REQUEST_URI:%{REDIRECT_REQUEST_URI}]
RewriteRule .* - [E=INFO_SCRIPT_FILENAME:%{SCRIPT_FILENAME}]
RewriteRule .* - [E=INFO_SCRIPT_GROUP:%{SCRIPT_GROUP}]
RewriteRule .* - [E=INFO_SCRIPT_NAME:%{SCRIPT_NAME}]
RewriteRule .* - [E=INFO_SCRIPT_URI:%{SCRIPT_URI}]
RewriteRule .* - [E=INFO_SCRIPT_URL:%{SCRIPT_URL}]
RewriteRule .* - [E=INFO_SCRIPT_USER:%{SCRIPT_USER}]
RewriteRule .* - [E=INFO_SERVER_ADDR:%{SERVER_ADDR}]
RewriteRule .* - [E=INFO_SERVER_ADMIN:%{SERVER_ADMIN}]
RewriteRule .* - [E=INFO_SERVER_NAME:%{SERVER_NAME}]
RewriteRule .* - [E=INFO_SERVER_PORT:%{SERVER_PORT}]
RewriteRule .* - [E=INFO_SERVER_PROTOCOL:%{SERVER_PROTOCOL}]
RewriteRule .* - [E=INFO_SERVER_SIGNATURE:%{SERVER_SIGNATURE}]
RewriteRule .* - [E=INFO_SERVER_SOFTWARE:%{SERVER_SOFTWARE}]
RewriteRule .* - [E=INFO_THE_REQUEST:%{THE_REQUEST}]
RewriteRule .* - [E=INFO_TIME:%{TIME}]
RewriteRule .* - [E=INFO_TIME_DAY:%{TIME_DAY}]
RewriteRule .* - [E=INFO_TIME_HOUR:%{TIME_HOUR}]
RewriteRule .* - [E=INFO_TIME_MIN:%{TIME_MIN}]
RewriteRule .* - [E=INFO_TIME_MON:%{TIME_MON}]
RewriteRule .* - [E=INFO_TIME_SEC:%{TIME_SEC}]
RewriteRule .* - [E=INFO_TIME_WDAY:%{TIME_WDAY}]
RewriteRule .* - [E=INFO_TIME_YEAR:%{TIME_YEAR}]
RewriteRule .* - [E=INFO_TZ:%{TZ}]
RewriteRule .* - [E=INFO_UNIQUE_ID:%{UNIQUE_ID}]
 
RequestHeader set INFO_API_VERSION "%{INFO_API_VERSION}e"
RequestHeader set INFO_AUTH_TYPE "%{INFO_AUTH_TYPE}e"
RequestHeader set INFO_CONTENT_LENGTH "%{INFO_CONTENT_LENGTH}e"
RequestHeader set INFO_CONTENT_TYPE "%{INFO_CONTENT_TYPE}e"
RequestHeader set INFO_DOCUMENT_ROOT "%{INFO_DOCUMENT_ROOT}e"
RequestHeader set INFO_GATEWAY_INTERFACE "%{INFO_GATEWAY_INTERFACE}e"
RequestHeader set INFO_HTTPS "%{INFO_HTTPS}e"
RequestHeader set INFO_HTTP_ACCEPT "%{INFO_HTTP_ACCEPT}e"
RequestHeader set INFO_HTTP_ACCEPT_LANGUAGE "%{INFO_HTTP_ACCEPT_LANGUAGE}e"
RequestHeader set INFO_HTTP_ACCEPT_ENCODING "%{INFO_HTTP_ACCEPT_ENCODING}e"
RequestHeader set INFO_HTTP_ACCEPT_CHARSET "%{INFO_HTTP_ACCEPT_CHARSET}e"
RequestHeader set INFO_HTTP_CACHE_CONTROL "%{INFO_HTTP_CACHE_CONTROL}e"
RequestHeader set INFO_HTTP_CONNECTION "%{INFO_HTTP_CONNECTION}e"
RequestHeader set INFO_HTTP_COOKIE "%{INFO_HTTP_COOKIE}e"
RequestHeader set INFO_HTTP_FORWARDED "%{INFO_HTTP_FORWARDED}e"
RequestHeader set INFO_HTTP_HOST "%{INFO_HTTP_HOST}e"
RequestHeader set INFO_HTTP_KEEP_ALIVE "%{INFO_HTTP_KEEP_ALIVE}e"
RequestHeader set INFO_HTTP_PROXY_CONNECTION "%{INFO_HTTP_PROXY_CONNECTION}e"
RequestHeader set INFO_HTTP_REFERER "%{INFO_HTTP_REFERER}e"
RequestHeader set INFO_HTTP_USER_AGENT "%{INFO_HTTP_USER_AGENT}e"
RequestHeader set INFO_IS_SUBREQ "%{INFO_IS_SUBREQ}e"
RequestHeader set INFO_ORIG_PATH_INFO "%{INFO_ORIG_PATH_INFO}e"
RequestHeader set INFO_ORIG_PATH_TRANSLATED "%{INFO_ORIG_PATH_TRANSLATED}e"
RequestHeader set INFO_ORIG_SCRIPT_FILENAME "%{INFO_ORIG_SCRIPT_FILENAME}e"
RequestHeader set INFO_ORIG_SCRIPT_NAME "%{INFO_ORIG_SCRIPT_NAME}e"
RequestHeader set INFO_PATH "%{INFO_PATH}e"
RequestHeader set INFO_PATH_INFO "%{INFO_PATH_INFO}e"
RequestHeader set INFO_QUERY_STRING "%{INFO_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_QUERY_STRING "%{INFO_REDIRECT_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_REMOTE_USER "%{INFO_REDIRECT_REMOTE_USER}e"
RequestHeader set INFO_REDIRECT_STATUS "%{INFO_REDIRECT_STATUS}e"
RequestHeader set INFO_REDIRECT_URL "%{INFO_REDIRECT_URL}e"
RequestHeader set INFO_REMOTE_ADDR "%{INFO_REMOTE_ADDR}e"
RequestHeader set INFO_REMOTE_HOST "%{INFO_REMOTE_HOST}e"
RequestHeader set INFO_REMOTE_IDENT "%{INFO_REMOTE_IDENT}e"
RequestHeader set INFO_REMOTE_PORT "%{INFO_REMOTE_PORT}e"
RequestHeader set INFO_REMOTE_USER "%{INFO_REMOTE_USER}e"
RequestHeader set INFO_REQUEST_FILENAME "%{INFO_REQUEST_FILENAME}e"
RequestHeader set INFO_REQUEST_METHOD "%{INFO_REQUEST_METHOD}e"
RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"
RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"
RequestHeader set INFO_SCRIPT_FILENAME "%{INFO_SCRIPT_FILENAME}e"
RequestHeader set INFO_SCRIPT_GROUP "%{INFO_SCRIPT_GROUP}e"
RequestHeader set INFO_SCRIPT_NAME "%{INFO_SCRIPT_NAME}e"
RequestHeader set INFO_SCRIPT_URI "%{INFO_SCRIPT_URI}e"
RequestHeader set INFO_SCRIPT_URL "%{INFO_SCRIPT_URL}e"
RequestHeader set INFO_SCRIPT_USER "%{INFO_SCRIPT_USER}e"
RequestHeader set INFO_SERVER_ADDR "%{INFO_SERVER_ADDR}e"
RequestHeader set INFO_SERVER_ADMIN "%{INFO_SERVER_ADMIN}e"
RequestHeader set INFO_SERVER_NAME "%{INFO_SERVER_NAME}e"
RequestHeader set INFO_SERVER_PORT "%{INFO_SERVER_PORT}e"
RequestHeader set INFO_SERVER_PROTOCOL "%{INFO_SERVER_PROTOCOL}e"
RequestHeader set INFO_SERVER_SIGNATURE "%{INFO_SERVER_SIGNATURE}e"
RequestHeader set INFO_SERVER_SOFTWARE "%{INFO_SERVER_SOFTWARE}e"
RequestHeader set INFO_THE_REQUEST "%{INFO_THE_REQUEST}e"
RequestHeader set INFO_TIME "%{INFO_TIME}e"
RequestHeader set INFO_TIME_DAY "%{INFO_TIME_DAY}e"
RequestHeader set INFO_TIME_HOUR "%{INFO_TIME_HOUR}e"
RequestHeader set INFO_TIME_MIN "%{INFO_TIME_MIN}e"
RequestHeader set INFO_TIME_MON "%{INFO_TIME_MON}e"
RequestHeader set INFO_TIME_SEC "%{INFO_TIME_SEC}e"
RequestHeader set INFO_TIME_WDAY "%{INFO_TIME_WDAY}e"
RequestHeader set INFO_TIME_YEAR "%{INFO_TIME_YEAR}e"
RequestHeader set INFO_TZ "%{INFO_TZ}e"
RequestHeader set INFO_UNIQUE_ID "%{INFO_UNIQUE_ID}e"
 
Options +FollowSymLinks +ExecCGI
DirectoryIndex /cgi-bin/rewrite-test/index.php
 
Header echo ^.*
 
AuthType Digest
AuthName "AskApache Pro"
AuthDigestFile /home/askapache/sites/askapache.com/.htpasswd-pro
AuthDigestDomain / http://www.askapache.com/cgi-bin/rewrite-test/ https://www.askapache.com/cgi-bin/rewrite-test/
Require user askapache
 
SetEnv MODSEC_ENABLE=On
 
SetEnvIfNoCase ^WWW-Auth "(.+)" HTTP_WWW_AUTHORIZATION=$1
SetEnvIfNoCase ^If "(.+)" HTTP_IF_MODIFIED_SINCE=$1
SetEnvIfNoCase ^If-None-Match$ "(.+)" HTTP_IF_NONE_MATCH=$1
SetEnvIfNoCase ^Cache-Control$ "(.+)" HTTP_CACHE_CONTROL=$1
SetEnvIfNoCase ^Connection$ "(.+)" HTTP_CONNECTION=$1
SetEnvIfNoCase ^Keep-Alive$ "(.+)" HTTP_KEEP_ALIVE=$1
SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
SetEnvIfNoCase ^Authorization$ ".+username=\"(.+)\".+" HTTP_REMOTE_USER=$1
SetEnvIfNoCase ^Content-Type$ "(.+)" HTTP_CONTENT_TYPE=$1
SetEnvIfNoCase ^Content-Length$ "(.+)" HTTP_CONTENT_LENGTH=$1
SetEnvIfNoCase Server_Addr "(.+)" SERVER_ADDR=$1
SetEnvIfNoCase Request_Method "(.+)" REQUEST_METHOD=$1
SetEnvIfNoCase Request_Protocol "(.+)" REQUEST_PROTOCOL=$1
SetEnvIfNoCase Request_URI "(.+)" REQUEST_URI=$1
 
ErrorDocument 100 /cgi-bin/rewrite-test/index.php?g=100
ErrorDocument 101 /cgi-bin/rewrite-test/index.php?g=101
ErrorDocument 102 /cgi-bin/rewrite-test/index.php?g=102
ErrorDocument 200 /cgi-bin/rewrite-test/index.php?g=200
ErrorDocument 201 /cgi-bin/rewrite-test/index.php?g=201
ErrorDocument 202 /cgi-bin/rewrite-test/index.php?g=202
ErrorDocument 203 /cgi-bin/rewrite-test/index.php?g=203
ErrorDocument 204 /cgi-bin/rewrite-test/index.php?g=204
ErrorDocument 205 /cgi-bin/rewrite-test/index.php?g=205
ErrorDocument 206 /cgi-bin/rewrite-test/index.php?g=206
ErrorDocument 207 /cgi-bin/rewrite-test/index.php?g=207
ErrorDocument 300 /cgi-bin/rewrite-test/index.php?g=300
ErrorDocument 301 /cgi-bin/rewrite-test/index.php?g=301
ErrorDocument 302 /cgi-bin/rewrite-test/index.php?g=302
ErrorDocument 303 /cgi-bin/rewrite-test/index.php?g=303
ErrorDocument 304 /cgi-bin/rewrite-test/index.php?g=304
ErrorDocument 305 /cgi-bin/rewrite-test/index.php?g=305
ErrorDocument 306 /cgi-bin/rewrite-test/index.php?g=306
ErrorDocument 307 /cgi-bin/rewrite-test/index.php?g=307
ErrorDocument 400 /cgi-bin/rewrite-test/index.php?g=400
ErrorDocument 401 /cgi-bin/rewrite-test/index.php?g=401
ErrorDocument 402 /cgi-bin/rewrite-test/index.php?g=402
ErrorDocument 403 /cgi-bin/rewrite-test/index.php?g=403
ErrorDocument 404 /cgi-bin/rewrite-test/index.php?g=404
ErrorDocument 405 /cgi-bin/rewrite-test/index.php?g=405
ErrorDocument 406 /cgi-bin/rewrite-test/index.php?g=406
ErrorDocument 407 /cgi-bin/rewrite-test/index.php?g=407
ErrorDocument 408 /cgi-bin/rewrite-test/index.php?g=408
ErrorDocument 409 /cgi-bin/rewrite-test/index.php?g=409
ErrorDocument 410 /cgi-bin/rewrite-test/index.php?g=410
ErrorDocument 411 /cgi-bin/rewrite-test/index.php?g=411
ErrorDocument 412 /cgi-bin/rewrite-test/index.php?g=412
ErrorDocument 413 /cgi-bin/rewrite-test/index.php?g=413
ErrorDocument 414 /cgi-bin/rewrite-test/index.php?g=414
ErrorDocument 415 /cgi-bin/rewrite-test/index.php?g=415
ErrorDocument 416 /cgi-bin/rewrite-test/index.php?g=416
ErrorDocument 417 /cgi-bin/rewrite-test/index.php?g=417
ErrorDocument 418 /cgi-bin/rewrite-test/index.php?g=418
ErrorDocument 419 /cgi-bin/rewrite-test/index.php?g=419
ErrorDocument 420 /cgi-bin/rewrite-test/index.php?g=420
ErrorDocument 421 /cgi-bin/rewrite-test/index.php?g=421
ErrorDocument 422 /cgi-bin/rewrite-test/index.php?g=422
ErrorDocument 423 /cgi-bin/rewrite-test/index.php?g=423
ErrorDocument 424 /cgi-bin/rewrite-test/index.php?g=424
ErrorDocument 425 /cgi-bin/rewrite-test/index.php?g=425
ErrorDocument 426 /cgi-bin/rewrite-test/index.php?g=426
ErrorDocument 500 /cgi-bin/rewrite-test/index.php?g=500
ErrorDocument 501 /cgi-bin/rewrite-test/index.php?g=501
ErrorDocument 502 /cgi-bin/rewrite-test/index.php?g=502
ErrorDocument 503 /cgi-bin/rewrite-test/index.php?g=503
ErrorDocument 504 /cgi-bin/rewrite-test/index.php?g=504
ErrorDocument 505 /cgi-bin/rewrite-test/index.php?g=505
ErrorDocument 506 /cgi-bin/rewrite-test/index.php?g=506
ErrorDocument 507 /cgi-bin/rewrite-test/index.php?g=507
ErrorDocument 508 /cgi-bin/rewrite-test/index.php?g=508
ErrorDocument 509 /cgi-bin/rewrite-test/index.php?g=509
ErrorDocument 510 /cgi-bin/rewrite-test/index.php?g=510
 
RewriteEngine On
RewriteBase /
 
RewriteRule .* - [E=IN_AUTH_TYPE:%{AUTH_TYPE}]
RewriteRule .* - [E=IN_CONTENT_LENGTH:%{CONTENT_LENGTH}]
RewriteRule .* - [E=IN_CONTENT_TYPE:%{CONTENT_TYPE}]
RewriteRule .* - [E=IN_DATE_GMT:%{DATE_GMT}]
RewriteRule .* - [E=IN_DATE_LOCAL:%{DATE_LOCAL}]
RewriteRule .* - [E=IN_DOCUMENT_NAME:%{DOCUMENT_NAME}]
RewriteRule .* - [E=IN_DOCUMENT_PATH_INFO:%{DOCUMENT_PATH_INFO}]
RewriteRule .* - [E=IN_DOCUMENT_ROOT:%{DOCUMENT_ROOT}]
RewriteRule .* - [E=IN_DOCUMENT_URI:%{DOCUMENT_URI}]
RewriteRule .* - [E=IN_GATEWAY_INTERFACE:%{GATEWAY_INTERFACE}]
RewriteRule .* - [E=IN_LAST_MODIFIED:%{LAST_MODIFIED}]
RewriteRule .* - [E=IN_PATH_INFO:%{PATH_INFO}]
RewriteRule .* - [E=IN_PATH_TRANSLATED:%{PATH_TRANSLATED}]
RewriteRule .* - [E=IN_QUERY_STRING:%{QUERY_STRING}]
RewriteRule .* - [E=IN_QUERY_STRING_UNESCAPED:%{QUERY_STRING_UNESCAPED}]
RewriteRule .* - [E=IN_REMOTE_ADDR:%{REMOTE_ADDR}]
RewriteRule .* - [E=IN_REMOTE_HOST:%{REMOTE_HOST}]
RewriteRule .* - [E=IN_REMOTE_IDENT:%{REMOTE_IDENT}]
RewriteRule .* - [E=IN_REMOTE_PORT:%{REMOTE_PORT}]
RewriteRule .* - [E=IN_REMOTE_USER:%{REMOTE_USER}]
RewriteRule .* - [E=IN_REDIRECT_HANDLER:%{REDIRECT_HANDLER}]
RewriteRule .* - [E=IN_REDIRECT_QUERY_STRING:%{REDIRECT_QUERY_STRING}]
RewriteRule .* - [E=IN_REDIRECT_REMOTE_USER:%{REDIRECT_REMOTE_USER}]
RewriteRule .* - [E=IN_REDIRECT_STATUS:%{REDIRECT_STATUS}]
RewriteRule .* - [E=IN_REDIRECT_URL:%{REDIRECT_URL}]
RewriteRule .* - [E=IN_REQUEST_METHOD:%{REQUEST_METHOD}]
RewriteRule .* - [E=IN_REQUEST_URI:%{REQUEST_URI}]
RewriteRule .* - [E=IN_SCRIPT_FILENAME:%{SCRIPT_FILENAME}]
RewriteRule .* - [E=IN_SCRIPT_NAME:%{SCRIPT_NAME}]
RewriteRule .* - [E=IN_SERVER_ADMIN:%{SERVER_ADMIN}]
RewriteRule .* - [E=IN_SERVER_NAME:%{SERVER_NAME}]
RewriteRule .* - [E=IN_SERVER_ADDR:%{SERVER_ADDR}]
RewriteRule .* - [E=IN_SERVER_PORT:%{SERVER_PORT}]
RewriteRule .* - [E=IN_SERVER_PROTOCOL:%{SERVER_PROTOCOL}]
RewriteRule .* - [E=IN_SERVER_SIGNATURE:%{SERVER_SIGNATURE}]
RewriteRule .* - [E=IN_SERVER_SOFTWARE:%{SERVER_SOFTWARE}]
RewriteRule .* - [E=IN_USER_NAME:%{USER_NAME}]
RewriteRule .* - [E=IN_TZ:%{TZ}]
RewriteRule .* - [E=IN_API_VERSION:%{API_VERSION}]
RewriteRule .* - [E=IN_HTTPS:%{HTTPS}]
RewriteRule .* - [E=IN_HTTP_ACCEPT:%{HTTP_ACCEPT}]
RewriteRule .* - [E=IN_HTTP_ACCEPT_CHARSET:%{HTTP_ACCEPT_CHARSET}]
RewriteRule .* - [E=IN_HTTP_ACCEPT_ENCODING:%{HTTP_ACCEPT_ENCODING}]
RewriteRule .* - [E=IN_HTTP_ACCEPT_LANGUAGE:%{HTTP_ACCEPT_LANGUAGE}]
RewriteRule .* - [E=IN_HTTP_CACHE_CONTROL:%{HTTP_CACHE_CONTROL}]
RewriteRule .* - [E=IN_HTTP_CONNECTION:%{HTTP_CONNECTION}]
RewriteRule .* - [E=IN_HTTP_COOKIE:%{HTTP_COOKIE}]
RewriteRule .* - [E=IN_HTTP_FORWARDED:%{HTTP_FORWARDED}]
RewriteRule .* - [E=IN_HTTP_HOST:%{HTTP_HOST}]
RewriteRule .* - [E=IN_HTTP_KEEP_ALIVE:%{HTTP_KEEP_ALIVE}]
RewriteRule .* - [E=IN_HTTP_PROXY_CONNECTION:%{HTTP_PROXY_CONNECTION}]
RewriteRule .* - [E=IN_HTTP_REFERER:%{HTTP_REFERER}]
RewriteRule .* - [E=IN_HTTP_USER_AGENT:%{HTTP_USER_AGENT}]
RewriteRule .* - [E=IN_IS_SUBREQ:%{IS_SUBREQ}]
RewriteRule .* - [E=IN_ORIG_PATH_INFO:%{ORIG_PATH_INFO}]
RewriteRule .* - [E=IN_ORIG_PATH_TRANSLATED:%{ORIG_PATH_TRANSLATED}]
RewriteRule .* - [E=IN_ORIG_SCRIPT_FILENAME:%{ORIG_SCRIPT_FILENAME}]
RewriteRule .* - [E=IN_ORIG_SCRIPT_NAME:%{ORIG_SCRIPT_NAME}]
RewriteRule .* - [E=IN_PATH:%{PATH}]
RewriteRule .* - [E=IN_PHP_SELF:%{PHP_SELF}]
RewriteRule .* - [E=IN_REQUEST_FILENAME:%{REQUEST_FILENAME}]
RewriteRule .* - [E=IN_REQUEST_TIME:%{REQUEST_TIME}]
RewriteRule .* - [E=IN_SCRIPT_GROUP:%{SCRIPT_GROUP}]
RewriteRule .* - [E=IN_SCRIPT_USER:%{SCRIPT_USER}]
RewriteRule .* - [E=IN_THE_REQUEST:%{THE_REQUEST}]
RewriteRule .* - [E=IN_TIME:%{TIME}]
RewriteRule .* - [E=IN_TIME_DAY:%{TIME_DAY}]
RewriteRule .* - [E=IN_TIME_HOUR:%{TIME_HOUR}]
RewriteRule .* - [E=IN_TIME_MIN:%{TIME_MIN}]
RewriteRule .* - [E=IN_TIME_MON:%{TIME_MON}]
RewriteRule .* - [E=IN_TIME_SEC:%{TIME_SEC}]
RewriteRule .* - [E=IN_TIME_WDAY:%{TIME_WDAY}]
RewriteRule .* - [E=IN_TIME_YEAR:%{TIME_YEAR}]
RewriteRule .* - [E=IN_PATH:%{PATH}]
RewriteRule .* - [E=IN_SCRIPT_URI:%{SCRIPT_URI}]
RewriteRule .* - [E=IN_SCRIPT_URL:%{SCRIPT_URL}]
RewriteRule .* - [E=IN_UNIQUE_ID:%{UNIQUE_ID}]
 
RewriteRule .* - [E=ENV_PATH:%{ENV:PATH}]
RewriteRule .* - [E=ENV_SCRIPT_URI:%{ENV:SCRIPT_URI}]
RewriteRule .* - [E=ENV_SCRIPT_URL:%{ENV:SCRIPT_URL}]
RewriteRule .* - [E=ENV_UNIQUE_ID:%{ENV:UNIQUE_ID}]
 
RequestHeader set AUTH_TYPE "%{IN_AUTH_TYPE}e"
RequestHeader set CONTENT_LENGTH "%{IN_CONTENT_LENGTH}e"
RequestHeader set CONTENT_TYPE "%{IN_CONTENT_TYPE}e"
RequestHeader set DATE_GMT "%{IN_DATE_GMT}e"
RequestHeader set DATE_LOCAL "%{IN_DATE_LOCAL}e"
RequestHeader set DOCUMENT_NAME "%{IN_DOCUMENT_NAME}e"
RequestHeader set DOCUMENT_PATH_INFO "%{IN_DOCUMENT_PATH_INFO}e"
RequestHeader set DOCUMENT_ROOT "%{IN_DOCUMENT_ROOT}e"
RequestHeader set DOCUMENT_URI "%{IN_DOCUMENT_URI}e"
RequestHeader set GATEWAY_INTERFACE "%{IN_GATEWAY_INTERFACE}e"
RequestHeader set LAST_MODIFIED "%{IN_LAST_MODIFIED}e"
RequestHeader set PATH_INFO "%{IN_PATH_INFO}e"
RequestHeader set PATH_TRANSLATED "%{IN_PATH_TRANSLATED}e"
RequestHeader set QUERY_STRING "%{IN_QUERY_STRING}e"
RequestHeader set QUERY_STRING_UNESCAPED "%{IN_QUERY_STRING_UNESCAPED}e"
RequestHeader set REMOTE_ADDR "%{IN_REMOTE_ADDR}e"
RequestHeader set REMOTE_HOST "%{IN_REMOTE_HOST}e"
RequestHeader set REMOTE_IDENT "%{IN_REMOTE_IDENT}e"
RequestHeader set REMOTE_PORT "%{IN_REMOTE_PORT}e"
RequestHeader set REMOTE_USER "%{IN_REMOTE_USER}e"
RequestHeader set REDIRECT_HANDLER "%{IN_REDIRECT_HANDLER}e"
RequestHeader set REDIRECT_QUERY_STRING "%{IN_REDIRECT_QUERY_STRING}e"
RequestHeader set REDIRECT_REMOTE_USER "%{IN_REDIRECT_REMOTE_USER}e"
RequestHeader set REDIRECT_STATUS "%{IN_REDIRECT_STATUS}e"
RequestHeader set REDIRECT_URL "%{IN_REDIRECT_URL}e"
RequestHeader set REQUEST_METHOD "%{IN_REQUEST_METHOD}e"
RequestHeader set REQUEST_URI "%{IN_REQUEST_URI}e"
RequestHeader set SCRIPT_FILENAME "%{IN_SCRIPT_FILENAME}e"
RequestHeader set SCRIPT_NAME "%{IN_SCRIPT_NAME}e"
RequestHeader set SCRIPT_URI "%{IN_SCRIPT_URI}e"
RequestHeader set SCRIPT_URL "%{IN_SCRIPT_URL}e"
RequestHeader set SERVER_ADMIN "%{IN_SERVER_ADMIN}e"
RequestHeader set SERVER_NAME "%{IN_SERVER_NAME}e"
RequestHeader set SERVER_ADDR "%{IN_SERVER_ADDR}e"
RequestHeader set SERVER_PORT "%{IN_SERVER_PORT}e"
RequestHeader set SERVER_PROTOCOL "%{IN_SERVER_PROTOCOL}e"
RequestHeader set SERVER_SIGNATURE "%{IN_SERVER_SIGNATURE}e"
RequestHeader set SERVER_SOFTWARE "%{IN_SERVER_SOFTWARE}e"
RequestHeader set UNIQUE_ID "%{IN_UNIQUE_ID}e"
RequestHeader set USER_NAME "%{IN_USER_NAME}e"
RequestHeader set TZ "%{IN_TZ}e"
RequestHeader set API_VERSION "%{IN_API_VERSION}e"
RequestHeader set HTTPS "%{IN_HTTPS}e"
RequestHeader set HTTP_ACCEPT "%{IN_HTTP_ACCEPT}e"
RequestHeader set HTTP_ACCEPT_CHARSET "%{IN_HTTP_ACCEPT_CHARSET}e"
RequestHeader set HTTP_ACCEPT_ENCODING "%{IN_HTTP_ACCEPT_ENCODING}e"
RequestHeader set HTTP_ACCEPT_LANGUAGE "%{IN_HTTP_ACCEPT_LANGUAGE}e"
RequestHeader set HTTP_CACHE_CONTROL "%{IN_HTTP_CACHE_CONTROL}e"
RequestHeader set HTTP_CONNECTION "%{IN_HTTP_CONNECTION}e"
RequestHeader set HTTP_COOKIE "%{IN_HTTP_COOKIE}e"
RequestHeader set HTTP_FORWARDED "%{IN_HTTP_FORWARDED}e"
RequestHeader set HTTP_HOST "%{IN_HTTP_HOST}e"
RequestHeader set HTTP_KEEP_ALIVE "%{IN_HTTP_KEEP_ALIVE}e"
RequestHeader set HTTP_PROXY_CONNECTION "%{IN_HTTP_PROXY_CONNECTION}e"
RequestHeader set HTTP_REFERER "%{IN_HTTP_REFERER}e"
RequestHeader set HTTP_USER_AGENT "%{IN_HTTP_USER_AGENT}e"
RequestHeader set IS_SUBREQ "%{IN_IS_SUBREQ}e"
RequestHeader set ORIG_PATH_INFO "%{IN_ORIG_PATH_INFO}e"
RequestHeader set ORIG_PATH_TRANSLATED "%{IN_ORIG_PATH_TRANSLATED}e"
RequestHeader set ORIG_SCRIPT_FILENAME "%{IN_ORIG_SCRIPT_FILENAME}e"
RequestHeader set ORIG_SCRIPT_NAME "%{IN_ORIG_SCRIPT_NAME}e"
RequestHeader set PATH "%{IN_PATH}e"
RequestHeader set PHP_SELF "%{IN_PHP_SELF}e"
RequestHeader set REQUEST_FILENAME "%{IN_REQUEST_FILENAME}e"
RequestHeader set REQUEST_TIME "%{IN_REQUEST_TIME}e"
RequestHeader set SCRIPT_GROUP "%{IN_SCRIPT_GROUP}e"
RequestHeader set SCRIPT_USER "%{IN_SCRIPT_USER}e"
RequestHeader set THE_REQUEST "%{IN_THE_REQUEST}e"
RequestHeader set TIME "%{IN_TIME}e"
RequestHeader set TIME_DAY "%{IN_TIME_DAY}e"
RequestHeader set TIME_HOUR "%{IN_TIME_HOUR}e"
RequestHeader set TIME_MIN "%{IN_TIME_MIN}e"
RequestHeader set TIME_MON "%{IN_TIME_MON}e"
RequestHeader set TIME_SEC "%{IN_TIME_SEC}e"
RequestHeader set TIME_WDAY "%{IN_TIME_WDAY}e"
RequestHeader set TIME_YEAR "%{IN_TIME_YEAR}e"
 
SetEnvIfNoCase ^WWW-Auth "(.+)" HTTP_WWW_AUTHORIZATION=$1
SetEnvIfNoCase ^If "(.+)" HTTP_IF_MODIFIED_SINCE=$1
SetEnvIfNoCase ^If-None-Match$ "(.+)" HTTP_IF_NONE_MATCH=$1
SetEnvIfNoCase ^Cache-Control$ "(.+)" HTTP_CACHE_CONTROL=$1
SetEnvIfNoCase ^Connection$ "(.+)" HTTP_CONNECTION=$1
SetEnvIfNoCase ^Keep-Alive$ "(.+)" HTTP_KEEP_ALIVE=$1
SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
SetEnvIfNoCase ^Content-Type$ "(.+)" HTTP_CONTENT_TYPE=$1
SetEnvIfNoCase ^Content-Length$ "(.+)" HTTP_CONTENT_LENGTH=$1
SetEnvIfNoCase ^Authorization$ ".+username=\"([a-zA-Z0-9]+)\".+" REMOTE_USER=$1
SetEnvIfNoCase Server_Addr "(.+)" SERVER_ADDR=$1
SetEnvIfNoCase Request_Method "(.+)" REQUEST_METHOD=$1
SetEnvIfNoCase Request_Protocol "(.+)" REQUEST_PROTOCOL=$1
SetEnvIfNoCase Request_URI "(.+)" REQUEST_URI=$1
 
RequestHeader set IF_MODIFIED_SINCE "%{HTTP_IF_MODIFIED_SINCE}e"
RequestHeader set IF_NONE_MATCH "%{HTTP_IF_NONE_MATCH}e"
RequestHeader set CACHE_CONTROL "%{HTTP_CACHE_CONTROL}e"
RequestHeader set CONNECTION "%{HTTP_CONNECTION}e"
RequestHeader set KEEP_ALIVE "%{HTTP_KEEP_ALIVE}e"
RequestHeader set AUTHORIZATION "%{HTTP_AUTHORIZATION}e"
RequestHeader set REMOTE_USER "%{REMOTE_USER}e"
RequestHeader set CONTENT_TYPE "%{HTTP_CONTENT_TYPE}e"
RequestHeader set CONTENT_LENGTH "%{HTTP_CONTENT_LENGTH}e"
RequestHeader set SERVER_ADDR "%{SERVER_ADDR}e"
RequestHeader set REQUEST_METHOD "%{REQUEST_METHOD}e"
RequestHeader set REQUEST_PROTOCOL "%{REQUEST_PROTOCOL}e"
RequestHeader set REQUEST_URI "%{REQUEST_URI}e"
 
RequestHeader set UNIQUE_ID "%{ENV_UNIQUE_ID}e"
RequestHeader set SCRIPT_URL "%{ENV_SCRIPT_URL}e"
RequestHeader set SCRIPT_URI "%{ENV_SCRIPT_URI}e"
RequestHeader set PATH "%{ENV_PATH}e"
 
Options +ExecCGI +FollowSymLinks
 
Order Deny,Allow
Deny from All
Allow from 208.113.134.190  64.111.114.111 208.113.134.203 208.113.152.201 env=REDIRECT_STATUS
Satisfy Any
 
SecFilterEngine Off

More Mod_Security (1)

#
# Order Deny,Allow
# First, all Allow directives are evaluated; at least one must match, or the request is rejected.
# Next, all Deny directives are evaluated. If any matches, the request is rejected.
# Last, any requests which do not match an Allow or a Deny directive are denied by default.
#
Order Allow,Deny
Allow from all
Deny from 217.219.
 
#Order Deny,Allow
# First, all Deny directives are evaluated; if any match, the request is denied unless it also matches an Allow directive.
# Any requests which do not match any Allow or Deny directives are permitted.
 
#SetEnvIf content-type (multipart/form-data)(.*) NEW_CONTENT_TYPE=application/x-www-form-urlencoded$2 OLD_CONTENT_TYPE=$1$2
#RequestHeader set content-type %{NEW_CONTENT_TYPE}e env=NEW_CONTENT_TYPE
SetEnvIfNoCase Content-Type "^multipart/form-data" !MODSEC_NOPOSTBUFFERING
SetEnvIfNoCase Content-Type "^application/x-www-form-urlencoded" !MODSEC_NOPOSTBUFFERING
SetEnv suppress-error-charset
SetEnvIfNoCase Content-Type "^multipart/form-data" !MODSEC_NOPOSTBUFFERING
 
SetEnvIf Request_URI "^/(cgi-bin/search\.php|cgi-bin/java\.cgi|wp-admin/.*)" MODSEC_ENABLE=Off
SetEnvIf Request_URI "^/(online-tools/js-compress.*)" "MODSEC_NOPOSTBUFFERING=Do not buffer file uploads"
SetEnvIfNoCase Remote_Addr ^208\.113\.134\.190$ MODSEC_ENABLE=Off
SetEnvIfNoCase Remote_Addr ^64\.111\.114\.111$ MODSEC_ENABLE=Off
 
### Section 1: Global Environment
#
# The directives in this section affect the overall operation of Apache,
# such as the number of concurrent requests it can handle or where it
# can find its configuration files.
#
#
# TZ: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv TZ America/Indianapolis
 
#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv SERVER_ADMIN webmaster@askapache.com
 
#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature Off
 
#
# Possible values for the Options directive are "None", "All",
# or any combination of:
#   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
#
# The Options directive is both complicated and important.  Please see
# http://httpd.apache.org/docs/2.0/mod/core.html#options
# for more information.
#
Options -Indexes -Includes -ExecCGI -MultiViews
 
#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
DirectoryIndex index.php
 
#
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#
Action php5-cgi /bin/php.cgi
 
#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
AddHandler php5-cgi .php .inc
 
#
# Commonly used filename extensions to character sets. You probably
# want to avoid clashes with the language extensions, unless you
# are good at carefully testing your setup after each change.
# See http://www.iana.org/assignments/character-sets for the
# official list of charset names and their respective RFCs.
#
AddDefaultCharset UTF-8
 
#
# AddType allows you to add to or override the MIME configuration
# file mime.types for specific file types.
#
#
AddType 'application/rdf+xml; charset=UTF-8' .rdf
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml.gz
AddType 'text/html; charset=UTF-8' .html
AddType 'text/html; charset=UTF-8' .html.gz
AddType application/octet-stream .rar .chm .bz2 .tgz .msi .pdf .exe
AddType application/vnd.ms-excel .csv
AddType application/x-httpd-php-source .phps
AddType application/x-pilot .prc .pdb
AddType application/x-shockwave-flash .swf
AddType application/xrds+xml .xrdf
AddType text/plain .ini .sh .bsh .bash .awk .nawk .gawk .csh .var .c .in .h .asc .md5 .sha .sha1
AddType video/x-flv .flv
 
#
# AddEncoding allows you to have certain browsers uncompress
# information on the fly. Note: Not all browsers support this.
# Despite the name similarity, the following Add* directives have nothing
# to do with the FancyIndexing customization directives above.
#
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
 
#
# DefaultType: the default MIME type the server will use for a document
# if it cannot otherwise determine one, such as from filename extensions.
# If your server contains mostly text or HTML documents, "text/plain" is
# a good value.  If most of your content is binary, such as applications
# or images, you may want to use "application/octet-stream" instead to
# keep browsers from trying to display binary files as though they are
# text.
#
DefaultType text/html

Error Documents

#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
#100 Continue
#101 Switching Protocols
#102 Processing
#200 OK
#201 Created
#202 Accepted
#203 Non-Authoritative Information
#204 No Content
#205 Reset Content
#206 Partial Content
#207 Multi-Status
#300 Multiple Choices
#301 Moved Permanently
#302 Found
#303 See Other
#304 Not Modified
#305 Use Proxy
#306 unused
#307 Temporary Redirect
#400 Bad Request
#401 Authorization Required
#402 Payment Required
#403 Forbidden
#404 Not Found
#405 Method Not Allowed
#406 Not Acceptable
#407 Proxy Authentication Required
#408 Request Time-out
#409 Conflict
#410 Gone
#411 Length Required
#412 Precondition Failed
#413 Request Entity Too Large
#414 Request-URI Too Large
#415 Unsupported Media Type
#416 Requested Range Not Satisfiable
#417 Expectation Failed
#418 unused
#419 unused
#420 unused
#421 unused
#422 Unprocessable Entity
#423 Locked
#424 Failed Dependency
#425 No code
#426 Upgrade Required
#500 Internal Server Error
#501 Method Not Implemented
#502 Bad Gateway
#503 Service Temporarily Unavailable
#504 Gateway Time-out
#505 HTTP Version Not Supported
#506 Variant Also Negotiates
#507 Insufficient Storage
#508 unused
#509 unused
#510 Not Extended
 
#ErrorDocument 100 /e/100_CONTINUE.html
#ErrorDocument 101 /e/101_SWITCHING_PROTOCOLS.html
#ErrorDocument 102 /e/102_PROCESSING.html
 
#ErrorDocument 200 /e/200_OK.html
#ErrorDocument 201 /e/201_CREATED.html
#ErrorDocument 202 /e/202_ACCEPTED.html
#ErrorDocument 203 /e/203_NON_AUTHORITATIVE.html
#ErrorDocument 204 /e/204_NO_CONTENT.html
#ErrorDocument 205 /e/205_RESET_CONTENT.html
#ErrorDocument 206 /e/206_PARTIAL_CONTENT.html
#ErrorDocument 207 /e/207_MULTI_STATUS.html
 
#ErrorDocument 300 /e/300_MULTIPLE_CHOICES.html
#ErrorDocument 301 /e/301_MOVED_PERMANENTLY.html
#ErrorDocument 302 /e/302_MOVED_TEMPORARILY.html
#ErrorDocument 303 /e/303_SEE_OTHER.html
#ErrorDocument 304 /e/304_NOT_MODIFIED.html
#ErrorDocument 305 /e/305_USE_PROXY.html
#ErrorDocument 307 /e/307_TEMPORARY_REDIRECT.html
 
ErrorDocument 400 /e/400_BAD_REQUEST.html
ErrorDocument 401 /e/401_UNAUTHORIZED.html
ErrorDocument 402 /e/402_PAYMENT_REQUIRED.html
ErrorDocument 403 /e/403_FORBIDDEN.html
#ErrorDocument 404 /e/404_NOT_FOUND.html
ErrorDocument 404 /index.php?error=404
ErrorDocument 405 /e/405_METHOD_NOT_ALLOWED.html
ErrorDocument 406 /e/406_NOT_ACCEPTABLE.html
ErrorDocument 407 /e/407_PROXY_AUTHENTICATION_REQUIRED.html
ErrorDocument 408 /e/408_REQUEST_TIME_OUT.html
ErrorDocument 409 /e/409_CONFLICT.html
ErrorDocument 410 /e/410_GONE.html
ErrorDocument 411 /e/411_LENGTH_REQUIRED.html
ErrorDocument 412 /e/412_PRECONDITION_FAILED.html
ErrorDocument 413 /e/413_REQUEST_ENTITY_TOO_LARGE.html
ErrorDocument 414 /e/414_REQUEST_URI_TOO_LARGE.html
ErrorDocument 415 /e/415_UNSUPPORTED_MEDIA_TYPE.html
ErrorDocument 416 /e/416_RANGE_NOT_SATISFIABLE.html
ErrorDocument 417 /e/417_EXPECTATION_FAILED.html
ErrorDocument 422 /e/422_UNPROCESSABLE_ENTITY.html
ErrorDocument 423 /e/423_LOCKED.html
ErrorDocument 424 /e/424_FAILED_DEPENDENCY.html
ErrorDocument 426 /e/426_UPGRADE_REQUIRED.html
ErrorDocument 500 /e/500_INTERNAL_SERVER_ERROR.html
ErrorDocument 501 /e/501_NOT_IMPLEMENTED.html
ErrorDocument 502 /e/502_BAD_GATEWAY.html
ErrorDocument 503 /e/503_SERVICE_UNAVAILABLE.html
ErrorDocument 504 /e/504_GATEWAY_TIME_OUT.html
ErrorDocument 505 /e/505_VERSION_NOT_SUPPORTED.html
ErrorDocument 506 /e/506_VARIANT_ALSO_VARIES.html
ErrorDocument 507 /e/507_INSUFFICIENT_STORAGE.html
ErrorDocument 510 /e/510_NOT_EXTENDED.html

Caching

h2>

#
#  HEADERS and CACHING
#
Header unset Pragma
FileETag None
Header unset ETag
 
# 1 YEAR
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
Header unset P3P
Header unset Pragma
FileETag None
Header unset ETag
Header set Cache-Control "public,max-age=29030400"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
</FilesMatch>
 
# 2 HOURS
<FilesMatch "\.(html|htm|xml|txt|xsl|rdf|rss)$">
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault A3600
</IfModule>
</FilesMatch>

Redirect Hack

#Redirect 400 /e/400
#Redirect 401 /e/401
#Redirect 402 /e/402
#Redirect 403 /e/403
Redirect 404 /index.php?error=404
#Redirect 405 /e/405
#Redirect 406 /e/406
#Redirect 407 /e/407
#Redirect 408 /e/408
#Redirect 409 /e/409
#Redirect 410 /e/410
#Redirect 411 /e/411
#Redirect 412 /e/412
#Redirect 413 /e/413
#Redirect 414 /e/414
#Redirect 415 /e/415
#Redirect 416 /e/416
#Redirect 417 /e/417
#Redirect 418 /e/418
#Redirect 419 /e/419
#Redirect 420 /e/420
#Redirect 421 /e/421
#Redirect 422 /e/422
#Redirect 423 /e/423
#Redirect 424 /e/424
#Redirect 425 /e/425
#Redirect 426 /e/426
#Redirect 500 /e/500
#Redirect 501 /e/501
#Redirect 502 /e/502
#Redirect 503 /e/503
#Redirect 504 /e/504
#Redirect 505 /e/505
#Redirect 506 /e/506
#Redirect 507 /e/507
#Redirect 508 /e/508
#Redirect 509 /e/509
#Redirect 510 /e/510

301 PERMANENT REDIRECTS

#
# Redirect allows you to tell clients about documents which used to exist in
# your server's namespace, but do not anymore. This allows you to tell the
# clients where to look for the relocated document.
#
Redirect 301 /12-lessons-for-those-afraid-of-css.html http://www.askapache.com/css/12-lessons-for-those-afraid-of-css.html
Redirect 301 /2006/htaccess/htaccesselite-ultimate-htaccess-article.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /2007/phpbb/sending-post-form-data-with-php-curl.html http://www.askapache.com/htaccess/sending-post-form-data-with-php-curl.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/php-and-ajax-shell-console.html http://www.askapache.com/tools/php-and-ajax-shell-console.html
Redirect 301 /27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html http://www.askapache.com/htaccess/27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html
Redirect 301 /404-google-wordpress-plugin.html http://www.askapache.com/seo/404-google-wordpress-plugin.html
Redirect 301 /503-service-temporarily-unavailable.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /Overview-about.rdf http://www.askapache.com/askapache-home.rdf
Redirect 301 /abbr-acronym.html http://www.askapache.com/xhtml/abbr-acronym.html
Redirect 301 /adsense-robots.html http://www.askapache.com/google/adsense-robots.html
Redirect 301 /alexa-toolbar-firefox.html http://www.askapache.com/tools/alexa-toolbar-firefox.html
Redirect 301 /allowing-access-from-1-static-ip-and-deny-the-rest.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /anti-virus-spyware-rootkit.html http://www.askapache.com/security/anti-virus-spyware-rootkit.html
Redirect 301 /apache-ssl-in-htaccess-examples.html http://www.askapache.com/htaccess/apache-ssl-in-htaccess-examples.html
Redirect 301 /awk-tutorial.html http://www.askapache.com/awk/awk-tutorial.html
Redirect 301 /best-adsense-optimization.html http://www.askapache.com/wordpress/best-adsense-optimization.html
Redirect 301 /commonly-used-htaccess-code-examples.html http://www.askapache.com/htaccess/commonly-used-htaccess-code-examples.html
Redirect 301 /css-background-image-sprite.html http://www.askapache.com/css/css-background-image-sprite.html
Redirect 301 /css-browser-screenshots.html http://www.askapache.com/tools/css-browser-screenshots.html
Redirect 301 /css-class-example.html http://www.askapache.com/css/css-class-example.html
Redirect 301 /curl-multi-downloads.html http://www.askapache.com/php/curl-multi-downloads.html
Redirect 301 /custom-boot-menu-in-windows-xp.html http://www.askapache.com/windows/custom-boot-menu-in-windows-xp.html
Redirect 301 /donate http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /donate/ http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /htaccess.txt http://z.askapache.com/p/htaccess.txt
Redirect 301 /htaccess/404-errorpages.html http://www.askapache.com/seo/google-ajax-search-seo-tips.html
Redirect 301 /htaccess/feedsmith http://www.askapache.com/htaccess/feedsmith-htaccess.html
Redirect 301 /htaccess/http-status-codes.html http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html
Redirect 301 /htaccess/instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /htaccess/speed-up-the-apache-web-server-with-configuration-hacks.html http://www.askapache.com/htaccess/apache-web-server-speed-configuration-hacks.html
Redirect 301 /instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /security/bypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/bypassing-vlanbypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/rigging-the-dreamhost-site-of-the-month-contest.html http://www.askapache.com/dreamhost/rigging-the-dreamhost-site-of-the-month-contest.html
Redirect 301 /seo/tailrankcom-robot.html http://www.askapache.com/seo/tailrank-robot.html
Redirect 301 /webmaster/caching-tutorial-for-webmasters.html http://www.askapache.com/htaccess/caching-tutorial-for-webmasters.html
Redirect 301 /webmaster/lft-traceroute-tool.html http://www.askapache.com/tools/lft-traceroute-tool.html

301 PERMANENT REDIRECTMATCH

#
#  PERMANENT REDIRECTMATCH
#
#RedirectMatch 301 ^/&(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/(.+)\.html/$ http://www.askapache.com/$1.html
RedirectMatch 301 ^/&amp(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/.*feed\.gif$ http://z.askapache.com/feed.gif
RedirectMatch 301 ^/([^/]+)//$ http://www.askapache.com/$1/
RedirectMatch 301 ^/(.+)/htaccesselite-ultimate-htaccess-article.html(.*) http://www.askapache.com/htaccess/apache-htaccess.html
RedirectMatch 301 ^/(.+)\.html/([a-z][a-z])/$ http://www.askapache.com/$1.html
RedirectMatch 301 ^/([\(]+)(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/([^9]*)9O1X.3y(.*)/(.*)$ http://www.askapache.com/$2
RedirectMatch 301 ^/.3y(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/200([0-9])/([0-9])(.*)$ http://www.askapache.com/top-100/
RedirectMatch 301 ^/200([0-9])/([^01])(.*)$ http://www.askapache.com/$2$3
RedirectMatch 301 ^/about/glossary(.*)$ http://www.askapache.com/glossary$1
RedirectMatch 301 ^/apache-speed(.*)$ http://www.askapache.com/htaccess/apache-speed$1
RedirectMatch 301 ^/category/(.+)$ http://www.askapache.com/$1
RedirectMatch 301 ^/docs/(.*)$ http://askapache.info/$1
RedirectMatch 301 ^/htaccess/feedsmith-htaccess(.*) http://www.askapache.com/htaccess/redirecting-wordpress-feeds-to-feedburner.html
RedirectMatch 301 ^/robots-txt(.*)$ http://www.askapache.com/robots.txt
RedirectMatch 301 ^/hosting/?$ http://www.askapache.com/web-hosting/
RedirectMatch 301 ^/.+favicon.ico$ http://www.askapache.com/favicon.ico
#RedirectMatch 301 ^/wp-content/uploads/(.*)$ http://z.askapache.com/uploads/$1
#RedirectMatch 301 ^/z/(.+)$ http://z.askapache.com/$1
#RedirectMatch 301 ^/(z|t|i|j|c|p)/(.*)$ http://z.askapache.com/$1/$2
 
#
#  TEMPORARY REDIRECTMATCH
#
RedirectMatch 307 ^/getflash/?$ http://www.adobe.com/shockwave/download/download.cgi?P1_Prod_Version=ShockwaveFlash
RedirectMatch 307 ^/dream/?$ http://www.askapache.com/dreamhost/
RedirectMatch 307 ^/(cse|apachecse|apachecsetest|apachesearch)/?$ http://www.google.com/coop/cse?cx=002660089121042511758%3Akk7rwc2gx0i
RedirectMatch 307 ^/search/lr-lang(.*)$ http://feeds.askapache.com/apache/htaccess
 
#
#  PERMANENT GONE
#
RedirectMatch 410 ^/funny(.*)

My Favorite modsec_v1 stuff

# Pass: Allows request to continue, further filters could still halt request.
# Allow: Allows matching requests through, will not be tested against other filters.
# Deny: Stops the request outright, returns a HTTP 500 error code by default.
# Status: Used to specify an alternate HTTP error code.
# Redirect: Matching requests are redirected to the provided URL.
# Exec: Allows execution of a local system binary or script.
# Log: Logs request only.
# Nolog: Does not log request.
# Chain: Allows you to create list of filters for more granulated security. All filters must be cleared before action is taken with the final filter.
 
SecFilterEngine On
SecFilterCheckURLEncoding On
SecFilterCheckUnicodeEncoding Off
SecFilterScanPOST On
 
#SecUploadKeepFiles On
#SecUploadDir /home/askapache/sites/askapache.com/logs/sec-upload
#SecUploadApproveScript /home/askapache/sites/askapache.com/htdocs/cgi-bin/check.sh
 
#SecRuleEngine On
#SecAuditEngine On
#SecAuditEngine Off
SecAuditEngine RelevantOnly
SecAuditLog /home/askapache/sites/askapache.com/logs/modsec_audit.log
SecFilterDebugLog /home/askapache/sites/askapache.com/logs/modsec_debug.log
SecFilterDebugLevel 1
#SecAuditLogRelevantStatus "^(?:1|2(?!00)|5|4(?!04))"
SecAuditLogRelevantStatus "^(1|2(?!00)|4([0-9](0|1|2|5|6|7|8|9))|5)"
SecFilterForceByteRange 1 255
 
#SecFilterInheritance Off
# 0 EMERGENCY - system is unusable
# 1 ALERT - action must be taken immediately
# 2 CRITICAL - critical conditions
# 3 ERROR - error conditions
# 4 WARNING - warning conditions
# 5 NOTICE - normal but significant conditions
# 6 INFO - informational
# 7 DEBUG - debug-level messages
 
SecFilterDefaultAction "deny,severity:6,status:403"
 
#SecFilterSelective REQUEST_METHOD "POST" "pass,auditlog,severity:6"
SecFilterSelective REQUEST_URI "^/(xmlrpc|wp-comments-post)\.php" "pass,log,auditlog,severity:6"
#SecFilterSelective REMOTE_ADDR ^$ "pass,log,auditlog,severity:6"
#SecFilterSelective REMOTE_ADDR ^203\.221\.91\.20$ "pass,log,auditlog,severity:6"
 
#SecFilterSelective REQUEST_URI "^/htaccess.*" "pass,log,auditlog"
#SecFilterSelective REQUEST_URI "^/feed.*" "pass,log,auditlog"
#SecFilterSelective REQUEST_URI "^/.*trackback.*" "pass,log,auditlog"
#SecFilterSelective REQUEST_URI "^/valid.*" "pass,log,auditlog"
 
#Enforce proper HTTP requests
SecFilterSelective SERVER_PROTOCOL "!^HTTP/(0\.9|1\.0|1\.1)$" "id:340000,rev:1,severity:6,msg:'Bad HTTP Protocol'"
 
# Only accept request encodings we know how to handle
SecFilterSelective REQUEST_METHOD "!^(GET|HEAD|POST)$" "chain,id:340001,rev:1,severity:6,msg:'Restricted HTTP function,status:405'"
SecFilterSelective HTTP_Content-Type "!(^$|^application/x-www-form-urlencoded$|^multipart/form-data)"
 
# Require Content-Length to be provided with every POST request
SecFilterSelective REQUEST_METHOD "^POST$" "chain,id:340003,rev:1,severity:6,msg:'Content Length not provided with POST',status:411"
SecFilterSelective HTTP_Content-Length "^$"
 
# Don't accept transfer encodings we know we don't handle
# (and you don't need it anyway)
SecFilterSelective HTTP_Transfer-Encoding "!^$" "id:340004,rev:1,severity:6,msg:'Dis-allowed Transfer Encoding'"
 
#Generic rule for allowed characters, adjust for your site before activating
##SecFilterSelective REQUEST_URI "!^[a-zA-Z0-9\.\+\_\/\-\?\=\&\%\#]+$" "chain,id:390002,rev:1,severity:4,msg:'Restricted HTTP character set'"
##SecFilterSelective REQUEST_URI "!^/(openid|wp-admin|wp-includes|wp-content|wp-login.php)"
 
#HTTP response splitting generic sigs
#SecFilter "Content-Length\:.*Content-Type\:.*Content-Type\:" "id:340005,rev:1,severity:4,msg:'HTTP response splitting'"
 
#HTTP response splitting generic sigs
#SecFilter "Content-Length\:" "chain,id:340006,rev:1,severity:4,msg:'HTTP response splitting'"
#SecFilter "Content-Type\:"
 
#catch smuggling attacks
#SecFilter "^(GET|POST).*Host:.*^(GET|POST)"  "id:300012,rev:1,severity:4,msg:'catch smuggling attacks'"
 
#XSS insertion into Content-Type
#SecFilterSelective THE_REQUEST "Content-Type\:.*(<[[:space:]]*(script|about|applet|activex|chrome)*>.*(script|about|applet|activex|chrome)[[:space:]]*>|onmouseover=|javascript\:)" "id:300002,rev:1,severity:4,msg:'XSS attack in Content-type header'"
 
#Code injection via content length
#SecFilterSelective HTTP_Content-Length|HTTP_USER_AGENT "\;(system|passthru|exec)\(" "id:330003,rev:1,severity:4,msg:'Code Injection in Content-Length header'"
 
#Don't accept chunked encodings modsecurity can not look at these, so this is a hole that can bypass your rules, the rule before this one should cover this, but hey paranoia is cheap
#SecFilterSelective HTTP_Transfer-Encoding "chunked" "id:300003,rev:1,severity:4,msg:'Chunked Transfer Encoding denied'"
 
##generic recursion signatures
#SecFilterSelective REQUEST_URI "\.\./\.\./" "id:300004,rev:2,severity:4,msg:'Generic Path Recursion1 denied'"
#SecFilterSelective THE_REQUEST "\.\|\./\.\|\./\.\|" "id:300005,rev:1,severity:4,msg:'Generic Path Recursion2 denied'"
#SecFilterSelective THE_REQUEST "\.\.\./" "id:300006,rev:1,severity:4,msg:'Bogus Path denied'"
 
#Generic PHP exploit signatures
#SecFilterSelective REQUEST_URI "<\?php (chr|fwrite|fopen|system|echr|passthru|popen|proc_open|shell_exec|exec|proc_nice|proc_terminate|proc_get_status|proc_close|pfsockopen|leak|apache_child_terminate|posix_kill|posix_mkfifo|posix_setpgid|posix_setsid|posix_setuid|phpinfo)\(.*\)\;" "id:330002,rev:1,severity:4,msg:'Generic PHP exploit pattern denied'"
 
#Prevent SQL injection in cookies
##SecFilterSelective COOKIE_VALUES "((select|grant|delete|insert|drop|alter|replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]+[[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]|UNION SELECT.*\'.*\'.*,[0-9].*INTO.*FROM)" "id:300011,rev:1,severity:4,msg:'Generic SQL injection in cookie'"
 
#Prevent SQL injection in UA
##SecFilterSelective HTTP_USER_AGENT "((select|grant|delete|insert|drop|alter|replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]+[[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]|UNION SELECT.*\'.*\'.*,[0-9].*INTO.*FROM)" "id:300012,rev:1,severity:4,msg:'Generic SQL injection in User Agent header'"
 
# Generic filter to prevent SQL injection attacks
# Understand that all SQL filters are very limited and are very difficult to prevent false postives and negatives.
# Please report false positives/negatives to mike@gotroot.com
#SecFilter "((select|grant|delete|insert|drop|alter|replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]+[[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]|UNION SELECT.*\'.*\'.*,[0-9].*INTO.*FROM)" "id:300013,rev:1,severity:4,msg:'Generic SQL injection protection'"
 
#generic XSS PHP attack types
##SecFilterSelective REQUEST_URI "\.php\?" "chain,id:300010,rev:1,severity:4,msg:'Generic PHP XSS exploit pattern denied'"
#SecFilter "(javascript\:/(.*new\x20ActiveXObject.*Sh\.regwrite|.*window\.opener\.document\.body.\innerHTML=window\.opener\.document\.body\.innerHTML\.replace)|onmouseover=\'javascript)"
 
#Generic XSS filter
#please report false positives
##SecFilterSelective REQUEST_URI "!/mt\.cgi" "chain,msg:'XSS2'"
#SecFilter "<[[:space:]]*(script|about|applet|activex|chrome)*>.*(script|about|applet|activex|chrome)[[:space:]]*>"
 
#XSS in referrer and UA headers
##SecFilterSelective HTTP_REFERER|HTTP_USER_AGENT "<[[:space:]]*(script|about|applet|activex|chrome)*>.*(script|about|applet|activex|chrome)[[:space:]]*>" "msg:'XSS3'"
 
#HTTP header PHP code injection attacks
##SecFilterSelective HTTP_CLIENT_IP|HTTP_USER_AGENT|HTTP_Referer "(<\?php|<[[:space:]]?\?[[:space:]]?php|<\? php)" "msg:'PHP1'"
 
#Generic PHP remote file injection
##SecFilterSelective REQUEST_URI "!(/do_command)" "chain,msg:'PHP2'"
##SecFilterSelective REQUEST_URI "\.php\?.*=(https?|ftp)\:/.*(cmd|command)="
 
#script, perl, etc. code in HTTP_Referer string
##SecFilterSelective HTTP_Referer "\#\!.*/" "msg:'perl script1'"
 
#remote file inclusion generic attack signature
#SecFilterSelective THE_REQUEST  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)\?" "chain,msg:'remote 1'"
#SecFilter "((name|pm_path|pagina|path|include_location|root|page|open)=(http|https|ftp)|(cmd|command|inc)=)"
 
#remote file inclusion generic attack signature
#SecFilterSelective THE_REQUEST  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)\?\&(cmd|command|inc|name)=" "msg:'remote2'"
 
#remote file inclusion generic attack signature
#SecFilterSelective ARGS  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)" "chain,msg:'file inclusion1'"
#SecFilter "\?\&(cmd|inc|name)="
 
#remote file inclusion generic attack signature
#SecFilterSelective ARGS  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)\?\&(cmd|inc|name)=" "msg:'file inclusion2'"
 
#remote file inclusion generic attack signature
#SecFilterSelective REQUEST_URI  "\.php\?.*=(http|https|ftp)\:/.*\?&cmd=" "msg:'file inclusion3'"
 
#Bogus file extensions generic signature
#SecFilterSelective THE_REQUEST  "[A-Za-z0-9]\.(gif|jpg|png|bmp)\.txt" "msg:'file extension'"
 
#PHP remote path attach generic signature
#SecFilterSelective REQUEST_URI  "\.ph(p(3|4)?).*path=(http|https|ftp)\:/" "msg:'remote path1'"
#SecFilterSelective REQUEST_URI  "\.php.*path=(http|https|ftp)\:/" "msg:'remote path2'"
 
#generic php attack sigs
#SecFilterSelective REQUEST_URI "(&(cmd|command)=(id|uname)\x20|cmd\?(cmd|command)=|(spy|cmd|cmd_out|sh)\.(gif|jpg|png|bmp|txt)\?&(cmd|command)=|\.php\?&(cmd|command)=)" "msg:'php attack1'"
 
# WEB-MISC apache directory disclosure attempt
#SecFilterSelective THE_REQUEST "////" "msg:'apache directory disclosure'"
 
#PHP defenses
##SecFilterSelective ARG_PHPSESSID "!^$" "msg:'something in phpsessid'"
##SecFilterSelective COOKIE_PHPSESSID "!^$" "msg:'something in cookie phpsessid'"
 
#PHP defenses
##SecFilterSelective COOKIE_ASKAPACHEID "!^[0-9a-z]*$" "msg:'bad value for cookie'"
 
#cross site scripting attempt TYPE + JAVASCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/javascript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + JAVASCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]application\/x-javascript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + JSCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/jscript" "msg:'cross-site1'"
 
# cross site scripting attempt STYLE + VBSCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/vbscript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + VBSCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]application\/x-vbscript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + ECMACRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/ecmascript" "msg:'cross-site1'"
 
# cross site scripting attempt STYLE + EXPRESSION
##SecFilterSelective THE_REQUEST "STYLE[\s]*=[\s]*[^>]expression[\s]*\(" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + EXPRESSION
##SecFilterSelective THE_REQUEST "[\s]*expression[\s]*\([^}]}[\s]*<\/STYLE>" "msg:'cross-site1'"
 
# cross site scripting attempt using XML
##SecFilterSelective THE_REQUEST "<!\[CDATA\[<\]\]>SCRIPT" "msg:'cross-site1'"
 
#cross site scripting attempt executing hidden Javascript
##SecFilterSelective THE_REQUEST "eval[\s]*\([\s]*[^\.]\.innerHTML[\s]*\)" "msg:'cross-site1'"
 
#cross site scripting attempt executing hidden Javascript
##SecFilterSelective THE_REQUEST "window\.execScript[\s]*\(" "msg:'cross-site1'"
 
#cross site scripting attempt to execute Javascript code
###SecFilterSelective THE_REQUEST "/(((URL|SRC|HREF|LOWSRC)[\s]*=)|(url[\s]*[\(]))[\s]*[\'\"]*javascript[\:]" "msg:'cross-site1'"
 
#cross site scripting HTML Image tag set to javascript attempt
#SecFilterSelective THE_REQUEST "img src=javascript" "msg:'cross-site1'"
 
#Fake image file shell attacvk
#SecFilterSelective HTTP_Content-Type "image/.*" "msg:'image shell 1'"
#SecFilterSelective POST_PAYLOAD "chr\(" "msg:'image shell2'"
 
#bogus graphics file
#SecFilterSelective HTTP_Content-Disposition "\.php"  "chain,msg:'bogus graphics'"
#SecFilterSelective HTTP_Content-Type "(image/gif|image/jpg|image/png|image/bmp)"
 
# Allow only letters, digits, underscore, and square brackets (for arrays)
# in variable names#
#SecFilterSelective ARGS_NAMES "!^[][a-zA-Z0-9_]+$"
 
#---------------------------------------------
# reject keywords that appear in POST or GET
#=============================================
SecFilterSignatureAction "nolog,auditlog,deny,severity:6,status:403"
 
SecFilterSelective REQUEST_URI  "^/.*/wp-comments-post\.php" "id:50200,msg:'WORDPRESS SPAM bad wp-comments-post location'"
 
<Files wp-comments-post.php>
# fail for empty comment fields
SecFilterSelective "ARG_comment_post_ID|ARG_submit" "^$" "id:50300,msg:'WORDPRESS SPAM MISSING comment_post_ID'"
SecFilterSelective "ARG_comment_post_ID" "!^[0-9]{1,6}$" "id:50301,msg:'WORDPRESS SPAM BAD comment_post_ID'"
SecFilterSelective "HTTP_Cookie" "^$" "id:50302,msg:'WORDPRESS SPAM No cookie'"
 
##SecFilterSelective "comment_post_DI" "^$" "id:50310,msg:'WORDPRESS SPAM MISSING comment_post_DI'"
##SecFilterSelective "comment_post_DI" "!^[0-9]{1,2}$" "id:50311,msg:'WORDPRESS SPAM MISSING comment_post_DI'"
</Files>
<pre>
 
<h2>MODSEC Continued.. Custom Anti-Spam (WordPress) I made</h2>
<pre>
SecFilterSignatureAction "nolog,noauditlog,deny,severity:6,redirect:http://www.askapache.com/feed/"
SecFilterSelective ARGS|THE_REQUEST "00bp\.com|360\.yahoo|987mb\.com|Ambien|American airline" "id:50010,msg:'SPAM 10'
SecFilterSelective ARGS|THE_REQUEST "Ativan|Caresoprodol|Darvocet|Ephedra|Ephedrine" "id:50011,msg:'SPAM 11'
SecFilterSelective ARGS|THE_REQUEST "Gambling|Lexapro|Tramadol|Venlafaxine" "id:50012,msg:'SPAM 12'
SecFilterSelective ARGS|THE_REQUEST "\[URL=|abgood|acura|acyclovir|adderall" "id:50013,msg:'SPAM 13'
SecFilterSelective ARGS|THE_REQUEST "adipex|alcohol|alprazolam|amateur|amrit" "id:50014,msg:'SPAM 14'
SecFilterSelective ARGS|THE_REQUEST "anal sex|analfinder|angelina jolie|asshole|axspace\.com" "id:50015,msg:'SPAM 15'
SecFilterSelective ARGS|THE_REQUEST "baccarat|bankrupt|bikini|biotic|black jack|blowjob" "id:50016,msg:'SPAM 16'
SecFilterSelective ARGS|THE_REQUEST "blackjack|blog\.360|brutality|buddhism|butalbital" "id:50017,msg:'SPAM 17'
SecFilterSelective ARGS|THE_REQUEST "cadillac|canalis|card credit|card stud|carisoprodol" "id:50018,msg:'SPAM 18'
SecFilterSelective ARGS|THE_REQUEST "carmen|cash advance|cash credit|casino|catch\.com" "id:50019,msg:'SPAM 19'
SecFilterSelective ARGS|THE_REQUEST "celebrex|celexa|cellulite|cheap|cheerleader" "id:50020,msg:'SPAM 20'
SecFilterSelective ARGS|THE_REQUEST "chevrolet|child abuse|cialis|cigarette|cipro" "id:50021,msg:'SPAM 21'
SecFilterSelective ARGS|THE_REQUEST "citroen|claritin|cleavage|clomid|codeine" "id:50022,msg:'SPAM 22'
SecFilterSelective ARGS|THE_REQUEST "consulting23|craps online|credit card|credit debt|crestor" "id:50023,msg:'SPAM 23'
SecFilterSelective ARGS|THE_REQUEST "dealership|debt free|desnudas|diazepam|dick" "id:50024,msg:'SPAM 24'
SecFilterSelective ARGS|THE_REQUEST "dildo|drugstore|earrings|endometrioma|endowment" "id:50025,msg:'SPAM 25'
SecFilterSelective ARGS|THE_REQUEST "erotic|estrogen|fioricet|francaise|freehost\.com" "id:50026,msg:'SPAM 26'
SecFilterSelective ARGS|THE_REQUEST "freehostia|freemb\.com|fuck|geocities\.com|hacking myspace" "id:50027,msg:'SPAM 27'
SecFilterSelective ARGS|THE_REQUEST "holdem|honda|hotels|hydrocodone|hypnotic" "id:50028,msg:'SPAM 28'
SecFilterSelective ARGS|THE_REQUEST "hyundai|implants|incest|instant approval|insurance" "id:50029,msg:'SPAM 29'
SecFilterSelective ARGS|THE_REQUEST "interracial|jaguar|jenny movie|johanson|kasino" "id:50030,msg:'SPAM 30'
SecFilterSelective ARGS|THE_REQUEST "lesbian|levitra|lipitor|loan|lolita" "id:50031,msg:'SPAM 31'
SecFilterSelective ARGS|THE_REQUEST "lorazepam|lorcet|lyrics|madamic|majorette" "id:50032,msg:'SPAM 32'
SecFilterSelective ARGS|THE_REQUEST "malaria|mastercar|masturbate|masturbation|maturewomen" "id:50033,msg:'SPAM 33'
SecFilterSelective ARGS|THE_REQUEST "mazda|medication|medicine|megsfree5\.com|mercedes" "id:50034,msg:'SPAM 34'
SecFilterSelective ARGS|THE_REQUEST "meridia|metformin|mitsubishi|mortgage|myspace profile" "id:50035,msg:'SPAM 35'
SecFilterSelective ARGS|THE_REQUEST "naked|neocool|nexium|nimire\.com|nissan" "id:50036,msg:'SPAM 36'
SecFilterSelective ARGS|THE_REQUEST "nokia|nude|nudism|nymph|open toe" "id:50037,msg:'SPAM 37'
SecFilterSelective ARGS|THE_REQUEST "oprodol|orgasm|oxycodone|oxycontin|packages" "id:50038,msg:'SPAM 38'
SecFilterSelective ARGS|THE_REQUEST "painrelief|pantyhose|paxil|payday|penis" "id:50039,msg:'SPAM 39'
SecFilterSelective ARGS|THE_REQUEST "percocet|pharmacy|phentermine|phetermine|phpbb_root" "id:50040,msg:'SPAM 40'
SecFilterSelective ARGS|THE_REQUEST "pictaboo|pictorial|pills|pissing|play craps" "id:50041,msg:'SPAM 41'
SecFilterSelective ARGS|THE_REQUEST "playgirl|pocker web|poker|pontiac|poquer" "id:50042,msg:'SPAM 42'
SecFilterSelective ARGS|THE_REQUEST "porn|pounder|prescription|preteen|prevacid" "id:50043,msg:'SPAM 43'
SecFilterSelective ARGS|THE_REQUEST "price1|prilosec|propecia|proza|prozac" "id:50044,msg:'SPAM 44'
SecFilterSelective ARGS|THE_REQUEST "puddled|pussy|refinance|rentals|replica" "id:50045,msg:'SPAM 45'
SecFilterSelective ARGS|THE_REQUEST "ringtones|roulette|screensaver|seduced|sexual" "id:50046,msg:'SPAM 46'
SecFilterSelective ARGS|THE_REQUEST "sexy|shemale|shiloh|singulair|site-host" "id:50047,msg:'SPAM 47'
SecFilterSelective ARGS|THE_REQUEST "slot machine|slot maschine|slots machine|solpip\.com|soma" "id:50048,msg:'SPAM 48'
SecFilterSelective ARGS|THE_REQUEST "sperm|starlets|supplier|suzuki|tadalafil" "id:50049,msg:'SPAM 49'
SecFilterSelective ARGS|THE_REQUEST "toyota|tylenol|ultram|valium|viagra" "id:50050,msg:'SPAM 50'
SecFilterSelective ARGS|THE_REQUEST "vigora|vioxx|wallpaper|warez|webcam" "id:50051,msg:'SPAM 51'
SecFilterSelective ARGS|THE_REQUEST "webpages\.com|wellbutrin|whitesluts|wholesale|whore" "id:50052,msg:'SPAM 52'
SecFilterSelective ARGS|THE_REQUEST "windshield|xanax|xenical|y lohan|yourgirls" "id:50053,msg:'SPAM 53'
SecFilterSelective ARGS|THE_REQUEST "youtube\.com|zantac|sex offenders|hotgay|Zoloft|celtic women" "id:50054,msg:'SPAM 54'
SecFilterSelective ARGS|THE_REQUEST "dollhouse|freehot|kardashian|oralsex" "id:50055,msg:'SPAM 55'
SecFilterSelective ARGS|THE_REQUEST "freeimghost" "id:50056,msg:'SPAM 56'

Unreleased AskApache Lightning code - caching plugin for WordPress

RewriteEngine On
RewriteBase /
 
RewriteCond %{REQUEST_URI} ^/(stats/|missing\.html|failed_auth\.html|test/).* [NC,OR]
RewriteCond %{ENV:REDIRECT_STATUS} =200
RewriteRule .* - [L]
 
RewriteCond %{HTTP_HOST} !=www.askapache.com
RewriteCond %{HTTP_HOST} !=z.askapache.com
RewriteRule .? http://www.askapache.com%{REQUEST_URI}%{QUERY_STRING} [R=301,L]
 
#RewriteCond %{HTTPS} =on
#RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /(wp-login.php|wp-admin)(.*)\ HTTP/ [NC]
#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(.*)\ HTTP/ [NC]
#RewriteRule .* http://www.askapache.com/$1 [R=301,L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(.+)%20(.+)\ HTTP/ [NC]
RewriteRule .* http://www.askapache.com/%1-%2 [R=301,L]
 
RewriteCond %{THE_REQUEST} ^GET\ /valid-html/.*\ HTTP/ [NC]
RewriteCond %{HTTP_REFERER} ^https?://www.askapache.com(.+).*$ [NC]
RewriteRule .* http://validator.w3.org/check?uri=http://www.askapache.com%1;ss=1;outline=1;debug [R=307,L,NE]
 
RewriteCond %{THE_REQUEST} ^GET\ /valid-css/.*\ HTTP/ [NC]
RewriteRule .* http://jigsaw.w3.org/css-validator/validator?uri=z.askapache.com/z/c/apache-10.css [R=301,L]
 
RewriteCond %{THE_REQUEST} ^GET\ /search/.*\ HTTP/ [NC]
RewriteCond %{QUERY_STRING} ^s=(.+)$ [NC]
RewriteRule .* http://www.askapache.com/search/%1? [R=302,L]
 
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}/X%{REQUEST_URI}index.html -f
RewriteRule ^(.*)$ /X%{REQUEST_URI}index.html [L]
 
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}/X%{REQUEST_URI} -f
RewriteRule ^(.*)$ /X%{REQUEST_URI} [L]

Passing Parameters to CGI through HTTP Headers

This deserves a note.. this trick lets you bypass so many hosting environment restrictions it’s not even funny… like you can run bash as your webserver instead of apache! (kinda…)

RewriteCond %{REQUEST_URI} ^/cgi-bin/(hash|java)\.cgi$ [NC]
RewriteCond %{HTTP:P3P} ^(([^:]+):(.+))$ [NC]
RewriteRule .* - [E=HTTP_JS:%{HTTP:P3P},E=HTTP_US:%2,E=HTTP_PA:%3]

Redirect to FeedBurner

Yes.. I was the first to do this (parse the real rewrites internally in wordpress for no bypassing possibilities)… 

#RewriteCond %{REQUEST_URI} ^/feed/.*?$ [NC,OR]
#RewriteCond %{HTTP_REFERER} ^https?://www.askapache.com/wp-admin(.+).*$ [NC]
RewriteCond %{HTTP_USER_AGENT} FeedBurner [NC]
RewriteRule .* - [S=1]
 
RewriteCond %{REQUEST_URI} ^/(feed|rdf|rss|rss2|atom).* [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp-(feed|rdf|rss|rss2|atom).* [NC,OR]
RewriteCond %{REQUEST_FILENAME} ^.*/(wp-atom|wp-feed|wp-rdf|wp-rss|wp-rss2)\.php$ [NC,OR]
RewriteCond %{REQUEST_FILENAME} ^.*/wp-includes/feed[^\.]*\.php$ [NC,OR]
RewriteCond %{QUERY_STRING} .*(feed|rdf|rss|rss2|atom).* [NC]
RewriteRule .* http://feeds.askapache.com/apache/htaccess? [R=302,L]

More Unreleased Caching Tests

# +ASKAPACHE CRAZYCACHE 2.3
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +ACACHE RULES
# +RULE
RewriteCond %{THE_REQUEST} ^(GET|HEAD)\ //?(.+)\.rdf\ HTTP/ [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-cache/%2.html/index.rdf -f
RewriteRule ^(.+)\.rdf$ /wp-cache/%2.html/index.rdf [L,S=3]
 
RewriteCond %{QUERY_STRING} !^.*cacheit.* [NC]
#RewriteCond %{HTTP_USER_AGENT} W3C_Validator [OR,NC]
#RewriteCond %{REMOTE_ADDR} = [OR]
 
RewriteCond %{REQUEST_METHOD} =POST [OR]
RewriteCond %{THE_REQUEST} ^$ [OR]
RewriteCond %{QUERY_STRING} nocache [OR]
RewriteCond %{REMOTE_ADDR} = [OR]
RewriteCond %{HTTP_COOKIE} ^.*wordpress_logged_in_.*$ [NC]
RewriteRule .+ - [S=2]
 
RewriteCond %{DOCUMENT_ROOT}/wp-cache%{REQUEST_URI} !-d
RewriteRule .+ - [S=1]
 
RewriteCond %{DOCUMENT_ROOT}/wp-cache%{REQUEST_URI}/index.html -f
RewriteRule .+ /wp-cache%{REQUEST_URI}/index.html [L]
# -RULE
# -ACACHE RULES
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE CRAZYCACHE 2.3
 
# +ASKAPACHE CRAZYCACHE 2.3
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +ACACHE RULES
# +RULE
FileETag None
AddOutputFilterByType DEFLATE text/html
 
AddType 'text/html; charset=UTF-8' .html
 
Header set P3P "policyref=\"http://www.askapache.com/w3c/p3p.xml\""
Header set X-Pingback "http://www.askapache.com/xmlrpc.php"
Header set Content-Language "en-US"
Header set Vary "Accept-Encoding,Accept"
 
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault M7200
</IfModule>
# -RULE
# -ACACHE RULES
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE CRAZYCACHE 2.3
 
Options +IndexesOptions +FollowSymLinks
AddHandler application/x-httpd-php .php
 
RewriteEngine On
RewriteBase /
RewriteCond %{QUERY_STRING} !^tyy+$ [NC]
RewriteCond %{REMOTE_USER} ^(.+)$
RewriteRule ^(.*)$ $1/-%1 [R=302,L]
Options +FollowSymLinks
DirectoryIndex index.php
ErrorDocument 404 /cgi-bin/pro/index.php
 
AuthName "Protection"
AuthUserFile /home/askapache/sites/askapache.com/.htpasswda1
AuthGroupFile /dev/null
AuthType Basic
Require valid-user
Satisfy Any
 
<IfModule mod_security.c>
SecFilterEngine Off
</IfModule>
Options +ExecCGI -Indexes -Includes +FollowSymLinks
Order Deny,Allow
Deny from All
Allow from 208.113.134.190 64.111.114.111  env=REDIRECT_STATUS
 
#SetEnvIf Remote_Addr ^$ MODSEC_ENABLE=Off
#SetEnvIf Server_Addr ^$ MODSEC_ENABLE=Off
#SetEnvIf Remote_Addr ^({SERVER_ADDR}e)$ GOOD=$1
#SetEnvIf Server_Addr GOOD R

Default HTACCESS

#
# Directives controlling the display of server-generated directory listings.
#
# Required modules: mod_autoindex, mod_alias
#
# To see the listing of a directory, the Options directive for the
# directory must include "Indexes", and the directory must not contain
# a file matching those listed in the DirectoryIndex directive.
#
Options +Indexes +MultiViews -ExecCGI +FollowSymLinks
DirectoryIndex index.html
 
#
#
# IndexOptions: Controls the appearance of server-generated directory
# listings.
#
IndexOptions FancyIndexing IconHeight=22 IconWidth=20 IgnoreClient NameWidth=* DescriptionWidth=* ScanHTMLTitles SuppressLastModified XHTML FoldersFirst SuppressHTMLPreamble
 
#
# AddIcon* directives tell the server which icon to show for different
# files or filename extensions.  These are only displayed for
# FancyIndexed directories.
#
 
AddIconByEncoding (CMP,http://z.askapache.com/i/s/compressed.gif) x-compress x-gzip
 
AddIconByType (IMG,http://z.askapache.com/i/s/image.gif) image/*
AddIconByType (SND,http://z.askapache.com/i/s/sound2.gif) audio/*
AddIconByType (TXT,http://z.askapache.com/i/s/text.gif) text/*
AddIconByType (VID,http://z.askapache.com/i/s/movie.gif) video/*
 
AddIcon http://z.askapache.com/i/s/hand.right.gif README
AddIcon http://z.askapache.com/i/s/folder.gif ^^DIRECTORY^^
AddIcon http://z.askapache.com/i/s/dir.png ^^DIRECTORY^^
AddIcon http://z.askapache.com/i/s/blank.gif ^^BLANKICON^^
AddDescription "100 Continue" 100* 100/index.html
AddDescription "101 Switching Protocols" 101* 101/index.html
AddDescription "102 Processing" 102* 102/index.html
AddDescription "200 OK" 200* 200/index.html
AddDescription "201 Created" 201* 201/index.html
AddDescription "202 Accepted" 202* 202/index.html
AddDescription "203 Non-Authoritative Information" 203* 203/index.html
AddDescription "204 No Content" 204* 204/index.html
AddDescription "205 Reset Content" 205* 205/index.html
AddDescription "206 Partial Content" 206* 206/index.html
AddDescription "207 Multi-Status" 207* 207/index.html
AddDescription "300 Multiple Choices" 300* 300/index.html
AddDescription "301 Moved Permanently" 301* 301/index.html
AddDescription "302 Found" 302* 302/index.html
AddDescription "303 See Other" 303* 303/index.html
AddDescription "304 Not Modified" 304* 304/index.html
AddDescription "305 Use Proxy" 305* 305/index.html
AddDescription "306 unused" 306* 306/index.html
AddDescription "307 Temporary Redirect" 307* 307/index.html
AddDescription "400 Bad Request" 400* 400/index.html
AddDescription "401 Authorization Required" 401* 401/index.html
AddDescription "402 Payment Required" 402* 402/index.html
AddDescription "403 Forbidden" 403* 403/index.html
AddDescription "404 Not Found" 404* 404/index.html
AddDescription "405 Method Not Allowed" 405* 405/index.html
AddDescription "406 Not Acceptable" 406* 406/index.html
AddDescription "407 Proxy Authentication Required" 407* 407/index.html
AddDescription "408 Request Time-out" 408* 408/index.html
AddDescription "409 Conflict" 409* 409/index.html
AddDescription "410 Gone" 410* 410/index.html
AddDescription "411 Length Required" 411* 411/index.html
AddDescription "412 Precondition Failed" 412* 412/index.html
AddDescription "413 Request Entity Too Large" 413* 413/index.html
AddDescription "414 Request-URI Too Large" 414* 414/index.html
AddDescription "415 Unsupported Media Type" 415* 415/index.html
AddDescription "416 Requested Range Not Satisfiable" 416* 416/index.html
AddDescription "417 Expectation Failed" 417* 417/index.html
AddDescription "418 unused" 418* 418/index.html
AddDescription "419 unused" 419* 419/index.html
AddDescription "420 unused" 420* 420/index.html
AddDescription "421 unused" 421* 421/index.html
AddDescription "422 Unprocessable Entity" 422* 422/index.html
AddDescription "423 Locked" 423* 423/index.html
AddDescription "424 Failed Dependency" 424* 424/index.html
AddDescription "425 No code" 425* 425/index.html
AddDescription "426 Upgrade Required" 426* 426/index.html
AddDescription "500 Internal Server Error" 500* 500/index.html
AddDescription "501 Method Not Implemented" 501* 501/index.html
AddDescription "502 Bad Gateway" 502* 502/index.html
AddDescription "503 Service Temporarily Unavailable" 503* 503/index.html
AddDescription "504 Gateway Time-out" 504* 504/index.html
AddDescription "505 HTTP Version Not Supported" 505* 505/index.html
AddDescription "506 Variant Also Negotiates" 506* 506/index.html
AddDescription "507 Insufficient Storage" 507* 507/index.html
AddDescription "508 unused" 508* 508/index.html
AddDescription "509 unused" 509* 509/index.html
AddDescription "510 Not Extended" 510* 510/index.html
AddDescription ".htaccess ErrorDocuments" *
 
#
# HeaderName is the name of a file which should be prepended to
# directory indexes.
ReadmeName /X/error/README.html
HeaderName /X/error/HEADER.html
 
# DefaultIcon is which icon to show for where none is explicitly set.
DefaultIcon http://z.askapache.com/i/s/generic.gif

Ok I’m done commenting..

#
# IndexIgnore is a set of filenames which directory indexing should ignore
# and not include in the listing.  Shell-style wildcarding is permitted.
#
IndexIgnore .??*  *_notes *~
RewriteEngine On
RewriteCond %{HTTPS} !=on [NC]
RewriteRule .* https://www.askapache.com%{REQUEST_URI}
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},E=REMOTE_USER:%{HTTP:Authorization}]
 
#RewriteCond %{QUERY_STRING} openid.mode=authorize
#RewriteCond %{REQUEST_URI} ^/openid.*$ [NC]
#RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},E=REMOTE_USER:%{HTTP:Authorization}]
Options +FollowSymLinks
DirectoryIndex index.html
 
#SetEnvIfNoCase ^Cookie$ "(.*)" HTTP_MY_COOKIE=$1
#SetEnvIfNoCase Remote_Addr "(.*)" HTTP_MY_REMOTE_ADDR=$1
ExpiresActive Off
FileETag None
Header unset Connection
Header set Connection "close"
Header unset Last-Modified
Header unset ETag
Header unset Accept-Ranges
Header unset Vary
Header unset Content-Type
Header unset X-Pingback
Header unset P3P
#Header add RouterBits "%D %t"
#Header add Location "http://www.askapache.com/feed/"
#Header add Found "http://askapache.com/feed/"
#Header add Content-Location "http://www.askapache.com/feed/"
#Header add Refresh "http://www.askapache.com/feed/"
#Header set Hi "%{HTTP_MY_REMOTE_ADDR}e"
### Section 1: Global Environment
#
# The directives in this section affect the overall operation of Apache,
# such as the number of concurrent requests it can handle or where it
# can find its configuration files.
#
#
# TZ: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv TZ America/Indianapolis
 
#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv SERVER_ADMIN webmaster@askapache.com
 
#
# Possible values for the Options directive are "None", "All",
# or any combination of:
#   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
#
# The Options directive is both complicated and important.  Please see
# http://httpd.apache.org/docs/2.0/mod/core.html#options
# for more information.
#
Options  +FollowSymLinks -ExecCGI -Indexes -Includes -MultiViews
 
#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
DirectoryIndex index.html index.php /index.php
 
#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<FilesMatch "^\.ht">
    Order allow,deny
    Deny from all
  Satisfy All
</FilesMatch>
 
#
# DefaultType: the default MIME type the server will use for a document
# if it cannot otherwise determine one, such as from filename extensions.
# If your server contains mostly text or HTML documents, "text/plain" is
# a good value.  If most of your content is binary, such as applications
# or images, you may want to use "application/octet-stream" instead to
# keep browsers from trying to display binary files as though they are
# text.
#
DefaultType text/html
 
#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature Off
 
##############################################
#           HEADERS and CACHING              #
##############################################
Header unset Pragma
Header unset Last-Modified
FileETag None
 
Redirect 400 /e/400
Redirect 401 /e/401
Redirect 402 /e/402
Redirect 403 /e/403
Redirect 404 /e/404
Redirect 405 /e/405
Redirect 406 /e/406
Redirect 407 /e/407
Redirect 408 /e/408
Redirect 409 /e/409
Redirect 410 /e/410
Redirect 411 /e/411
Redirect 412 /e/412
Redirect 413 /e/413
Redirect 414 /e/414
Redirect 415 /e/415
Redirect 416 /e/416
Redirect 417 /e/417
Redirect 418 /e/418
Redirect 419 /e/419
Redirect 420 /e/420
Redirect 421 /e/421
Redirect 422 /e/422
Redirect 423 /e/423
Redirect 424 /e/424
Redirect 425 /e/425
Redirect 426 /e/426
Redirect 500 /e/500
Redirect 501 /e/501
Redirect 502 /e/502
Redirect 503 /e/503
Redirect 504 /e/504
Redirect 505 /e/505
Redirect 506 /e/506
Redirect 507 /e/507
Redirect 508 /e/508
Redirect 509 /e/509
Redirect 510 /e/510
 

#
# Redirect allows you to tell clients about documents which used to exist in
# your server's namespace, but do not anymore. This allows you to tell the
# clients where to look for the relocated document.
##############################################
#          PERMANENT REDIRECTS               #
##############################################
Redirect 301 /12-lessons-for-those-afraid-of-css.html http://www.askapache.com/css/12-lessons-for-those-afraid-of-css.html
Redirect 301 /2006/htaccess/htaccesselite-ultimate-htaccess-article.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /2007/phpbb/sending-post-form-data-with-php-curl.html http://www.askapache.com/htaccess/sending-post-form-data-with-php-curl.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/php-and-ajax-shell-console.html http://www.askapache.com/tools/php-and-ajax-shell-console.html
Redirect 301 /27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html http://www.askapache.com/htaccess/27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html
Redirect 301 /404-google-wordpress-plugin.html http://www.askapache.com/seo/404-google-wordpress-plugin.html
Redirect 301 /503-service-temporarily-unavailable.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /Overview-about.rdf http://www.askapache.com/askapache-home.rdf
Redirect 301 /abbr-acronym.html http://www.askapache.com/xhtml/abbr-acronym.html
Redirect 301 /adsense-robots.html http://www.askapache.com/google/adsense-robots.html
Redirect 301 /alexa-toolbar-firefox.html http://www.askapache.com/tools/alexa-toolbar-firefox.html
Redirect 301 /allowing-access-from-1-static-ip-and-deny-the-rest.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /anti-virus-spyware-rootkit.html http://www.askapache.com/security/anti-virus-spyware-rootkit.html
Redirect 301 /apache-ssl-in-htaccess-examples.html http://www.askapache.com/htaccess/apache-ssl-in-htaccess-examples.html
Redirect 301 /awk-tutorial.html http://www.askapache.com/awk/awk-tutorial.html
Redirect 301 /best-adsense-optimization.html http://www.askapache.com/wordpress/best-adsense-optimization.html
Redirect 301 /commonly-used-htaccess-code-examples.html http://www.askapache.com/htaccess/commonly-used-htaccess-code-examples.html
Redirect 301 /css-background-image-sprite.html http://www.askapache.com/css/css-background-image-sprite.html
Redirect 301 /css-browser-screenshots.html http://www.askapache.com/tools/css-browser-screenshots.html
Redirect 301 /css-class-example.html http://www.askapache.com/css/css-class-example.html
Redirect 301 /curl-multi-downloads.html http://www.askapache.com/php/curl-multi-downloads.html
Redirect 301 /custom-boot-menu-in-windows-xp.html http://www.askapache.com/windows/custom-boot-menu-in-windows-xp.html
Redirect 301 /donate http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /donate/ http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /htaccess.txt http://z.askapache.com/p/htaccess.txt
Redirect 301 /htaccess/404-errorpages.html http://www.askapache.com/seo/google-ajax-search-seo-tips.html
Redirect 301 /htaccess/feedsmith http://www.askapache.com/htaccess/feedsmith-htaccess.html
Redirect 301 /htaccess/http-status-codes.html http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html
Redirect 301 /htaccess/instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /htaccess/speed-up-the-apache-web-server-with-configuration-hacks.html http://www.askapache.com/htaccess/apache-web-server-speed-configuration-hacks.html
Redirect 301 /instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /security/bypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/bypassing-vlanbypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/rigging-the-dreamhost-site-of-the-month-contest.html http://www.askapache.com/dreamhost/rigging-the-dreamhost-site-of-the-month-contest.html
Redirect 301 /seo/tailrankcom-robot.html http://www.askapache.com/seo/tailrank-robot.html
Redirect 301 /webmaster/caching-tutorial-for-webmasters.html http://www.askapache.com/htaccess/caching-tutorial-for-webmasters.html
Redirect 301 /webmaster/lft-traceroute-tool.html http://www.askapache.com/tools/lft-traceroute-tool.html
 
##############################################
#          PERMANENT REDIRECTMATCH           #
##############################################
#RedirectMatch 301 ^/&(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/&amp(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/([^/]+)//$ http://www.askapache.com/$1/
RedirectMatch 301 ^/(.+)/htaccesselite-ultimate-htaccess-article.html(.*) http://www.askapache.com/htaccess/apache-htaccess.html
RedirectMatch 301 ^/(.+)\.html/([a-z][a-z])/$ http://www.askapache.com/$1.html
RedirectMatch 301 ^/([\(]+)(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/([^9]*)9O1X.3y(.*)/(.*)$ http://www.askapache.com/$2
RedirectMatch 301 ^/.3y(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/200([0-9])/([0-9])(.*)$ http://www.askapache.com/top-100/
RedirectMatch 301 ^/200([0-9])/([^01])(.*)$ http://www.askapache.com/$2$3
RedirectMatch 301 ^/about/glossary(.*)$ http://www.askapache.com/glossary$1
RedirectMatch 301 ^/apache-speed(.*)$ http://www.askapache.com/htaccess/apache-speed$1
RedirectMatch 301 ^/category/(.+)$ http://www.askapache.com/$1
RedirectMatch 301 ^/docs/(.*)$ http://askapache.info/$1
RedirectMatch 301 ^/htaccess/feedsmith-htaccess(.*) http://www.askapache.com/htaccess/redirecting-wordpress-feeds-to-feedburner.html
RedirectMatch 301 ^/robots-txt(.*)$ http://www.askapache.com/robots.txt
#RedirectMatch 301 ^/wp-content/uploads/(.*)$ http://z.askapache.com/uploads/$1
RedirectMatch 301 ^/z/(.+)$ http://z.askapache.com/$1
RedirectMatch 301 ^/(z|t|i|j|c|p)/(.*)$ http://z.askapache.com/$1/$2
RedirectMatch 301 ^/hosting/?$ http://www.askapache.com/web-hosting/
 
##############################################
#          TEMPORARY REDIRECTMATCH           #
##############################################
RedirectMatch 307 ^/getflash/?$ http://www.adobe.com/shockwave/download/download.cgi?P1_Prod_Version=ShockwaveFlash
RedirectMatch 307 ^/dream/?$ http://www.askapache.com/dreamhost/
RedirectMatch 307 ^/(cse|apachecse|apachecsetest|apachesearch)/?$ http://www.google.com/coop/cse?cx=002660089121042511758%3Akk7rwc2gx0i
 

#
# Commonly used filename extensions to character sets. You probably
# want to avoid clashes with the language extensions, unless you
# are good at carefully testing your setup after each change.
# See http://www.iana.org/assignments/character-sets for the
# official list of charset names and their respective RFCs.
#
AddDefaultCharset UTF-8
 
#
# AddType allows you to add to or override the MIME configuration
# file mime.types for specific file types.
#
#
AddType 'application/rdf+xml; charset=UTF-8' .rdf
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml.gz
AddType 'text/html; charset=UTF-8' .html
AddType 'text/html; charset=UTF-8' .html.gz
AddType application/octet-stream .rar .chm .bz2 .tgz .msi .pdf .exe
AddType application/vnd.ms-excel .csv
AddType application/x-httpd-php-source .phps
AddType application/x-pilot .prc .pdb
AddType application/x-shockwave-flash .swf
AddType application/xrds+xml .xrdf
AddType text/plain .ini .sh .bsh .bash .awk .nawk .gawk .csh .var .c .in .h .asc .md5 .sha .sha1
AddType video/x-flv .flv
 
#
# AddEncoding allows you to have certain browsers uncompress
# information on the fly. Note: Not all browsers support this.
# Despite the name similarity, the following Add* directives have nothing
# to do with the FancyIndexing customization directives above.
#
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
 
#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
AddHandler php-cgi .php
 
#
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#
Action php-cgi /cgi-bin/php.cgi
 
#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
#100 Continue
#101 Switching Protocols
#102 Processing
#200 OK
#201 Created
#202 Accepted
#203 Non-Authoritative Information
#204 No Content
#205 Reset Content
#206 Partial Content
#207 Multi-Status
#300 Multiple Choices
#301 Moved Permanently
#302 Found
#303 See Other
#304 Not Modified
#305 Use Proxy
#306 unused
#307 Temporary Redirect
#400 Bad Request
#401 Authorization Required
#402 Payment Required
#403 Forbidden
#404 Not Found
#405 Method Not Allowed
#406 Not Acceptable
#407 Proxy Authentication Required
#408 Request Time-out
#409 Conflict
#410 Gone
#411 Length Required
#412 Precondition Failed
#413 Request Entity Too Large
#414 Request-URI Too Large
#415 Unsupported Media Type
#416 Requested Range Not Satisfiable
#417 Expectation Failed
#418 unused
#419 unused
#420 unused
#421 unused
#422 Unprocessable Entity
#423 Locked
#424 Failed Dependency
#425 No code
#426 Upgrade Required
#500 Internal Server Error
#501 Method Not Implemented
#502 Bad Gateway
#503 Service Temporarily Unavailable
#504 Gateway Time-out
#505 HTTP Version Not Supported
#506 Variant Also Negotiates
#507 Insufficient Storage
#508 unused
#509 unused
#510 Not Extended
 
#ErrorDocument 100 /X/err/1/HTTP_CONTINUE.html
#ErrorDocument 101 /X/err/1/HTTP_SWITCHING_PROTOCOLS.html
#ErrorDocument 102 /X/err/1/HTTP_PROCESSING.html
 
#ErrorDocument 200 /X/err/2/HTTP_OK.html
#ErrorDocument 201 /X/err/2/HTTP_CREATED.html
#ErrorDocument 202 /X/err/2/HTTP_ACCEPTED.html
#ErrorDocument 203 /X/err/2/HTTP_NON_AUTHORITATIVE.html
#ErrorDocument 204 /X/err/2/HTTP_NO_CONTENT.html
#ErrorDocument 205 /X/err/2/HTTP_RESET_CONTENT.html
#ErrorDocument 206 /X/err/2/HTTP_PARTIAL_CONTENT.html
#ErrorDocument 207 /X/err/2/HTTP_MULTI_STATUS.html
 
#ErrorDocument 300 /X/err/HTTP_MULTIPLE_CHOICES.html
#ErrorDocument 301 /X/err/HTTP_MOVED_PERMANENTLY.html
#ErrorDocument 302 /X/err/HTTP_MOVED_TEMPORARILY.html
#ErrorDocument 303 /X/err/HTTP_SEE_OTHER.html
#ErrorDocument 304 /X/err/HTTP_NOT_MODIFIED.html
#ErrorDocument 305 /X/err/HTTP_USE_PROXY.html
#ErrorDocument 307 /X/err/HTTP_TEMPORARY_REDIRECT.html
#ErrorDocument 404 /X/err/HTTP_NOT_FOUND.html
 
ErrorDocument 400 /X/err/4/HTTP_BAD_REQUEST.html
ErrorDocument 401 /X/err/4/HTTP_UNAUTHORIZED.html
ErrorDocument 402 /X/err/4/HTTP_PAYMENT_REQUIRED.html
ErrorDocument 403 /X/err/4/HTTP_FORBIDDEN.html
ErrorDocument 405 /X/err/4/HTTP_METHOD_NOT_ALLOWED.html
ErrorDocument 406 /X/err/4/HTTP_NOT_ACCEPTABLE.html
ErrorDocument 407 /X/err/4/HTTP_PROXY_AUTHENTICATION_REQUIRED.html
ErrorDocument 408 /X/err/4/HTTP_REQUEST_TIME_OUT.html
ErrorDocument 409 /X/err/4/HTTP_CONFLICT.html
ErrorDocument 410 /X/err/4/HTTP_GONE.html
ErrorDocument 411 /X/err/4/HTTP_LENGTH_REQUIRED.html
ErrorDocument 412 /X/err/4/HTTP_PRECONDITION_FAILED.html
ErrorDocument 413 /X/err/4/HTTP_REQUEST_ENTITY_TOO_LARGE.html
ErrorDocument 414 /X/err/4/HTTP_REQUEST_URI_TOO_LARGE.html
ErrorDocument 415 /X/err/4/TTP_UNSUPPORTED_MEDIA_TYPE.html
ErrorDocument 416 /X/err/4/HTTP_RANGE_NOT_SATISFIABLE.html
ErrorDocument 417 /X/err/4/HTTP_EXPECTATION_FAILED.html
ErrorDocument 422 /X/err/4/HTTP_UNPROCESSABLE_ENTITY.html
ErrorDocument 423 /X/err/4/HTTP_LOCKED.html
ErrorDocument 424 /X/err/4/HTTP_FAILED_DEPENDENCY.html
ErrorDocument 426 /X/err/4/HTTP_UPGRADE_REQUIRED.html
 
ErrorDocument 500 /X/err/5/HTTP_INTERNAL_SERVER_ERROR.html
ErrorDocument 501 /X/err/5/HTTP_NOT_IMPLEMENTED.html
ErrorDocument 502 /X/err/5/HTTP_BAD_GATEWAY.html
ErrorDocument 503 /X/err/5/HTTP_SERVICE_UNAVAILABLE.html
ErrorDocument 504 /X/err/5/HTTP_GATEWAY_TIME_OUT.html
ErrorDocument 505 /X/err/5/HTTP_VERSION_NOT_SUPPORTED.html
ErrorDocument 506 /X/err/5/HTTP_VARIANT_ALSO_VARIES.html
ErrorDocument 507 /X/err/5/HTTP_INSUFFICIENT_STORAGE.html
ErrorDocument 510 /X/err/5/HTTP_NOT_EXTENDED.html
ErrorDocument 404 /index.php?error=404
 
# 1 YEAR
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
Header unset P3P
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</FilesMatch>
 
# 2 HOURS
<FilesMatch "\.(html|htm|xml|txt|xsl)$">
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault A3600
</IfModule>
</FilesMatch>
 
RewriteEngine On
RewriteBase /
 
RewriteCond %{ENV:REDIRECT_STATUS} =200
RewriteRule .* - [L]
 
RewriteCond %{REQUEST_METHOD} !^(GET|HEAD) [OR]
#RewriteCond %{QUERY_STRING} !^$ [OR]
RewriteCond %{HTTP_COOKIE} ^.*(comment_author_|wordpress|wp-postpass_).*$ [NC]
RewriteRule ^(.*)$ - [S=6]
 
RewriteCond %{HTTP_USER_AGENT} .*W3C_Validator.* [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml.gz -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml.gz [L,S=5]
 
RewriteCond %{HTTP:Accept} application/xhtml\+xml [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml.gz -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml.gz [L,S=4]
 
RewriteCond %{HTTP:Accept-Encoding} gzip [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.html.gz -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.html.gz [L,S=3]
 
RewriteCond %{HTTP_USER_AGENT} .*W3C_Validator.* [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml [L,S=2]
 
RewriteCond %{HTTP:Accept} application/xhtml\+xml [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml [L,S=1]
 
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.html -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.html [L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(.+)\.rdf\ HTTP/ [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/rdf/%1.html/index.rdf -f
RewriteRule ^(.*)$ /wp-content/rdf/%1.html/index.rdf [L,S=1]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(error|w3c|openid)(/?.*)\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /X/%1%2 [L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(y_key_bf61afd498f7623a\.html|google3bb7b7e1032ad0d4\.html|yadis\.xrdf|askapache-home\.rdf|os-description\.xml|labels\.rdf|gnu-fdl\.txt|wlmmanifest\.xml|robots\.txt)\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .* /X/%1 [L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /X(.*)\ HTTP/ [NC]
RewriteRule .* - [F]
 
RewriteRule ^osq?(.*)$ /wp-content/plugins/wp-opensearch.php?$1 [QSA,L]
 
#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-login(.+)\ HTTP/ [NC]
#RewriteRule .+ https://www.askapache.com/wp-login%1 [R,L]
 
RewriteCond %{REQUEST_URI} ^/cgi-bin/(hash|java)\.cgi$ [NC]
RewriteCond %{HTTP:P3P} ^(([^:]+):(.+))$ [NC]
RewriteRule .* - [E=HTTP_JS:%{HTTP:P3P},E=HTTP_US:%2,E=HTTP_PA:%3]
 

RewriteCond %{HTTP_ACCEPT} application/xrds\+xml
RewriteCond %{HTTP_ACCEPT} !application/xrds\+xml\s*;\s*q\s*=\s*0(\.0{1,3})?\s*(,|$)
RewriteRule ^$ http://www.askapache.com/yadis.xrdf [R,L]
 
RewriteCond %{HTTP_USER_AGENT} !^(FeedBurner|FeedValidator|talkr.com).* [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(feed|rdf|rss|rss2|atom)/?.*\ HTTP/ [NC]
#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(feed|rdf|rss|rss2|atom|wp-atom|wp-feed|wp-rdf|wp-rss|wp-rss2).*\ HTTP/ [NC]
RewriteRule .* http://feeds.askapache.com/apache/htaccess? [R=302,L]
 
#RewriteCond %{REMOTE_ADDR} ^$
#RewriteCond %{REQUEST_FILENAME} !-f
#RewriteCond %{REQUEST_FILENAME} !-d
#RewriteRule ^(.+)\.phps$ /cgi-bin/phps.php?file=$1.php [L,NC]

Real-Life Htaccess Files from My Server originally appeared on AskApache.com

]]> http://www.askapache.com/htaccess/real-world-htaccess-files.html/feed/ 4 30x Faster WP-Super Cache Site Speed http://www.askapache.com/web-hosting/super-speed-secrets.html http://www.askapache.com/web-hosting/super-speed-secrets.html#comments Thu, 18 Mar 2010 15:43:21 +0000 AskApache http://www.askapache.com/?p=3220

NOT a typo.. 30x is measurable, well-documented, and easily tested. This is what open-source is about. I haven’t had time to post much the past year, I'm always working! So I wanted to make up for that by publishing an article on a topic that would blow your mind and be something that you could actually start using and really get some benefit out of it. This is one of those articles that the majority of web hosting companies would love to see in paperback, so they could burn it.

]]> Top showing swap and memory

I haven’t had time to post much the past year, so I wanted to make up for that by publishing an article on a topic that would blow your mind and be something that you could actually start using and really get some benefit out of it. This is one of those articles that the majority of web hosting companies would love to see in paperback, so they could burn it. Now ask yourself, if a webhost makes money based on how much memory, bandwidth, and data used by a customer, what would they not want their customers to do? That’s right, they do not want their customers to learn how to minimize and drastically reduce these moneymakers. They get giddy when you complain about slow-site-speed, or that it takes a long time for your site to load, because they have exactly the right answer- upgrade your memory, bandwidth, and data by purchasing a more expensive plan.

WARNING!! This article has some seriously advanced stuff in it, pretty far beyond my skill level as well (getting there). I personally shutdown some of my own servers with various webhosts because of this.. Note I said personally, not intentionally. Even after spending almost a year (this has been in my drafts folder a long time) using TMPFS on as many machines as I can, I still make mistakes (gotta pay attention!) and lose a tmpfs folder.. Oh and if you go experimenting with this stuff on your web host, you will almost definately, most certainly be on the road to getting your account terminated if you are with one of the cheap hosts. They hate this stuff because it cuts right into the heart of their profit curves and can seriously disrupt a poorly configured machine. DO NOT TRY THIS!! (except and of course on your own development machines). Of course the whole point of this article is how you can take advantage of this incredible filesystem to get crazy speed improvements.. Those are the follow up articles ;)

For those of you who thought modifying your server httpd.conf and htaccess files is very dangerous, you are right. But this is not like that, this is dangerous in the sense that if you try to rush through with your super amazing “copy and paste skills” (script kids) you will easily lose entire folders. That’s because TMPFS is stored in RAM/Memory, and upon reboot RAM is cleared. I personally loathe disclaimers, and if you look around you will see there aren’t many even with all my sloppy poorly documented articles… So be careful if you feel up to going further.

Introducing tmpfs

If I had to explain tmpfs in one breath, I’d say that tmpfs is like a ramdisk, but different. Like a ramdisk, tmpfs can use your RAM, but it can also use your swap devices for storage. And while a traditional ramdisk is a block device and requires a mkfs command of some kind before you can actually use it, tmpfs is a filesystem, not a block device; you just mount it, and it’s there. All in all, this makes tmpfs the niftiest RAM-based filesystem I’ve had the opportunity to meet.

Beware of WebHosts

What is a modern day web hosting company? What costs do they actually have? A webhost’s only unique ability is their connection to the Internet. That is why you can see such tremendous link speed. Other than that they consist of servers that are getting smaller and cheaper for them every month. The servers they use are generally just like any computer, except much larger and built specifically for multi-tasking.

Virtualization allows you to run multiple applications and operating systems independently on a single server. Additionally, administrators can quickly move workloads from one virtual workspace to another — easily prioritizing business needs while maximizing server resources….

Virtualization removes the limitations of the traditional IT approach, enabling a single PowerEdge server to operate multiple applications simultaneously in “virtual machines”

Hosting Company Tricks

Web hosts like to vaguely describe their products as if you are buying your own powerful machine, but in reality you get placed on the same machine as hundreds or thousands of other customers, and the server basically creates an operating system for each customer using virtualization technology. Everyone on the machine literally is sharing the same RAM and resources, many times even sharing IP address’s, and the virtualization software lets them limit the amount of memory / cpu / disk / and bandwidth for each of these virtual machines. That is why so often when a web host has an outage they make big public announcements and it appears that hundreds or thousands of their customers have been affected.. One of their server farm machines goes offline and it literally takes down all the customers virtualized machines with it.

Why it gets Evil

Don’t get me wrong, I absolutely love this technology, both the hardware virtualization and the software side, but what I truly do not appreciate is how these companies take advantage of their customers every day and know it. Here’s what they do, they make justifications about why one plan costs more than another, and these justifications are always about the same thing: CPU’s, how fast the data can crunch.. RAM/Memory: How fast and how much your server can handle in terms of traffic… Disk Usage: How much storage you have… And finally bandwidth: How fast can people get data off your sites, and how many people can connect.

Now lets think for a second. The webhost has a BIG computer/server/machine that has MASSIVE amounts of RAM, DISK, PROCESSING power, and NETWORK bandwidth.. but just like anything they all have limits. So if this machine has 10GB of RAM, and the webhost offered plans that have 1GB of RAM, then on that machine they can only have 10 customers right? WRONG. If each customer pays $100/month, then of course they would love to have as many customers on that machine as possible. This builtin incentive is just the reality and isn’t anyone fault.

Where it gets Evil

Here’s what goes on.. all the host advertises is the 1GB of guaranteed RAM with your machine, but for even if the web server was fairly busy it would never use all of that ram because all the software is careful not to use too much, or has no need for any RAM. Runtime libraries and internal caches use ram, but it’s not directly accessed by the customer, only the software. What happens is when those 10 customers aren’t using 100% of their ram, which never happens, then the virtualization technology can use that RAM elsewhere. So technically you do have 1GB of RAM available, but if you aren’t using it then it is essentially FREE RAM that they can sell to another customer. The only way this wouldn’t work of course is if all 11 customers somehow used 100% of RAM simultaneously, at that point the 11th customer would be ramless. But that is impossible because the system is a load-balancing system that provides both an upper and a lower limit to how much RAM is allotted to each virtual machine.

It sounds unrealistic but I see server farms all the time that are stuffed full of virtual machines, like situations where there are 100 1GB customers all sharing 10GB of RAM.. no-one uses the whole 1GB allotted to them as the maximum amount they can use, and they don’t know because it appears they have a lot of free RAM, but really that is virtual RAM and could be used by anyone else on the machine.

Where it gets Fun (for me)

The HTOP command in full color to manage mysqlThis is actually even worse for anyone who is using what they call “shared-hosting” which is the budget hosting that is the most common. With shared-hosting there is actually some skill involved on the hosting companies part, like real linux skills. In this setup they may or more often may not use any virtualization software. It’s just a vanilla multi-user server machine where each customer gets a restricted unix account that powers their website using the same system as thousands of others on the box. This is usually dirt cheap because it costs so little to do, but alot of companies charge outrageous amounts for shared-hosting because they make it look really full-featured, which it can be, they just don’t mention 1000 other people use the same machine, hard-drive, /tmp directory, network device, IP address, etc.. Alot of the times the cheaper end of the spectrum is where the most gifted system administrators are located, they are so good with linux administration that they could fit 10 customers and 100 websites on an XBOX converted to run linux, and you’d think you got a great deal until you found out! lol. Anyone alive is able to buy more hardware to expand their capacity to take on more customers, but it takes a lot of knowhow and real skill to have that many users on 1 machine. I’ve seen pretty extreme cases that are analogous to the XBOX example (which is possible by the way).

I personally love shared-hosting environments, because for those of us who know almost as much or more than the system administrators running the machine we are able to use a disproportionate (legally) amount of the CPU and RAM available on the system. So for example my sites would all show up fast and be able to handle more traffic than several other customers combined. Not because
anything has been circumvented, but because I am able to access and utilize as much of the guaranteed 1GB of RAM that I am paying for every month, which is usually just a few bucks. The downside is that when you have corporate sites or really high-traffic sites then you are forced to move to a more powerful machine..

This leads to a familiar situation for some of you.. When your site starts becoming popular and you are getting a lot of traffic, this means that your site could be using 10x the amount of RAM and Bandwidth of any other customer in that server farm. And what that really means to the webhost is that you are costing them 10x what anyone else is.. And if they removed you, they would have the space for 10 new customers to take your place, and they would make 10x more money. DreamHost is notorious for terminating accounts because of that.. It happened to me except I was given the option to pay 5x more a month for their “upgrade” to a VPS. Giant shared-hosts advertise like crazy how they offer unlimited bandwidth, but when you start using 100x more bandwidth than anyone on your server you are costing them 100x what you are paying them, every month. That’s why you will never see a webhost offering this kind of unlimited bandwidth that doesn’t require you to sign a contract giving them permission to terminate your account for any reason. Seriously read the fine print at DreamHost or anywhere else, it’s included because that is a core part of their business to terminate anyone using too much bandwidth since that is bandwidth they can’t sell to dozens of other customers. That’s why I eventually closed my account with them and moved to a legitimate company, it’s a great host for spammers though.

Back in the mid-90′s I was doing a lot of war-dialing with my modem and discovering all sorts of networks and machines, many of them were Unix and Solaris based public systems, and when I managed to gain access to the system and found myself staring at a unix shell I was very excited but also a total idiot. In those days of using the phone networks to research unknown systems it was very difficult for anyone to actually get the phone company to trace a call, so instead of what happens today where it is child’s play to trace an IP address, back then it was a very real back-and-forth battle between the system admin and whoever was gaining access to their system. Essentially, I would gain a shell or some kind of terminal, and just go at it trying to figure out what it could do, trying all kinds of commands. Inevitably this would eventually alert even the laziest admin and they would proceed to attempt to lock me out. It was great sport and extremely addictive. When my favorite system (a massive sun machine in the basement of a big library) finally locked me out and I couldn’t get back in I went to my local library and got some reading material — one of my favorites was the red hat bible. I was able to acquire my own computer and the first thing I did was install red hat linux onto it from the discs included with the book. For the next several years I was essentially offline, all we had at home was a modem and it was becoming difficult to locate any more systems in my area code.. I was into phreaking of course as well, but I never was able to make free long-distance war-dialing a reality. So I just read the books and learned what I could. I would also goto the library when I could in order to use their machines which were connected to the internet (before aol it was much different than today’s internet) and since my time was short I would download as many documents as I could so that I could read them offline. The TLDP documentation that we know today was around back then in various forms, and I read every HOWTO in the index, though not understanding half. The other big resource I found for really intense reading was the kernel documentation, which admitedly I still don’t comprehend 1/4th of.. I try and peruse all the new documents when a new kernel is released, since the kernel is where all the real action is, hence the military authoritative name, and that is how I discovered one of the coolest features of Linux that I have found. TMPFS!

TMPFS kills the RAMDISK

Ok so we all know what RAM is, it’s the memory cards that most people never see that is used by the computer to store and access data that all programs need. RAM is very expensive compared to most PC components, because it’s what makes a computer blazing fast or slow. So real quick lets look at a few (there are not many) ways that various linux hackers use RAM in non-conventional ways in the past.

Tmpfs is a file system which keeps all files in virtual memory. Everything is temporary in the sense that no files will be created on your hard drive. If you reboot, everything in tmpfs will be lost.

In contrast to RAM disks, which get allocated a fixed amount of physical RAM, tmpfs grows and shrinks to accommodate the files it contains and is able to swap unneeded pages out to swap space.

Like a ramdisk, tmpfs can use your RAM, but it can also use your swap devices for storage. And while a traditional ramdisk is a block device and requires a mkfs command of some kind before you can actually use it, tmpfs is a filesystem, not a block device; you just mount it, and it’s there. All in all, this makes tmpfs the niftiest RAM-based filesystem I’ve had the opportunity to meet.

If I had to explain tmpfs in one breath, I’d say that tmpfs is like a ramdisk, but different. Like a ramdisk, tmpfs can use your RAM, but it can also use your swap devices for storage. And while a traditional ramdisk is a block device and requires a mkfs command of some kind before you can actually use it, tmpfs is a filesystem, not a block device; you just mount it, and it’s there. All in all, this makes tmpfs the niftiest RAM-based filesystem I’ve had the opportunity to meet.


What kind of filesystem is used on your server to store all your site files? EXT4, REISERFS, EXT3, NFS, etc.. are the usual filesystems, Windows users are limited to the NTFS filesystem. A filesystem is different than a device, a device is a hard-drive disk. A filesystem is how the device is formatted to allow for file and folder structures. A hard drive is slow compared to RAM, no question about that. So what if instead of your server serving files off a hard-drive it served files stored in RAM? 30x faster thats what happens!

I just figured out how to store my cached static files created by WP-Super Cache in my server’s RAM, and the difference is unbelievable. My “AskApache Crazy Cache” plugin basically forces WP-Super Cache, Hyper Cache, etc.. to recreate a static cached file for every page on a blog. For the AskApache.com site this takes around 3 minutes to complete. Once I switched to using this new method of storing the files on RAM I am able to re-cache the entire site in about 15 seconds!!!!

tmpfs is a dynamically expandable/shrinkable ramdisk, and will
# use almost no memory if not populated with files

Tmpfs is a file system which keeps all files in virtual memory.

Everything in tmpfs is temporary in the sense that no files will be created on your hard drive. If you unmount a tmpfs instance, everything stored therein is lost.

tmpfs puts everything into the kernel internal caches and grows and shrinks to accommodate the files it contains and is able to swap unneeded pages out to swap space. It has maximum size limits which can be adjusted on the fly via ‘mount -o remount …’

If you compare it to ramfs (which was the template to create tmpfs) you gain swapping and limit checking. Another similar thing is the RAM disk (/dev/ram*), which simulates a fixed size hard disk in physical RAM, where you have to create an ordinary filesystem on top. Ramdisks cannot swap and you do not have the possibility to resize them.

Since tmpfs lives completely in the page cache and on swap, all tmpfs pages currently in memory will show up as cached. It will not show up as shared or something like that. Further on you can check the actual RAM+swap use of a tmpfs instance with df(1) and du(1).

Both tmpfs and ramfs mount will give you the power of fast reading and writing files from and to the primary memory. When you test this on a small file, you may not see a huge difference. You’ll notice the difference only when you write large amount of data to a file with some other processing overhead such as network.

TMPFS uses RAM+SWAP

TMPFS is another filesystem with uniquely cool capabilities. It stores any files contained within it on RAM and in SWAP which means your server can access any files stored on TMPFS without even having to access the disk, which according to technical stats is around 30 times faster than accessing a file off disk.

Some other cool aspects of TMPFS are that it intelligently and automatically sizes itself to be just alittle bigger then it needs to be. So when you remove files to a folder stored on a TMPFS filesystem, the TMPFS filesystem shrinks by allocating less RAM and/or SWAP. Conversely when adding files to TMPFS it grows larger. You can set the max-size and max-number-of-files as a mount option to make sure your TMPFS never uses all of the available RAM and SWAP, which would halt your server.

Swap

Find the swap size.

# free -m -t
             total       used       free     shared    buffers     cached
Mem:           458         93        364          0          0          0
-/+ buffers/cache:         93        364
Swap:          900          0        900
Total:        1358         93       1264
Adding 3004144k swap on /dev/sdb2.  Priority:-1 extents:1 across:3004144k
Adding 2096472k swap on /dev/sda3.  Priority:-2 extents:1 across:2096472k

Using TMPFS for Cache

The method here will show how to create and use a TMPFS filesystem to hold all the static files created by WP-Super Cache. These static files are served to visitors instead of loading php for every request, so by moving those static files to TMPFS your server will be able to access and start sending your site to the browser 30x faster!

The WP-Super Cache plugin stores all the static files in the wp-content/cache folder of your WordPress installation, so to enable TMPFS we simply will create a new TMPFS filesystem and mount it to the wp-content/cache folder. That makes anything in that folder (all the static files) be part of the TMPFS filesystem.

Boosting Cache with TMPFS

There are a lot of maybe new concepts surrounding TMPFS and it may seem too complicated, but the process of actually setting up a robust tmpfs to use for wp-super-cache’s cache folder is actually very simple. As long as you have shell access to your server and the permissions required (any sudo or private server should be good to go) you can set this up in a couple minutes and not really have to give it a second thought or debug anything. Here’s the process I’ve used on several client sites.

  1. Create a TMPFS Filesystem and Mount at /wp-content/cache/
  2. Restore TMPFS Cached Files across Reboots
  3. Keep a semi-current mirror of the TMPFS files on Disk


Create TMPFS at wp-content/cache

/etc/fstab

tmpfs /home/askapache/wp-content/cache tmpfs defaults,size=2g,noexec,nosuid,uid=648,gid=648,mode=1755 0 0

Restoring TMPFS across Reboots

In /etc/rc.local

ionice -c3 -n7 nice -n 19 rsync -ahv --stats --delete /_b/tmpfs/cache/ /home/askapache/wp-content/cache/ 1>/dev/null

Mirroring TMPFS to Disk

Cronjob entry

*/5 * * * * /usr/bin/ionice -c3 -n7 /bin/nice -n 19 /usr/bin/rsync -ah --stats --delete /home/askapache/wp-content/cache/ /_b/tmpfs/cache/ 1>/dev/null

/tmp, /var/run, and /var/lock

The directories /tmp, /var/run, and /var/lock contain files that are not needed across reboots. This means they are ideal candidates for tmpfs. HEre’s how to do it.

tmpfs /var/run tmpfs defaults,rw,nosuid,mode=0755 0 0
tmpfs /var/lock tmpfs defaults,rw,noexec,nosuid,nodev,mode=1777 0 0

Resize /dev/shm

You can view your current /dev/shm size with the command df -ha|grep /dev/shm then if you want to resize that use the command:

mount -t tmpfs -o remount,size-2G,rw,nosuid,nodev tmpfs /dev/shm
Secure /dev/shm:
 
Step 1: Edit your /etc/fstab:
 
nano -w /etc/fstab
 
Locate:
 
none /dev/shm tmpfs defaults,rw 0 0
 
Change it to:
 
none /dev/shm tmpfs defaults,nosuid,noexec,rw 0 0
 
Step 2: Remount /dev/shm:
 
mount -o remount /dev/shm
 
guilt makes extensive use of the '$$' shell variable for temporary
files in /tmp. This is a serious security vulnerability; on multi-user
systems it allows an attacker to clobber files with something like the
following:
 
for i in `seq 1 32768`; do
ln -sf /etc/passwd /tmp/guilt.log.$i;
done
 
(In this example, if root does e.g. 'guilt push', /etc/passwd will get
clobbered.)


Securing and Using /tmp

tmpfs mount parameters

A good way to find a good tmpfs upper-bound is to use top to monitor your system’s swap usage during peak usage periods. Then, make sure that you specify a tmpfs upper-bound that’s slightly less than the sum of all free swap and free RAM during these peak usage times.

mode=1777 sets sticky bit on directory. Only file owners can delete files in this directory.

The following parameters accept a suffix k, m or g for Ki, Mi, Gi (binary kilo, mega and giga) and can be changed on remount.

  • size: Override default maximum size of the filesystem. The size is given in bytes, and rounded down to entire pages. The default is half of the memory.The limit of allocated bytes for this tmpfs instance. The default is half of your physical RAM without swap. If you oversize your tmpfs instances the machine will deadlock since the OOM handler will not be able to free that memory.
  • nr_inodes: Set number of inodes.
  • nr_blocks: Set number of blocks.
  • mode: The permissions as an octal number
  • uid: The user id
  • gid: The group id
mount -t tmpfs -o size=10G,nr_inodes=10k,mode=700 tmpfs /mytmpfs

Will give you tmpfs instance on /mytmpfs which can allocate 10GB RAM/SWAP in 10240 inodes and it is only accessible by root.

Using tmpfs for /tmp storage

Many users find it very convenient to use tmpfs for /tmp and /var/tmp which does a number of positive things. Any temporary files are instead created in RAM not your hard-drive, which means that reading/writing/accessing those temporary files by various processes doesn’t slow down your hard-drive read/writes/accesses for your other processes. This also has a side-effect of making your hard-drive have a longer life as it reduces activity by a huge amount.

Remember that tmpfs uses both RAM and swap, so make sure your machine has a large swapfile, like gigabytes. If your tmpfs consumes all the swap and RAM then you are screwed, so make sure that you correctly set the mount options for the tmpfs so that it doesn’t do that. If your /tmp or /var/tmp gets filled with tmp files that for some reason don’t get deleted except at reboot, and your machine has a very high uptime, then you will want to run some cron jobs to periodically clean the /tmp and /var/tmp directories of older files…

Here’s an example scenario: let’s say that we have an existing filesystem mounted at /tmp. However, we decide that we’d like to start using tmpfs for /tmp storage.

with recent 2.4 kernels, you can mount your new /tmp filesystem without getting the “device is busy” error: 

mount tmpfs /tmp -t tmpfs -o size=64m

With a single command, your new tmpfs /tmp filesystem is mounted at /tmp, on top of the already-mounted partition, which can no longer be directly accessed. However, while you can’t get to the original /tmp, any processes that still have open files on this original filesystem can continue to access them. And, if you umount your tmpfs-based /tmp, your original mounted /tmp filesystem will reappear. In fact, you can mount any number of filesystems to the same mountpoint, and the mountpoint will act like a stack; unmount the current filesystem, and the last-most-recently mounted filesystem will reappear from underneath.

Bind Mounts

Using bind mounts, we can mount all, or even part of an already-mounted filesystem to another location, and have the filesystem accessible from both mountpoints at the same time!

For example, you can use bind mounts to mount your existing /tmp filesystem to /sites/askapache.com/tmp, as follows:

mount --bind /tmp /sites/askapache.com/tmp

Now, if you look inside /sites/askapache.com/tmp, you’ll see your /tmp filesystem and all its files. And if you modify a file on your /tmp filesystem, you’ll see the modifications in /sites/askapache.com/tmp as well. This is because they are one and the same filesystem; the kernel is simply mapping the filesystem to two different mountpoints for us.

Note that when you mount a filesystem somewhere else, any filesystems that were mounted to mountpoints inside the bind-mounted filesystem will not be moved along. In other words, if you have /tmp/cache on a separate filesystem, the bind mount we performed above will leave /sites/askapache.com/tmp/cache empty. You’ll need an additional bind mount command to allow you to browse the contents of /tmp/cache at /sites/askapache.com/tmp/cache:

mount --bind /tmp/cache /sites/askapache.com/tmp/cache

Bind mounting and /dev/shm

glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for POSIX shared memory (shm_open, shm_unlink). Adding the following line to /etc/fstab should take care of this:

tmpfs  /dev/shm  tmpfs  defaults  0 0

Many systems by default have a tmpfs filesystem mounted at /dev/shm that defaults to a size of half of your physical RAM without swap. Say you decide that you’d like to start using tmpfs for /tmp, which currently lives on your root filesystem. Rather than mounting a new tmpfs filesystem to /tmp (which is possible), you may decide that you’d like the new /tmp to share the currently mounted /dev/shm filesystem. However, while you could bind mount /dev/shm to /tmp and be done with it, your /dev/shm contains some directories that you don’t want to appear in /tmp. So, what do you do? How about this:

mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
mount --bind /dev/shm/tmp /tmp

In this example, we first create a /dev/shm/tmp directory and then give it 1777 perms, the proper permissions for /tmp. Now that our directory is ready, we can mount /dev/shm/tmp, and only /dev/shm/tmp to /tmp. So, while /tmp/foo would map to /dev/shm/tmp/foo, there’s no way for you to access the /dev/shm/bar file from /tmp.


/etc/default/tmpfs WorkAround

$ cat /etc/default/tmpfs
# SHM_SIZE sets the maximum size (in bytes) that the /dev/shm tmpfs can use.
# If this is not set then the size defaults to the value of TMPFS_SIZE
# if that is set; otherwise to the kernel's default.
#
# The size will be rounded down to a multiple of the page size, 4096 bytes.
SHM_SIZE=524288000
# TMPFS_SIZE sets the max size that /dev/shm can use.  By default, the
# kernel sets this upper limit to half of available memory.
TMPFS_SIZE=524288000

RSYNC vs. CP

rsync [options]  SRC DEST
rsync -av --delete --stats /home/wincom/public_html/wp-content/cache/ /backups/tmp-mnt/cache/
-a, --archive               archive mode; same as -rlptgoD (no -H)
-r, --recursive             recurse into directories
-l, --links                 copy symlinks as symlinks
-p, --perms                 preserve permissions
-t, --times                 preserve times
-g, --group                 preserve group
-o, --owner                 preserve owner (super-user only)
-D                          same as --devices --specials
    --devices               preserve device files (super-user only)
    --specials              preserve special files
 -h, --human-readable        output numbers in a human-readable format
     --progress              show progress during transfer

Mount Options

The following options apply to any file system that is being mounted (but not every file system actually honors them)

  • async All I/O to the file system should be done asynchronously.
  • atime Update inode access time for each access. This is the default.
  • auto Can be mounted with the -a option.
  • defaults Use default options: rw, suid, dev, exec, auto, nouser, and async.
  • dev Interpret character or block special devices on the file system.
  • exec Permit execution of binaries.
  • group Allow an ordinary (i.e., non-root) user to mount the file system if one of his groups matches the group of the device. This option implies the options nosuid and nodev (unless overridden by subsequent options, as in the option line group,dev,suid).
  • mand Allow mandatory locks on this filesystem. See fcntl(2).
  • _netdev The filesystem resides on a device that requires network access (used to prevent the system from attempting to mount these filesystems until the network has been enabled on the system).
  • noatime Do not update inode access times on this file system (e.g, for faster access on the news spool to speed up news servers).
  • nodiratime Do not update directory inode access times on this filesystem.
  • noauto Can only be mounted explicitly (i.e., the -a option will not cause the file system to be mounted).
  • nodev Do not interpret character or block special devices on the file system.
  • noexec Do not allow direct execution of any binaries on the mounted file system. (Until recently it was possible to run binaries anyway using a command like /lib/ld*.so /mnt/binary. This trick fails since Linux 2.4.25 / 2.6.0.)
  • nomand Do not allow mandatory locks on this filesystem.
  • nosuid Do not allow set-user-identifier or set-group-identifier bits to take effect. (This seems safe, but is in fact rather unsafe if you have suidperl(1) installed.)
  • nouser Forbid an ordinary (i.e., non-root) user to mount the file system. This is the default.
  • owner Allow an ordinary (i.e., non-root) user to mount the file system if he is the owner of the device. This option implies the options nosuid and nodev (unless overridden by subsequent options, as in the option line owner,dev,suid).
  • remount Attempt to remount an already-mounted file system. This is commonly used to change the mount flags for a file system, especially to make a readonly file system writeable. It does not change device or mount point.
  • ro Mount the file system read-only.
  • _rnetdev Like _netdev, except “fsck -a” checks this filesystem during rc.sysinit.
  • rw Mount the file system read-write.
  • suid Allow set-user-identifier or set-group-identifier bits to take effect.
  • sync All I/O to the file system should be done synchronously. In case of media with limited number of write cycles (e.g. some flash drives) “sync” may cause life-cycle shortening.
  • dirsync All directory updates within the file system should be done synchronously. This affects the following system calls: creat, link, unlink, symlink, mkdir, rmdir, mknod and rename.
  • user Allow an ordinary user to mount the file system. The name of the mounting user is written to mtab so that he can unmount the file system again. This option implies the options noexec, nosuid, and nodev (unless overridden by subsequent options, as in the option line user,exec,dev,suid).
  • users Allow every user to mount and unmount the file system. This option implies the options noexec, nosuid, and nodev (unless overridden by subsequent options, as in the option line users,exec,dev,suid).

Filesystems

You can find out what is filesystems are in place by using one of the following linux commands:

cat /etc/fstab
cat /etc/mtab
cat /proc/mounts
df -a

/etc/fstab

       /etc/fstab        file system table
       /etc/mtab         table of mounted file systems
       /etc/mtab~        lock file
       /etc/mtab.tmp     temporary file
       /etc/filesystems  a list of filesystem types to try

From /etc/mtab

none /tmp tmpfs size=128m,mode=1777 0 0

From /proc/mounts

none /tmp tmpfs rw,nodev,relatime,size=131072k 0 0


/etc/fstab

It is possible that files /etc/mtab and /proc/mounts don’t match. The first file is based only on the mount command options, but the content of the second file also depends on the kernel and others settings (e.g. remote NFS server. In particular case the mount command may reports unreliable information about a NFS mount point and the /proc/mounts file usually contains more reliable information.)

This file is used in three ways:

  1. The following command (usually given in a bootscript) causes all file systems mentioned in fstab (of the proper type and/or having or not having the proper options) to be mounted as indicated, except for those whose line contains the noauto keyword. Adding the -F option will make mount fork, so that the filesystems are mounted simultaneously. 
    mount -a [-t type] [-O optlist]
  2. When mounting a file system mentioned in fstab, it suffices to give only the device, or only the mount point.
  3. Normally, only the superuser can mount file systems. However, when fstab contains the user option on a line, anybody can mount the corresponding system.

The programs mount and umount maintain a list of currently mounted file systems in the file /etc/mtab.

Only the user that mounted a filesystem can unmount it again. If any user should be able to unmount, then use users instead of user in the fstab line. The owner option is similar to the user option, with the restriction that the user must be the owner of the special file. The group option is similar, with the restriction that the user must be member of the group of the special file.

The order of records in fstab is important because fsck(8), mount(8), and umount(8) sequentially iterate through fstab doing their thing.

The first field, (fs_spec)

Describes the block special device or remote filesystem to be mounted. For ordinary mounts it will hold (a link to) a block special device node (as created by mknod(8)) for the device to be mounted, like ‘/dev/cdrom’ or ‘/dev/sdb7’. For NFS mounts one will have :

, e.g., ‘knuth.aeb.nl:/’. For procfs, use ‘proc’.

Instead of giving the device explicitly, one may indicate the (ext2 or xfs) filesystem that is to be mounted by its UUID or volume label (cf. e2label(8) or xfs_admin(8)), writing LABEL=

The second field, (fs_file)

Describes the mount point for the filesystem. For swap partitions, this field should be specified as ‘none’. If the name of the mount point contains spaces these can be escaped as ‘\040’.

The third field, (fs_vfstype), describes the type of the filesystem. Linux supports lots of filesystem types, such as adfs, affs, autofs, coda, coherent, cramfs, devpts, efs, ext2, ext3, hfs, hpfs, iso9660, jfs, minix, msdos, ncpfs, nfs, ntfs, proc, qnx4, reiserfs, romfs, smbfs, sysv, tmpfs, udf, ufs, umsdos, vfat, xenix, xfs, and possibly others. For more details, see mount(8). For the filesystems currently supported by the running kernel, see /proc/filesystems. An entry swap denotes a file or partition to be used for swapping, cf. swapon(8). An entry ignore causes the line to be ignored. This is useful to show disk partitions which are currently unused.

The fourth field, (fs_mntops)

Describes the mount options associated with the filesystem. It is formatted as a comma separated list of options. It contains at least the type of mount plus any additional options appropriate to the filesystem type. For documentation on the available options for non-nfs file systems, see mount(8). For documentation on all nfs-specific options have a look at nfs(5).

Common for all types of file system are the options:

  • noauto: (do not mount when “mount -a” is given, e.g., at boot time)
  • user: (allow a user to mount)
  • owner: (allow device owner to mount)
  • pamconsole: (allow a user at the console to mount)
  • comment: (e.g., for use by fstab-maintaining programs).

The fifth field, (fs_freq)

Used for these filesystems by the dump(8) command to determine which filesystems need to be dumped. If the fifth field is not present, a value of zero is returned and dump will assume that the filesystem does not need to be dumped.

The sixth field, (fs_passno)

Used by the fsck(8) program to determine the order in which filesystem checks are done at reboot time. The root filesystem should be specified with a fs_passno of 1, and other filesystems should have a fs_passno of 2. Filesystems within a drive will be checked sequentially, but filesystems on different drives will be checked at the same time to utilize parallelism available in the hardware. If the sixth field is not present or zero, a value of zero is returned and fsck will assume that the filesystem does not need to be checked.

More Reading

Experiment: MySQL tmpdir on tmpfs

In MySQL, the tmpdir path is mainly used for disk-based sorts (if the sort_buffer_size is not enough) and disk-based temp tables. The latter cannot always be avoided even if you made tmp_table_size and max_heap_table_size quite large, since MEMORY tables don’t support TEXT/BLOB type columns, and also since you just really don’t want to run the risk of exceeding available memory by setting these things too large.


Use tmpfs for MySQL

--tmpdir=path, -t path

The path of the directory to use for creating temporary files. It might be useful if your default /tmp directory resides on a partition that is too small to hold temporary tables. Starting from MySQL 4.1.0, this option accepts several paths that are used in round-robin fashion. Paths should be separated by colon characters (“:”) on Unix and semicolon characters (“;”) on Windows, NetWare, and OS/2. If the MySQL server is acting as a replication slave, you should not set –tmpdir to point to a directory on a memory-based file system or to a directory that is cleared when the server host restarts. For more information about the storage location of temporary files, see Section A.1.4.4, “Where MySQL Stores Temporary Files”. A replication slave needs some of its temporary files to survive a machine restart so that it can replicate temporary tables or LOAD DATA INFILE operations. If files in the temporary file directory are lost when the server restarts, replication fails.

On Unix, MySQL uses the value of the TMPDIR environment variable as the path name of the directory in which to store temporary files. If TMPDIR is not set, MySQL uses the system default, which is usually /tmp, /var/tmp, or /usr/tmp.

If the file system containing your temporary file directory is too small, you can use the –tmpdir option to mysqld to specify a directory in a file system where you have enough space.

Starting from MySQL 4.1, the –tmpdir option can be set to a list of several paths that are used in round-robin fashion. Paths should be separated by colon characters (“:”) on Unix and semicolon characters (“;”) on Windows, NetWare, and OS/2.
Note

To spread the load effectively, these paths should be located on different physical disks, not different partitions of the same disk.

If the MySQL server is acting as a replication slave, you should not set –tmpdir to point to a directory on a memory-based file system or to a directory that is cleared when the server host restarts. A replication slave needs some of its temporary files to survive a machine restart so that it can replicate temporary tables or LOAD DATA INFILE operations. If files in the temporary file directory are lost when the server restarts, replication fails.

MySQL creates all temporary files as hidden files. This ensures that the temporary files are removed if mysqld is terminated. The disadvantage of using hidden files is that you do not see a big temporary file that fills up the file system in which the temporary file directory is located.


Shell Script for Firefox tmpfs

#!/bin/bash
### Bind temporary directories to /dev/shm ###
# I do this instead of mounting tmpfs on the #
# directories, so less memory gets wasted.   #
##############################################
mkdir /dev/shm/{tmp,lock}
mount --bind /dev/shm/tmp /tmp
mount --bind /dev/shm/tmp /var/tmp
mount --bind /dev/shm/lock /var/lock
chmod 1777 /dev/shm/{tmp,lock}

Hey! You made it!@ at least to the bottom of the page.. I still have to finish this article, so check back in a few months.

30x Faster WP-Super Cache Site Speed originally appeared on AskApache.com

]]> http://www.askapache.com/web-hosting/super-speed-secrets.html/feed/ 7 PortaPutty Auto-Reconnecting SSH Tunnels on an Encrypted TrueCrypt Portable USB Key w GPG http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html#comments Tue, 23 Feb 2010 10:11:11 +0000 AskApache http://askapache.com/?p=4001 Ok I just came back up to write the intro.. I'm trying to keep it short to avoid getting bogged down by the coolness of each step. Here is what goes on. When I logon to my XP machine at work, I bring my usb key and plug it in first. On logging a window pops up first and it's a password prompt to mount my encrypted drive leonardo. It also checks a keyfile that is located on my usb key, but all I do now is type in my password. That causes my encrypted folder to be accessible to me like a normal drive, and it autoruns a startup batch file.

The batch file causes Portable versions of Firefox (all my bookmarks, my settings) to load, and launches Portable Mozilla Thunderbird (IMAP makes this work well), which is my favorite program (great GPG features and open-source!). Also Some Adobe CS4 software is loaded from the hard drive, like DreamWeaver. In the background, a service we created executes a PortaPuttY plink command to create forwarded tunnels from various remote servers and accounts, all using key-based encryption. These tunnels are automatically reconnected if they are disconnected, meaning you can use a socks 5 if you want or even better!

Part 1 of 5

]]> Ok I just came back up to write the intro.. I’m trying to keep it short to avoid getting bogged down by the coolness of each step. Here is what goes on. When I logon to my XP machine at work, I bring my usb key and plug it in first. On logging a window pops up first and it’s a password prompt to mount my encrypted drive leonardo. It also checks a keyfile that is located on my usb key, but all I do now is type in my password. That causes my encrypted folder to be accessible to me like a normal drive, and it autoruns a startup batch file. The batch file causes Portable versions of Firefox (all my bookmarks, my settings) to load, and launches Portable Mozilla Thunderbird (IMAP makes this work well), which is my favorite program (great GPG features and open-source!). Also Some Adobe CS4 software is loaded from the hard drive, like DreamWeaver.

The batch file also runs PortaPuttY plink to create forwarded tunnels from various remote servers and accounts, all using key-based encryption. This includes dynamic SOCKS 4/5 tunnels, VPN tun device tunnels, and of course the basic SSH port forwarding tunnels that are so powerful. These tunnels are automatically reconnected if they are disconnected, using simple windows builtin command-line tools. And believe me it was not easy to figure out how to make this all work using plink ( essentially the same as putty minus the gui ), I literally had to use almost all of my Windows kung fu to finally end up with this.

Using MyEnTunnel

Initially I was using the MyEnTunnel program combined with a custom windows batch install script I wrote to handle the tunnels.

The tunnels are very important to making things easy while improving security. It’s not easy to understand at first, but basically it means you can now connect to ANY IP address:port as if you were on that very machine connecting to localhost, like if you pinged yourself!. The result is any traffic you want is now encrypted, and you can set up your servers to only accept connections from localhost, which could save you tons of memory, bandwidth, and security attack vectors to think about. So I configure everything to use these tunnels as proxies, like Mozilla Thunderbird and Chrome, Firefox, Pidgeon, all portable versions and running from my encrypted usb drive.

This means you can walk into my house with that usb key, plug into any computer here, and surf the web/check your emails all across SSH… I know for a fact I wouldn’t be able to snoop that traffic! There is a lot of exciting things going on around here, new servers and all.. Its going to take a couple more posts for me to finish this up, enjoy the article and comment.

Buy a couple USB Mini Drives

The first thing to do, is purchase a USB thumb drive.. My favorite store, TigerDirect.com, has over 104 tiny usb drives for under $24.. I’ve used them since the late 90′s.

I bought some 4GB PNY’s the size of a fingernail at a gas station and they are amazing, way faster than say a dvd drive. Just try to do some research of the differences between the 16GB vs the $4 1GB drives.. You want speed because the whole drive will be encrypted. If you can afford the super excellent and crazy fast ones, hey send me one! Buying cheap means you can buy 3 or 4 so you can always have backups. This device will make you Internationally mobile, untethered from a box, maybe getting some work done at a cafe in Florenze, or at a beach hotel in Miami. Keep dreaming, but that is more possible with a better organized system.

Backup the USB Drive

You only need to know 1 way that works, there are several. The way I do backups is to copy the entire disk image of the usb, that way I can always access it in case of usb key failure, which does happen. Free software like CloneZilla live CD with its crazy cluster computing power, or Self Image, which is free for both linux and Windows. And you could never go wrong with GHOST, one of the first to make mega bucks in the market.. it’s some seriously impressive software but not open-source. Even easier for some is to just set a cron job for dd to pipe the entire drive image to a remote computer using netcat, or sshfs, or curlftpfs, or just simple ssh like below. Once setup (without stupid, bulky, dangerous software), the files on your encrypted usb don’t change often, otherwise I would want to sync a backup to happen automatically every X number of logins or days (test logfile time in bash_profile?)..

SSH Back-ups To Remote Server

Files and data on your drives slow it down tremendously, meaning a web server storing backups locally is slower than one storing them externally.

Notice how much safer this command is by optimizing both the CPU and DISK I/O.. Though it’s much smarter to create a new separate ssh user, one with no shell and a passwordless safer key-based encryption. Then in your /etc/security/limits.conf file or your initscript you can cause that user to have nice -19 and ionice -c2 -n7 priority set all the time automatically, since sshd, compression, and disk writing are this accounts only job. turboslow is an alias defined in a ssh_config file so you don’t have to type the host, port, and settings each time.

#
# much better ways to do this on google!!!!!!!
#
ionice -c2 -n7 nice -n 19 dd if=/dev/sdb2 bs=1k conv=sync,noerror | gzip -c | ssh turboslow "dd of=sdb2.gz bs=1k"

Note that you may decide it would be better to configure the ssh connection to a less CPU intensive algorithm, perhaps even protocol 1 and DES. That’s perfectly alright, but the tradeoff is that the encryption can be broken much quicker, and so you would have to implement a cron job to create new keys on both ends of the tunnel every few hours.. It’s really not a big deal to setup, kind of sweet way to use key-based encryption. Also, important files ( those containing passwords, any database ) are encrypted before transport using private GPG keys, which don’t need to be changed. The other thing to think about too is only letting your main PC send/write on the backup host, so the backup host is only authorized to rx and can never login back to yours.

Hey! the Internet is a dangerous place you better believe it! And it’s only going to get more interesting with cloud computing’s breakthrough’s… More people who know they’re way around… I can always use an extra server, I’d love to expand my network another node without having to pay for it (free cloud computing?), so make sure your servers are locked up strenuously. Not super perfect, just a little unique or creative in your defense to avoid any coming super-worm’s that may be employing vast arsenals of the deadliest attack-engines like metasploit.. Scarry rumors.

Compression Speeds: PBZip2, Rzip, Lzop, Gzip

Probably the fastest is to use rsync over ssh, which is what I’m doing, since the algorithms used by rsync are much faster and safer. Rsync also lets you specify a compression program, so depending on your machine you will want pbzip2 (for multi processors) or rzip which are the 2 fastest I know of, though I have had some reliability issues with rzip for gigabyte transfers. Pbzip2 is amazing, blew me away the first time being 8x faster (8 CPUs) then anything. You can get it and compile a static binary for your thumb drive if want at Parallel BZIP2 (PBZIP2). Heavy code, re: this note by Jeff Gilchrist

NOTE: If you are looking for a parallel BZIP2 that works on cluster machines, you should check out MPIBZIP2 which was designed for a distributed-memory message-passing architecture.

tar cpf "$G" --use-compress-prog=pbzip2 ./

Parallel BZIP2 (PBZIP2)

Benchmarking for Performance

Finally a couple tips, you should get an idea what the device can do, format it a few times for linux and test it on windows, and vice versa.. Some drives are too small or too old and can only support fat32 filesystems on winblows, you DO NOT want fat32 because this drive is going to be 100% encrypted and then 100% transparently decrypted as you use it, 

# note this is 512MB
dd if=/dev/sda1 of=/dev/null bs=512 count=1000000
512000000 bytes (512 MB) copied, 5.16588 s, 99.1 MB/s

Part II: Encrypted AutoRunning USB Key with TrueCrypt

Now this section anyone can do, it’s so easy on Windows. What I’m going to show you how to do is get setup the right way super-fast. There are many ways to use TrueCrypt, it’s one of the nicest built software programs’s I’ve ever used… Sadly, it is not licensed open-source, and that is often a deal-breaker for security-conscious folks or anti-pirate anarchists. From the very helpful TrueCrypt web site:

Further Reading

The real fun doesn’t start till all the automation starts, automating all of that from a couple batch files I wrote, one click setup. Kind of like building your own knoppix for when you have to use Windows. To begin this tutorial, setup a truecrypt traveller setup on your usb and also install the portaputty package onto the usb. You do this by creating a 3GB or whatever file on the usb and then mounting that file like you would mount an iso file. I will show the Windows Batch file I use and the tricks with Windows Volume names and how to consistently make it all work. Then we will setup MyEnTunnel with a customized batch file that forces all puttys to use portaputty (sweet hack stolen from sysinternals pagedefrag tool).Stay Tuned!

PortaPutty Auto-Reconnecting SSH Tunnels on an Encrypted TrueCrypt Portable USB Key w GPG originally appeared on AskApache.com

]]> http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html/feed/ 7 Optimize a Website for Speed, Security, and Easy Management http://www.askapache.com/htaccess/optimize-website-files-cache-security.html http://www.askapache.com/htaccess/optimize-website-files-cache-security.html#comments Fri, 19 Feb 2010 00:45:26 +0000 AskApache http://www.askapache.com/?p=1380 Learn how to setup, configure, secure, optimize, and create a low-maintenance website the AskApache way. I'm piecing together all the hacks, tricks, methods, and ideas discussed throughout this blog and all across Netdom and glueing them all together to show you how to have the most optimized, crazy fastest, and best website setup I can think of.

]]> optimized server setupOver the past 10 or so years I’ve been directly or indirectly involved in configuring/administrating/hacking thousands of websites, and I realized today that I’ve actually learned quite a bit about how to really make them work hard for me, instead of the other way around. It came as a mild shock to think of where I was back then vs. now because the improvements and optimizations are hundreds of smaller improvements, but taken together, the optimization hacks I’ve found through trial and error and much reading are as Donald would say, YOOUUGE compared to a basic website setup.

I use this awesome skeleton setup for all my high-paying clients sorry poor people! and also of course on this blog, which I use as a bleeding-edge dev server for my crazy testing. So realize that I’m already past this setup and using it to do cooler stuff. In order for you to use these more advanced ideas, you first need to get up to speed on what I’m doing so you know what I’m talking about. This article tries to help you accomplish that… remains to be seen.

An Optimized Website, The Real Deal

This first article is to give you some ideas and get you thinking and reading before the first article in this series comes out. This series details how to setup, configure, secure, optimize, and manage a website the best possible way I can come up with. It pieces together all the AskApache hacks and tricks and uses methods and ideas discussed all over this blog and all over the net and glues them all together to show you how to have the most optimized, fastest, best website setup I can think of.

Knowing the why and how behind the operation of a Web Server allows us to optimize that operation. For this example we will be creating the website www.askapache.com, which will be running WordPress and php. We will also set up static.askapache.com to serve all of our sites uploads, images, css and javascript files, flash files, etc. with advanced caching and security using Apache Server .htaccess files. So lets get started and take a look at this site structure for a moment.

/home/askapache.com
|-- /home/askapache.com/backups/
|-- /home/askapache.com/public_html/
|-- /home/askapache.com/inc/
|-- /home/askapache.com/logs/
|-- /home/askapache.com/static/
|-- /home/askapache.com/tmp/
|-- /home/askapache.com/.htpasswd-basic
`-- /home/askapache.com/.htpasswd-digest
  • /backups/ – For encrypted backups of WordPress database and site files. And any other backups.
  • /public_html/ – The document root for www.askapache.com
  • /inc/ – Folder to keep your php include files for extra security and easy management.
  • /logs/ – Save your php, apache, and other logs here or create symlinks to them.
  • /static/ – The document root for static.askapache.com
  • /tmp/ – Only need this if your host doesn’t already have a /tmp folder

Strong Security, Top to Bottom

Optimize a Website for Speed, Security, and Easy ManagementSimply by implementing correct access permissions, file permissions, password protection and segmenting various folders and services we are already ahead of the game. I’ve always taken security extremely seriously, so you can benefit from alot of the simple solutions I’m recommending for a really locked down site.

Indeed, security is a major part of every step of this setup process, as security concerns are what drives a lot of the motivations I have for coming up with this setup in the first place. We will be doing very simple but very effective site security like the following items, which is a short list compared to everything we will be doing.

  • Fixing file permissions automatically
  • Searching for modified files on the server
  • Encrypting your backups
  • Get alerted to breakin attempts
  • Block tons of bad clients
  • Disallowing cgi scripts or any other handlers, just serve files.
  • Configuring PHP
  • Password Protection for certain areas

Ready for Warfare?

My past work for an Internet Service Provider, followed by 4 years of auditing the security of organizations external/internal networks has given me a fresh perspective on website security, and I think it allows me to see what would really be effective at preventing and killing attacks. In fact just last night I was once again doing some research into some off-the-wall security topics, and I discovered a new defense method that I will be writing about very soon. I believe that this new method, could be quickly adopted and implemented by hosting providers and software developers, which would result in us finally taking the Internet back from all those zombies and robots. This method will be discussed in great detail soon, and will be a core part of this site setups security and optimization.

Built to Bleed Speed

Optimize a Website for Speed, Security, and Easy ManagementServe’s files as fast and efficiently as possible using advanced caching, HTTP Protocols, php/server configurations.

Many of the articles and research on this blog is about improving the speed and efficiency of your website. In fact that is why I am helping develop open-source software to block spammers from WordPress blogs… not because I’m bothered by the spam, but because they make the net slow! So lets look at some of the ideas we’ll be implementing.

Many techniques I’ve been using and tweaking for several years, and recently many of them were included in the high-performance websites list. Of course we will be taking a look at this list in practical terms, meaning almost all of it, the caching, compression, etc., will be automated in keeping with our “comfort” goal, which is to say we want to make the Web Developer and Server Admin’s lives as easy and comfy as possible. After all, we do the work right?

  1. Reduce HTTP requests – Reducing 304 requests with Cache-Control Headers
  2. Use a customized php.ini – Creating and using a custom PHP.ini
  3. Add an Expires header – Caching with mod_expires on Apache
  4. Gzip components
  5. Make CSS and unobtrusive Javascript as external files not inline
  6. Reduce DNS lookups – Use Static IP address, use a subdomain with static IP address for static content.
  7. Minimize Javascript – Refactor the code, compress with dojo
  8. Avoid external redirects – Use internal redirection with mod_rewrite, The correct way to redirect with 301
  9. Turn off ETags – Prevent Caching with htaccess
  10. Make AJAX cacheable and small

AskApache.com, Fastest Site Ever!

Ok it might not be the #1, but surely the top 10.. ;)

I’m very proud of the performance I am able to achieve on this site. Very proud. I started looking for ways to improve the wp-cache and wp-super-cache WordPress plugins, and came up with hacks for both of them.. but they still didn’t do what I wanted so I started from scratch and wrote my own caching plugin.

With much more advanced caching options and unquestionably higher performance and lower time usage on the machine. I’m hesitant to release it to the public until I get faded on it.. I just really love it.. it has been running my site for several months now and I keep finding ways to improve it.. Stay tuned.

One feature it has is the ability to allow negotiation of a resource between apache and the client. Think almost transparent mutli-lingual blogs, mutliple formats per document (look at the rdf for this page for an example*). But that plugin is the future and this is the present.. so back to it we go.

Pamper the Webmaster with Extreme Comfort

Optimize a Website for Speed, Security, and Easy ManagementThis section alone would make this setup appealing. I have developed all types of techniques and methods to make my life as easy as possible. I could literally DIE right now and this blog would continue to run and operate for years on its own. The general philosophy that I have used to get to where I can goto the beach with my laptop and do all this crazy stuff is the idea of perfection. That may sound a little put-offish, but it basically means I will focus in on one very specific area for improvement or research and just get sick with it. Most of this blogs articles are enlightening examples of this in action. I will take a relatively unknown or unused piece of code or software and experiment with it until I feel I have it down, then I move on to the next item of never-ending research. Mostly I think this is just plain habit from when I was studying security. I’m much better at this then that :)

Apache ErrorDocuments

The 57 HTTP Status Codes and Apache ErrorDocuments article is a prime example. I was simply searching for an authoritative list of HTTP status codes, an issue not many web people find worth their time, and that search led to some of the most useful stuff I’ve found about the Web

This “Comfort” article will include multi-language, intelligent, and optimized error documents for handling any type of HTTP error with class and allow us to stop spammers, save bandwidth, redirect correctly, etc.. You will probably be surprised at all the uses an Apache ErrorDocument can have.. It IS one of the foundations of the HTTP-based Net after all.

Emphasis on Easy Upgrades

The whole setup is geared to make hassle-free WordPress/PHP/application upgrades possible by keeping different types of files in separate places, keeping backups, other misc tricks and since all of these files are in /home/askapache.com, your FTP connection can still access every file easily. Sometimes security and optimizing your server can lead to it being more of a pain to do updates, backups, and general maintenance. This article tries to overwhelm the balance with a trifecta of goals.

Move to a new host? Ok!

Comfort to me also means being able to pack up the whole website and database and move to another web host in under an hour. I can move the whole AskApache site to one of several other hosting providers accounts I have in about 30minutes. If this was a clients site or I was getting paid more, I’d also be focused on round-robin DNS technology, balance-load setups, and just go crazy making it fast.

Staying Online, Improving Uptime

Ever since I started sharing information and software to stop all these resource hogging zombies attacking everything I’ve been attacked several times. Normally I get over 10K exploit attempts or requests per day, which I pretty much block 100%. But a few times they’ve actually tried to DDOS me off the net in a distributed attack. I have implemented several “poor mans” techniques to put up your best effort at surviving, which I did. Basically you want to configure your server to KILL connections just as fast as possible and prevent your server resources from skyrocketing and surpassing your quotas. A skilled attacker could easily shut you down even without the use of a widespread botnet if they are clever, which could be devastating to your small blog or site if it goes down at a crucial instant.

Organization with Templates and Systems

I used to work with a guy who did alot of the coldfusion programming for us, and I used to cringe every time I was called in to upgrade a site or do a re-design. Files and folders EVERYWHERE! Literally images in every folder, multiple index.html, index1.html, index-old.html, and on and on it went.. It would take me hours just to reverse-engineer the site enough so I could modify files on it without having some unkown consequence happen.

Do You Have a Cluttered Desktop?

Everyone has this problem, what I do all the time is just grab everything on my desktop and put it in a folder named with the date. Then the process repeats itself and invariably a few months later I’m looking at a cluttered screen again.

This absolutely is the worst thing that can happen to a website, worst for security, comfort for webmaster, and speed. So this setup addresses that issue completely heads on. With all the different pages, tools, and resources available on this blog, I can almost promise you that my site has less files than yours. No small feat to be sure, but worth every second I spent researching how to do it now that its on and popping.

What’s a Website really?

All hosts are different, but any host worth their salt is running some kind of BSD/Linux operating system, and that is good news because those operating systems all use very similarly excellent file/folder structures with huge organization systems. If your web hosting provider is running on a Windows based operating system or other locked/proprietary software than this article is not for you and I would recommend switching hosts to a BSD/Linux open-source operating system.

Listening for Requests with Web Hosting and DNS

First you set your website up so it can start serving.

  1. You buy your domain name, which just gives you the right to use it.
  2. You pay your webhost for an account on their machine running a Server connected to the Net via a fast connection link.
  3. You pay a DNS provider to redirect requests for your domain name to be sent to your webhosts machine running the server.

Sub-Domain for Serving Assets

This is a very cool method I’ve been using more and more frequently because it makes updates, upgrades, and changes so much easier to manage. And segmenting various parts of the site is smart security, and even smarter in the way of speeding up a website and keeping your servers running mean and lean.

Full Site Structure Expanded

/home/askapache.com
|-- /home/askapache.com/backups/
|-- /home/askapache.com/public_html/
|   |-- /home/askapache.com/public_html/about/
|   |-- /home/askapache.com/public_html/admin/
|   |-- /home/askapache.com/public_html/cgi-bin/
|   |-- /home/askapache.com/public_html/.htaccess
|   |-- /home/askapache.com/public_html/index.php
|   `-- /home/askapache.com/public_html/robots.txt
|-- /home/askapache.com/inc/
|   |-- /home/askapache.com/inc/config.inc.php
|   `-- /home/askapache.com/inc/settings.inc.php
|-- /home/askapache.com/logs/
|   |-- /home/askapache.com/logs/access.log
|   |-- /home/askapache.com/logs/error.log
|   |-- /home/askapache.com/logs/logins.log
|   |-- /home/askapache.com/logs/modsec_audit.log
|   |-- /home/askapache.com/logs/modsec_debug.log
|   `-- /home/askapache.com/logs/php_error.log
|-- /home/askapache.com/static/
|   |-- /home/askapache.com/static/css/
|   |-- /home/askapache.com/static/flv/
|   |-- /home/askapache.com/static/img/
|   |-- /home/askapache.com/static/js/
|   |-- /home/askapache.com/static/mp3/
|   |-- /home/askapache.com/static/pdf/
|   |-- /home/askapache.com/static/swf/
|   |-- /home/askapache.com/static/.htaccess
|   |-- /home/askapache.com/static/index.html
|   `-- /home/askapache.com/static/robots.txt
|-- /home/askapache.com/tmp/
|-- /home/askapache.com/.htpasswd-basic
`-- /home/askapache.com/.htpasswd-digest

Full Expanded Structure

/home/askapache.com
|-- /home/askapache.com/backups/
|-- /home/askapache.com/public_html/
|   |-- /home/askapache.com/public_html/about/
|   |   `-- /home/askapache.com/public_html/about/index.html
|   |-- /home/askapache.com/public_html/admin/
|   |   |-- /home/askapache.com/public_html/admin/.htaccess
|   |   `-- /home/askapache.com/public_html/admin/index.html
|   |-- /home/askapache.com/public_html/cgi-bin/
|   |   |-- /home/askapache.com/public_html/cgi-bin/bin/
|   |   |   |-- /home/askapache.com/public_html/cgi-bin/bin/.htaccess
|   |   |   |-- /home/askapache.com/public_html/cgi-bin/bin/php.cgi*
|   |   |   |-- /home/askapache.com/public_html/cgi-bin/bin/php.ini
|   |   |   |-- /home/askapache.com/public_html/cgi-bin/bin/php4.cgi*
|   |   |   `-- /home/askapache.com/public_html/cgi-bin/bin/php5.cgi*
|   |   |-- /home/askapache.com/public_html/cgi-bin/private/
|   |   |   |-- /home/askapache.com/public_html/cgi-bin/private/.htaccess
|   |   |   |-- /home/askapache.com/public_html/cgi-bin/private/debug.php
|   |   |   `-- /home/askapache.com/public_html/cgi-bin/private/stats.php
|   |   |-- /home/askapache.com/public_html/cgi-bin/.htaccess
|   |   |-- /home/askapache.com/public_html/cgi-bin/login.php
|   |   |-- /home/askapache.com/public_html/cgi-bin/printenv.cgi*
 
|   |   `-- /home/askapache.com/public_html/cgi-bin/redir.cgi*
|   |-- /home/askapache.com/public_html/.htaccess
|   |-- /home/askapache.com/public_html/index.php
|   `-- /home/askapache.com/public_html/robots.txt
|-- /home/askapache.com/inc/
|   |-- /home/askapache.com/inc/config.php
|   `-- /home/askapache.com/inc/functions.php
|-- /home/askapache.com/logs/
|   |-- /home/askapache.com/logs/access.log
|   |-- /home/askapache.com/logs/error.log
|   |-- /home/askapache.com/logs/logins.log
|   |-- /home/askapache.com/logs/modsec_audit.log
|   |-- /home/askapache.com/logs/modsec_debug.log
|   `-- /home/askapache.com/logs/php_error.log
|-- /home/askapache.com/static/
|   |-- /home/askapache.com/static/css/
|   |   `-- /home/askapache.com/static/css/apache.css
|   |-- /home/askapache.com/static/flv/
|   |   `-- /home/askapache.com/static/flv/apache.flv
|   |-- /home/askapache.com/static/img/
|   |   |-- /home/askapache.com/static/img/apache.gif
|   |   |-- /home/askapache.com/static/img/apache.jpg
|   |   `-- /home/askapache.com/static/img/apache.png
|   |-- /home/askapache.com/static/js/
|   |   `-- /home/askapache.com/static/js/apache.js
|   |-- /home/askapache.com/static/mp3/
|   |   `-- /home/askapache.com/static/mp3/apache.mp3
|   |-- /home/askapache.com/static/pdf/
|   |   `-- /home/askapache.com/static/pdf/apache.pdf
|   |-- /home/askapache.com/static/swf/
|   |   `-- /home/askapache.com/static/swf/apache.swf
|   |-- /home/askapache.com/static/.htaccess
|   |-- /home/askapache.com/static/index.html
|   `-- /home/askapache.com/static/robots.txt
|-- /home/askapache.com/tmp/
|-- /home/askapache.com/.htpasswd-basic
`-- /home/askapache.com/.htpasswd-digest

The buzz about apache and open-source is very real, apache is becoming more of a discussed topic as people realize the power and importance of Doing it and Doing it and Doing it well. – Movable Type Apache Installs made easy, Checking out Apache Web logs, Securing WordPress with .htaccess, WordPress Permalinks and .htaccess, New search and replace module for apache!, creating an .htaccess template, .htaccess allow directive

Check back in a week for the first article, or better yet subscribe to my rss feed or use the comment form below to get notified.

Optimize a Website for Speed, Security, and Easy Management originally appeared on AskApache.com

]]> http://www.askapache.com/htaccess/optimize-website-files-cache-security.html/feed/ 9 HTTP Status Codes and .htaccess ErrorDocuments http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html#comments Mon, 04 Jan 2010 20:56:15 +0000 AskApache http://www.askapache.com.com/htaccess/wow-i-served-a-page-for-every-single-http-status-code-and-saved-headers-and-content.html There are a total of 57 HTTP Status Codes recognized by the Apache Web Server. Wouldn't you like to see what all those headers and their output, ErrorDocuments look like?

]]> I was trying to find an official, authoritative list of HTTP Status Codes but I kept finding lists that weren’t authoritative or complete. So I searched and found my answer in the Apache HTTP Server source code. Once I had the exact HTTP Status Codes and resulting Error Documents sent by Apache, I researched deeper into HTTP Status Codes by reading as many related RFC’s as I could find, and several other software source codes were explored. This is the most authoritative list I know of, if you can do better leave a comment and I’ll update it. Another thing to keep in mind, the Status code number itself is what is used by software and hardware to make determinations, the phrase returned by the status code is for the human only and does not have any weight other than informing the user.. So “503 Service Unavailable”, “503 Service Temporarily Unavailable”, and “503 Get the heck outta here” are all completely valid.

Update March 9, 2009: A lot of sites on the web have updated their HTTP status code lists to include the HTTP Status codes listed on this page, including Wikipedia, IANA, W3C, and others, so rest assured this info is accurate and complete. If you’d like to see how to create custom error pages for all of these errors like mine /show-error-506 , then check out this detailed tutorial I just posted.

Contents

57 APACHE HTTP STATUS RESPONSE CODES

Once I compiled the list of Apache recognized HTTP Status Codes, I was dying to see them all in action (i.e. the corresponding ErrorDocument). At first I thought I would have to create a php or perl script emulating each of the 57 HTTP Status Codes, a tedious undertaking I wasn’t about to do. Instead I “asked Apache” by searching the Apache HTTP Documentation for ambiguity sending Status Codes and/or triggering ErrorDocuments with an Apache Directive.
While reading up on mod_alias and the Redirect directive I found:

Apache Docs

Syntax: Redirect [status] URL-path URL The status argument can be used to return other HTTP status codes. Other status codes can be returned by giving the numeric status code as the value of status. If the status is between 300 and 399, the URL argument must be present, otherwise it must be omitted.

100 Continue
ErrorDocument Continue | Sample 100 Continue
This means that the server has received the request headers, and that the client should proceed to send the request body (in case of a request which needs to be sent; for example, a POST request). If the request body is large, sending it to a server when a request has already been rejected based upon inappropriate headers is inefficient. To have a server check if the request could be accepted based on the requests headers alone, a client must send Expect: 100-continue as a header in its initial request (see RFC 2616 14.20 Expect header) and check if a 100 Continue status code is received in response before continuing (or receive 417 Expectation Failed and not continue).
101 Switching Protocols
ErrorDocument Switching Protocols | Sample 101 Switching Protocols
This means the requester has asked the server to switch protocols and the server is acknowledging that it will do so.[3]
102 Processing
ErrorDocument Processing | Sample 102 Processing
(WebDAV) – (RFC 2518 )
200 OK
ErrorDocument OK | Sample 200 OK
Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request the response will contain an entity describing or containing the result of the action.
201 Created
ErrorDocument Created | Sample 201 Created
The request has been fulfilled and resulted in a new resource being created.
202 Accepted
ErrorDocument Accepted | Sample 202 Accepted
The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place.
203 Non-Authoritative Information
ErrorDocument Non-Authoritative Information | Sample 203 Non-Authoritative Information
The server successfully processed the request, but is returning information that may be from another source.
204 No Content
ErrorDocument No Content | Sample 204 No Content
The server successfully processed the request, but is not returning any content.
205 Reset Content
ErrorDocument Reset Content | Sample 205 Reset Content
The server successfully processed the request, but is not returning any content. Unlike a 204 response, this response requires that the requester reset the document view.
206 Partial Content
ErrorDocument Partial Content | Sample 206 Partial Content
The server is delivering only part of the resource due to a range header sent by the client. This is used by tools like wget to enable resuming of interrupted downloads, or split a download into multiple simultaneous streams.
207 Multi-Status
ErrorDocument Multi-Status | Sample 207 Multi-Status
(WebDAV) – The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made.
226 IM Used
ErrorDocument IM Used | Sample 226 IM Used
The server has fulfilled a GET request for the resource, and the response is a representation of the result of one or more instance-manipulations applied to the current instance. The actual current instance might not be available except by combining this response with other previous or future responses, as appropriate for the specific instance-manipulation(s).
300 Multiple Choices
ErrorDocument Multiple Choices | Sample 300 Multiple Choices
Indicates multiple options for the resource that the client may follow. It, for instance, could be used to present different format options for video, list files with different extensions, or word sense disambiguation.
301 Moved Permanently
ErrorDocument Moved Permanently | Sample 301 Moved Permanently
This and all future requests should be directed to the given URI.
302 Found
ErrorDocument Found | Sample 302 Found
This is the most popular redirect code[citation needed], but also an example of industrial practice contradicting the standard. HTTP/1.0 specification (RFC 1945 ) required the client to perform a temporary redirect (the original describing phrase was “Moved Temporarily”), but popular browsers implemented it as a 303 See Other. Therefore, HTTP/1.1 added status codes 303 and 307 to disambiguate between the two behaviours. However, the majority of Web applications and frameworks still use the 302 status code as if it were the 303.
303 See Other
ErrorDocument See Other | Sample 303 See Other
The response to the request can be found under another URI using a GET method. When received in response to a PUT, it should be assumed that the server has received the data and the redirect should be issued with a separate GET message.
304 Not Modified
ErrorDocument Not Modified | Sample 304 Not Modified
Indicates the resource has not been modified since last requested. Typically, the HTTP client provides a header like the If-Modified-Since header to provide a time against which to compare. Utilizing this saves bandwidth and reprocessing on both the server and client, as only the header data must be sent and received in comparison to the entirety of the page being re-processed by the server, then resent using more bandwidth of the server and client.
305 Use Proxy
ErrorDocument Use Proxy | Sample 305 Use Proxy
Many HTTP clients (such as Mozilla[4] and Internet Explorer) do not correctly handle responses with this status code, primarily for security reasons.
306 Switch Proxy
ErrorDocument Switch Proxy | Sample 306 Switch Proxy
No longer used.
307 Temporary Redirect
ErrorDocument Temporary Redirect | Sample 307 Temporary Redirect
In this occasion, the request should be repeated with another URI, but future requests can still use the original URI. In contrast to 303, the request method should not be changed when reissuing the original request. For instance, a POST request must be repeated using another POST request.
400 Bad Request
ErrorDocument Bad Request | Sample 400 Bad Request
The request contains bad syntax or cannot be fulfilled.
401 Unauthorized
ErrorDocument Unauthorized | Sample 401 Unauthorized
Similar to 403 Forbidden, but specifically for use when authentication is possible but has failed or not yet been provided. The response must include a WWW-Authenticate header field containing a challenge applicable to the requested resource. See Basic access authentication and Digest access authentication.
402 Payment Required
ErrorDocument Payment Required | Sample 402 Payment Required
The original intention was that this code might be used as part of some form of digital cash or micropayment scheme, but that has not happened, and this code has never been used.
403 Forbidden
ErrorDocument Forbidden | Sample 403 Forbidden
The request was a legal request, but the server is refusing to respond to it. Unlike a 401 Unauthorized response, authenticating will make no difference.
404 Not Found
ErrorDocument Not Found | Sample 404 Not Found
The requested resource could not be found but may be available again in the future. Subsequent requests by the client are permissible.
405 Method Not Allowed
ErrorDocument Method Not Allowed | Sample 405 Method Not Allowed
A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource.
406 Not Acceptable
ErrorDocument Not Acceptable | Sample 406 Not Acceptable
The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request.
407 Proxy Authentication Required
ErrorDocument Proxy Authentication Required | Sample 407 Proxy Authentication Required
Required
408 Request Timeout
ErrorDocument Request Timeout | Sample 408 Request Timeout
The server timed out waiting for the request.
409 Conflict
ErrorDocument Conflict | Sample 409 Conflict
Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.
410 Gone
ErrorDocument Gone | Sample 410 Gone
Indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed; however, it is not necessary to return this code and a 404 Not Found can be issued instead. Upon receiving a 410 status code, the client should not request the resource again in the future. Clients such as search engines should remove the resource from their indexes.
411 Length Required
ErrorDocument Length Required | Sample 411 Length Required
The request did not specify the length of its content, which is required by the requested resource.
412 Precondition Failed
ErrorDocument Precondition Failed | Sample 412 Precondition Failed
The server does not meet one of the preconditions that the requester put on the request.
413 Request Entity Too Large
ErrorDocument Request Entity Too Large | Sample 413 Request Entity Too Large
The request is larger than the server is willing or able to process.
414 Request-URI Too Long
ErrorDocument Request-URI Too Long | Sample 414 Request-URI Too Long
The URI provided was too long for the server to process.
415 Unsupported Media Type
ErrorDocument Unsupported Media Type | Sample 415 Unsupported Media Type
The request did not specify any media types that the server or resource supports. For example the client specified that an image resource should be served as image/svg+xml, but the server cannot find a matching version of the image.
416 Requested Range Not Satisfiable
ErrorDocument Requested Range Not Satisfiable | Sample 416 Requested Range Not Satisfiable
The client has asked for a portion of the file, but the server cannot supply that portion (for example, if the client asked for a part of the file that lies beyond the end of the file).
417 Expectation Failed
ErrorDocument Expectation Failed | Sample 417 Expectation Failed
The server cannot meet the requirements of the Expect request-header field.
418 I’m a teapot
ErrorDocument I’m a teapot | Sample 418 I’m a teapot
The HTCPCP server is a teapot. The responding entity MAY be short and stout. Defined by the April Fools specification RFC 2324. See Hyper Text Coffee Pot Control Protocol for more information.
422 Unprocessable Entity
ErrorDocument Unprocessable Entity | Sample 422 Unprocessable Entity
(WebDAV) (RFC 4918 ) – The request was well-formed but was unable to be followed due to semantic errors.
423 Locked
ErrorDocument Locked | Sample 423 Locked
(WebDAV) (RFC 4918 ) – The resource that is being accessed is locked
424 Failed Dependency
ErrorDocument Failed Dependency | Sample 424 Failed Dependency
(WebDAV) (RFC 4918 ) – The request failed due to failure of a previous request (e.g. a PROPPATCH).
425 Unordered Collection
ErrorDocument Unordered Collection | Sample 425 Unordered Collection
Defined in drafts of WebDav Advanced Collections, but not present in “Web Distributed Authoring and Versioning (WebDAV) Ordered Collections Protocol” (RFC 3648).
426 Upgrade Required
ErrorDocument Upgrade Required | Sample 426 Upgrade Required
(RFC 2817 ) – The client should switch to TLS/1.0.
449 Retry With
ErrorDocument Retry With | Sample 449 Retry With
A Microsoft extension. The request should be retried after doing the appropriate action.
500 Internal Server Error
ErrorDocument Internal Server Error | Sample 500 Internal Server Error
A generic error message, given when no more specific message is suitable.
501 Not Implemented
ErrorDocument Not Implemented | Sample 501 Not Implemented
The server either does not recognise the request method, or it lacks the ability to fulfil the request.
502 Bad Gateway
ErrorDocument Bad Gateway | Sample 502 Bad Gateway
The server was acting as a gateway or proxy and received an invalid response from the upstream server.
503 Service Unavailable
ErrorDocument Service Unavailable | Sample 503 Service Unavailable
The server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state.
504 Gateway Timeout
ErrorDocument Gateway Timeout | Sample 504 Gateway Timeout
The server was acting as a gateway or proxy and did not receive a timely request from the upstream server.
505 HTTP Version Not Supported
ErrorDocument HTTP Version Not Supported | Sample 505 HTTP Version Not Supported
The server does not support the HTTP protocol version used in the request.
506 Variant Also Negotiates
ErrorDocument Variant Also Negotiates | Sample 506 Variant Also Negotiates
(RFC 2295 ) – Transparent content negotiation for the request, results in a circular reference.
507 Insufficient Storage
ErrorDocument Insufficient Storage | Sample 507 Insufficient Storage
(WebDAV) (RFC 4918 )
509 Bandwidth Limit Exceeded
ErrorDocument Bandwidth Limit Exceeded | Sample 509 Bandwidth Limit Exceeded
(Apache bw/limited extension) – This status code, while used by many servers, is not specified in any RFCs.
510 Not Extended
ErrorDocument Not Extended | Sample 510 Not Extended
(RFC 2774 ) – Further extensions to the request are required for the server to fulfil it.

1xx Info / Informational

HTTP_INFO – Request received, continuing process. Indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line.

2xx Success / OK

HTTP_SUCCESS – The action was successfully received, understood, and accepted. Indicates that the client’s request was successfully received, understood, and accepted.

3xx Redirect

HTTP_REDIRECT – The client must take additional action to complete the request. Indicates that further action needs to be taken by the user-agent in order to fulfill the request. The action required may be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A user agent should not automatically redirect a request more than 5 times, since such redirections usually indicate an infinite loop.

4xx Client Error

HTTP_CLIENT_ERROR – The request contains bad syntax or cannot be fulfilled. Indicates case where client seems to have erred. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition.

5xx Server Error

HTTP_SERVER_ERROR – The server failed to fulfill an apparently valid request. Indicate cases in which the server is aware that it has erred or is incapable of performing the request. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. User agents should display any included entity to the user. These response codes are applicable to any request method.

Quick Start to triggering ErrorDocuments for each Status Code

Let start with a quick and easy example. Add the following Redirect rules to your htaccess file, then open your browser and goto each url like yoursite.com/e/400. Don’t create an /e/ directory or any files.

Redirect 400 /e/400
Redirect 503 /e/503
Redirect 405 /e/405

Apache ErrorDocument Results
error 503

Automate the ErrorDocument Triggering

The htaccess Redirects

When a Status code is encountered, Apache outputs the Header and the ErrorDocument for that error code. So you can view any Header and the default ErrorDocument, by causing that numerical error code, which is caused by the Status Code.

For instance, if you request a file that doesn’t exist, a 404 Not Found Header is issued and the corresponding ErrorDocument is served with the 404 Not Found Header.

Redirect 100 /e/100
Redirect 101 /e/101
Redirect 102 /e/102
Redirect 200 /e/200
Redirect 201 /e/201
Redirect 202 /e/202
Redirect 203 /e/203
Redirect 204 /e/204
Redirect 205 /e/205
Redirect 206 /e/206
Redirect 207 /e/207
Redirect 300 /e/300 http://www.askapache.com/?s=300
Redirect 301 /e/301 http://www.askapache.com/?s=301
Redirect 302 /e/302 http://www.askapache.com/?s=302
Redirect 303 /e/303 http://www.askapache.com/?s=303
Redirect 304 /e/304 http://www.askapache.com/?s=304
Redirect 305 /e/305 http://www.askapache.com/?s=305
Redirect 306 /e/306 http://www.askapache.com/?s=306
Redirect 307 /e/307 http://www.askapache.com/?s=307
Redirect 400 /e/400
Redirect 401 /e/401
Redirect 402 /e/402
Redirect 403 /e/403
Redirect 404 /e/404
Redirect 405 /e/405
Redirect 406 /e/406
Redirect 407 /e/407
Redirect 408 /e/408
Redirect 409 /e/409
Redirect 410 /e/410
Redirect 411 /e/411
Redirect 412 /e/412
Redirect 413 /e/413
Redirect 414 /e/414
Redirect 415 /e/415
Redirect 416 /e/416
Redirect 417 /e/417
Redirect 418 /e/418
Redirect 419 /e/419
Redirect 420 /e/420
Redirect 421 /e/421
Redirect 422 /e/422
Redirect 423 /e/423
Redirect 424 /e/424
Redirect 425 /e/425
Redirect 426 /e/426
Redirect 500 /e/500
Redirect 501 /e/501
Redirect 502 /e/502
Redirect 503 /e/503
Redirect 504 /e/504
Redirect 505 /e/505
Redirect 506 /e/506
Redirect 507 /e/507
Redirect 508 /e/508
Redirect 509 /e/509
Redirect 510 /e/510

PHP script that gets and outputs the Headers/Content

Now all I have to do is add 57 Redirect Directives to my htaccess, and then request each of them 1 at a time from my browser to see the result, and use a packet sniffing program like WireShark to see the Headers. Uh, scratch that, that would take way too long!

Instead I hacked up a simple php script using cURL to automate sending GET Requests to each of the 57 Redirect URL-paths. A side benefit of using the php script is that it performs all 57 Requests concurrently and saves each Requests returned headers and content to an output buffer. After all 57 have been queried, the output buffer is flushed to the browser.

<?php
$SITENAME='http://www.askapache.com';
 
$CODES = array(array('100','101','102'),
array('200','201','202','203','204','205','206','207'),
array('300','301','302','303','304','305','306','307'),
array('400','401','402','403','404','405','406','407','408','409','410','411','412','413',
'414','415','416','417','418','419','420','421','422','423','424','425','426'),
array('500','501','502','503','504','505','506','507','508','509','510'));
 
$TMPSAVETO='/tmp/'.time().'.txt';
 
# if file exists then delete it
if(is_file($TMPSAVETO))unlink($TMPSAVETO);
 
foreach($CODES as $keyd => $res)
{
foreach($res as $key)
{
$ch = curl_init("$SITENAME/e/$key");
$fp = fopen ($TMPSAVETO, "a");
curl_setopt ($ch, CURLOPT_FILE, $fp);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION ,1);
curl_setopt ($ch, CURLOPT_HEADER ,1);
curl_exec ($ch);
curl_close ($ch);
fclose ($fp);
}
}
$OUT='';
ob_start();
header ("Content-Type: text/plain;");
readfile($TMPSAVETO);
$OUT=ob_get_clean();
echo $OUT;
unlink($TMPSAVETO);
exit;
?>

Headers and Content Returned

100 Continue 

HTTP/1.1 100 Continue
<html>
<head>
<title>100 Continue</title>
</head>
<body>
<h1>Continue</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

101 Switching Protocols 

HTTP/1.1 101 Switching Protocols<html>
<head>
<title>101 Switching Protocols</title>
</head>
<body>
<h1>Switching Protocols</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

102 Processing 

HTTP/1.1 102 Processing
X-Pad: avoid browser bug<html>
<head>
<title>102 Processing</title>
</head>
<body>
<h1>Processing</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

200 OK 

HTTP/1.1 200 OK
<html>
<head>
<title>200 OK</title>
</head>
<body>
<h1>OK</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

201 Created 

HTTP/1.1 201 Created
<html>
<head>
<title>201 Created</title>
</head>
<body>
<h1>Created</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

202 Accepted 

HTTP/1.1 202 Accepted
<html>
<head>
<title>202 Accepted</title>
</head>
<body>
<h1>Accepted</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

203 Non-Authoritative Information 

HTTP/1.1 203 Non-Authoritative Information
<html>
<head>
<title>203 Non-Authoritative Information</title>
</head>
<body>
<h1>Non-Authoritative Information</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

204 No Content 

HTTP/1.1 204 No Content
Content-Type: text/plain; charset=UTF-8
 

205 Reset Content 

HTTP/1.1 205 Reset Content<html>
<head>
<title>205 Reset Content</title>
</head>
<body>
<h1>Reset Content</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

206 Partial Content 

HTTP/1.1 206 Partial Content<html>
<head>
<title>206 Partial Content</title>
</head>
<body>
<h1>Partial Content</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

207 Multi-Status 

HTTP/1.1 207 Multi-Status
X-Pad: avoid browser bug<html>
<head>
<title>207 Multi-Status</title>
</head>
<body>
<h1>Multi-Status</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

300 Multiple Choices 

HTTP/1.1 300 Multiple Choices
Location: http://www.askapache.com/?s=300<html>
<head>
<title>300 Multiple Choices</title>
</head>
<body>
<h1>Multiple Choices</h1>
</body>
</html>

301 Moved Permanently 

HTTP/1.1 301 Moved Permanently
Location: http://www.askapache.com/?s=301<html>
<head>
<title>301 Moved Permanently</title>
</head>
<body>
<h1>Moved Permanently</h1>
<p>The document has moved  <a href="http://www.askapache.com/?s=301">here</a> .</p>
</body>
</html>

302 Found 

HTTP/1.1 302 Found
Location: http://www.askapache.com/?s=302<html>
<head>
<title>302 Found</title>
</head>
<body>
<h1>Found</h1>
<p>The document has moved  <a href="http://www.askapache.com/?s=302">here</a> .</p>
</body>
</html>

303 See Other 

HTTP/1.1 303 See Other
Location: http://www.askapache.com/?s=303<html>
<head>
<title>303 See Other</title>
</head>
<body>
<h1>See Other</h1>
<p>The answer to your request is located  <a href="http://www.askapache.com/?s=303">here</a> .</p>
</body>
</html>

304 Not Modified 

HTTP/1.1 304 Not Modified

305 Use Proxy 

HTTP/1.1 305 Use Proxy
Location: http://www.askapache.com/?s=305<html>
<head>
<title>305 Use Proxy</title>
</head>
<body>
<h1>Use Proxy</h1>
<p>This resource is only accessible through the proxy
    http://www.askapache.com/?s=305<br />You will need to configure your client to use that proxy.</p>
</body>
</html>

306 unused 

HTTP/1.1 306 unused
Location: http://www.askapache.com/?s=306<html>
<head>
<title>306 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

307 Temporary Redirect 

HTTP/1.1 307 Temporary Redirect
Location: http://www.askapache.com/?s=307<html>
<head>
<title>307 Temporary Redirect</title>
</head>
<body>
<h1>Temporary Redirect</h1>
<p>The document has moved  <a href="http://www.askapache.com/?s=307">here</a> .</p>
</body>
</html>

400 Bad Request 

HTTP/1.1 400 Bad Request
Connection: close<html>
<head>
<title>400 Bad Request</title>
</head>
<body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.</p>
</body>
</html>

401 Authorization Required 

HTTP/1.1 401 Authorization Required<html>
<head>
<title>401 Authorization Required</title>
</head>
<body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
    are authorized to access the document
    requested.  Either you supplied the wrong
    credentials (e.g., bad password), or your
    browser doesn't understand how to supply
    the credentials required.</p>
<p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body>
</html>

402 Payment Required 

HTTP/1.1 402 Payment Required<html>
<head>
<title>402 Payment Required</title>
</head>
<body>
<h1>Payment Required</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

403 Forbidden 

HTTP/1.1 403 Forbidden<html>
<head>
<title>403 Forbidden</title>
</head>
<body>
<h1>Forbidden</h1>
<p>You don't have permission to access /e/403
    on this server.</p>
<p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body>
</html>

404 Not Found 

HTTP/1.1 404 Not Found<html>
<head>
<title>404 Not Found</title>
</head>
<body>
<h1>Not Found</h1>
<p>The requested URL /e/404 was not found on this server.</p>
</body>
</html>

NOTE:
You will most definately want to check out and use the Google 404 Error Page if you run WordPress.

405 Method Not Allowed 

HTTP/1.1 405 Method Not Allowed
Allow: TRACE
<html>
<head>
<title>405 Method Not Allowed</title>
</head>
<body>
<h1>Method Not Allowed</h1>
<p>The requested method GET is not allowed for the URL /e/405.</p>
</body>
</html>

406 Not Acceptable 

HTTP/1.1 406 Not Acceptable
<html>
<head>
<title>406 Not Acceptable</title>
</head>
<body>
<h1>Not Acceptable</h1>
<p>An appropriate representation of the requested resource /e/406 could not be found on this server.</p>
</body>
</html>

407 Proxy Authentication Required 

HTTP/1.1 407 Proxy Authentication Required<html>
<head>
<title>407 Proxy Authentication Required</title>
</head>
<body>
<h1>Proxy Authentication Required</h1>
<p>This server could not verify that you
    are authorized to access the document
    requested.  Either you supplied the wrong
    credentials (e.g., bad password), or your
    browser doesn't understand how to supply
    the credentials required.</p>
</body>
</html>

408 Request Time-out 

HTTP/1.1 408 Request Time-out
Connection: close
<html>
<head>
<title>408 Request Time-out</title>
</head>
<body>
<h1>Request Time-out</h1>
<p>Server timeout waiting for the HTTP request from the client.</p>
</body>
</html>

409 Conflict 

HTTP/1.1 409 Conflict
<html>
<head>
<title>409 Conflict</title>
</head>
<body>
<h1>Conflict</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

410 Gone 

HTTP/1.1 410 Gone
<html>
<head>
<title>410 Gone</title>
</head>
<body>
<h1>Gone</h1>
<p>The requested resource<br />/e/410<br />is no longer available on this server and there is no forwarding address.
    Please remove all references to this resource.</p>
</body>
</html>

411 Length Required 

HTTP/1.1 411 Length Required
Connection: close
<html>
<head>
<title>411 Length Required</title>
</head>
<body>
<h1>Length Required</h1>
<p>A request of the requested method GET requires a valid Content-length.</p>
</body>
</html>

412 Precondition Failed 

HTTP/1.1 412 Precondition Failed
<html>
<head>
<title>412 Precondition Failed</title>
</head>
<body>
<h1>Precondition Failed</h1>
<p>The precondition on the request for the URL /e/412 evaluated to false.</p>
</body>
</html>

413 Request Entity Too Large 

HTTP/1.1 413 Request Entity Too Large
Connection: close<html>
<head>
<title>413 Request Entity Too Large</title>
</head>
<body>
<h1>Request Entity Too Large</h1>
The requested resource<br />/e/413<br />does not allow request data with GET requests, or the amount of data provided in
the request exceeds the capacity limit.
</body>
</html>

414 Request-URI Too Large 

HTTP/1.1 414 Request-URI Too Large
Connection: close<html>
<head>
<title>414 Request-URI Too Large</title>
</head>
<body>
<h1>Request-URI Too Large</h1>
<p>The requested URL's length exceeds the capacity
    limit for this server.</p>
</body>
</html>

415 Unsupported Media Type 

HTTP/1.1 415 Unsupported Media Type<html>
<head>
<title>415 Unsupported Media Type</title>
</head>
<body>
<h1>Unsupported Media Type</h1>
<p>The supplied request data is not in a format
    acceptable for processing by this resource.</p>
</body>
</html>

416 Requested Range Not Satisfiable 

HTTP/1.1 416 Requested Range Not Satisfiable

417 Expectation Failed 

HTTP/1.1 417 Expectation Failed<html>
<head>
<title>417 Expectation Failed</title>
</head>
<body>
<h1>Expectation Failed</h1>
<p>The expectation given in the Expect request-header
    field could not be met by this server.</p>
<p>The client sent<pre>
    Expect: </p>
</body>
</html>

418 I’m a teapot 

HTTP/1.1 418 I'm a teapot<html>
<head>
<title>418 I'm a teapot</title>
</head>
<body>
<h1>I'm a teapot</h1>
<p>Unfortunately this coffee machine is out of coffee.</p>
</body>
</html>

419 unused 

HTTP/1.1 419 unused<html>
<head>
<title>419 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

420 unused 

HTTP/1.1 420 unused<html>
<head>
<title>420 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

421 unused 

HTTP/1.1 421 unused<html>
<head>
<title>421 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

422 Unprocessable Entity 

HTTP/1.1 422 Unprocessable Entity<html>
<head>
<title>422 Unprocessable Entity</title>
</head>
<body>
<h1>Unprocessable Entity</h1>
<p>The server understands the media type of the
    request entity, but was unable to process the
    contained instructions.</p>
</body>
</html>

423 Locked 

HTTP/1.1 423 Locked<html>
<head>
<title>423 Locked</title>
</head>
<body>
<h1>Locked</h1>
<p>The requested resource is currently locked.
    The lock must be released or proper identification
    given before the method can be applied.</p>
</body>
</html>

424 Failed Dependency 

HTTP/1.1 424 Failed Dependency<html>
<head>
<title>424 Failed Dependency</title>
</head>
<body>
<h1>Failed Dependency</h1>
<p>The method could not be performed on the resource
    because the requested action depended on another
    action and that other action failed.</p>
</body>
</html>

425 No code 

HTTP/1.1 425 No code<html>
<head>
<title>425 No code</title>
</head>
<body>
<h1>No code</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

426 Upgrade Required 

HTTP/1.1 426 Upgrade Required<html>
<head>
<title>426 Upgrade Required</title>
</head>
<body>
<h1>Upgrade Required</h1>
<p>The requested resource can only be retrieved
    using SSL.  The server is willing to upgrade the current
    connection to SSL, but your client doesn't support it.
    Either upgrade your client, or try requesting the page
    using https:// </p>
</body>
</html>

500 Internal Server Error 

HTTP/1.1 500 Internal Server Error
Connection: close<html>
<head>
<title>500 Internal Server Error</title>
</head>
<body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
<p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body>
</html>

501 Method Not Implemented 

HTTP/1.1 501 Method Not Implemented
Allow: TRACE
Connection: close<html>
<head>
<title>501 Method Not Implemented</title>
</head>
<body>
<h1>Method Not Implemented</h1>
<p>GET to /e/501 not supported.</p>
</body>
</html>

502 Bad Gateway 

HTTP/1.1 502 Bad Gateway
X-Pad: avoid browser bug<html>
<head>
<title>502 Bad Gateway</title>
</head>
<body>
<h1>Bad Gateway</h1>
<p>The proxy server received an invalid
    response from an upstream server.</p>
</body>
</html>

503 Service Temporarily Unavailable 

HTTP/1.1 503 Service Temporarily Unavailable
Connection: close<html>
<head>
<title>503 Service Temporarily Unavailable</title>
</head>
<body>
<h1>Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your
    request due to maintenance downtime or capacity
    problems. Please try again later.</p>
</body>
</html>

504 Gateway Time-out 

HTTP/1.1 504 Gateway Time-out<html>
<head>
<title>504 Gateway Time-out</title>
</head>
<body>
<h1>Gateway Time-out</h1>
<p>The proxy server did not receive a timely response
    from the upstream server.</p>
</body>
</html>

505 HTTP Version Not Supported 

HTTP/1.1 505 HTTP Version Not Supported<html>
<head>
<title>505 HTTP Version Not Supported</title>
</head>
<body>
<h1>HTTP Version Not Supported</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

506 Variant Also Negotiates 

HTTP/1.1 506 Variant Also Negotiates<html>
<head>
<title>506 Variant Also Negotiates</title>
</head>
<body>
<h1>Variant Also Negotiates</h1>
<p>A variant for the requested resource
    <pre>
    /e/506
    </pre>
    is itself a negotiable resource. This indicates a configuration error.</p>
</body>
</html>

507 Insufficient Storage 

HTTP/1.1 507 Insufficient Storage<html>
<head>
<title>507 Insufficient Storage</title>
</head>
<body>
<h1>Insufficient Storage</h1>
<p>The method could not be performed on the resource
    because the server is unable to store the
    representation needed to successfully complete the
    request.  There is insufficient free space left in
    your storage allocation.</p>
</body>
</html>

508 unused 

HTTP/1.1 508 unused<html>
<head>
<title>508 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

509 unused 

HTTP/1.1 509 unused
<html>
<head>
<title>509 unused</title>
</head>
<body>
<h1>unused</h1>
<p>The server encountered an internal error or misconfigurationand was unable to complete your request.</p>
<p>Please contact the server administrator, a@s.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.</p>
<p>More information about this error may be available in the server error log.</p>
</body>
</html>

510 Not Extended 

HTTP/1.1 510 Not Extended
X-Pad: avoid browser bug
<html>
<head>
<title>510 Not Extended</title>
</head>
<body>
<h1>Not Extended</h1>
<p>A mandatory extension policy in the request is not
    accepted by the server for this resource.</p>
</body>
</html>

Apache Source Code

httpd.h

From httpd.h 

/**
* The size of the static array in http_protocol.c for storing
* all of the potential response status-lines (a sparse table).
* A future version should dynamically generate the apr_table_t at startup.
*/
#define RESPONSE_CODES 57
#define HTTP_CONTINUE            100
#define HTTP_SWITCHING_PROTOCOLS       101
#define HTTP_PROCESSING          102
#define HTTP_OK              200
#define HTTP_CREATED             201
#define HTTP_ACCEPTED            202
#define HTTP_NON_AUTHORITATIVE       203
#define HTTP_NO_CONTENT          204
#define HTTP_RESET_CONTENT         205
#define HTTP_PARTIAL_CONTENT         206
#define HTTP_MULTI_STATUS          207
#define HTTP_MULTIPLE_CHOICES        300
#define HTTP_MOVED_PERMANENTLY       301
 
#define HTTP_MOVED_TEMPORARILY       302
#define HTTP_SEE_OTHER           303
#define HTTP_NOT_MODIFIED          304
#define HTTP_USE_PROXY           305
#define HTTP_TEMPORARY_REDIRECT      307
#define HTTP_BAD_REQUEST           400
#define HTTP_UNAUTHORIZED          401
#define HTTP_PAYMENT_REQUIRED        402
#define HTTP_FORBIDDEN           403
#define HTTP_NOT_FOUND           404
#define HTTP_METHOD_NOT_ALLOWED      405
#define HTTP_NOT_ACCEPTABLE        406
#define HTTP_PROXY_AUTHENTICATION_REQUIRED 407
#define HTTP_REQUEST_TIME_OUT        408
#define HTTP_CONFLICT            409
#define HTTP_GONE              410
#define HTTP_LENGTH_REQUIRED         411
#define HTTP_PRECONDITION_FAILED       412
#define HTTP_REQUEST_ENTITY_TOO_LARGE    413
#define HTTP_REQUEST_URI_TOO_LARGE     414
#define HTTP_UNSUPPORTED_MEDIA_TYPE    415
#define HTTP_RANGE_NOT_SATISFIABLE     416
#define HTTP_EXPECTATION_FAILED      417
#define HTTP_UNPROCESSABLE_ENTITY      422
#define HTTP_LOCKED            423
#define HTTP_FAILED_DEPENDENCY       424
#define HTTP_UPGRADE_REQUIRED        426
#define HTTP_INTERNAL_SERVER_ERROR     500
#define HTTP_NOT_IMPLEMENTED         501
#define HTTP_BAD_GATEWAY           502
#define HTTP_SERVICE_UNAVAILABLE       503
#define HTTP_GATEWAY_TIME_OUT        504
#define HTTP_VERSION_NOT_SUPPORTED     505
#define HTTP_VARIANT_ALSO_VARIES       506
#define HTTP_INSUFFICIENT_STORAGE      507
#define HTTP_NOT_EXTENDED          510
 
/** is the status code informational */
#define ap_is_HTTP_INFO(x)     (((x) >= 100)&&((x) < 200))
/** is the status code OK ?*/
#define ap_is_HTTP_SUCCESS(x)    (((x) >= 200)&&((x) < 300))
/** is the status code a redirect */
#define ap_is_HTTP_REDIRECT(x)   (((x) >= 300)&&((x) < 400))
/** is the status code a error (client or server) */
#define ap_is_HTTP_ERROR(x)    (((x) >= 400)&&((x) < 600))
/** is the status code a client error  */
#define ap_is_HTTP_CLIENT_ERROR(x) (((x) >= 400)&&((x) < 500))
/** is the status code a server error  */
#define ap_is_HTTP_SERVER_ERROR(x) (((x) >= 500)&&((x) < 600))
/** is the status code a (potentially) valid response code?  */
#define ap_is_HTTP_VALID_RESPONSE(x) (((x) >= 100)&&((x) < 600))
 
/** should the status code drop the connection */
#define ap_status_drops_connection(x) \
(((x) == HTTP_BAD_REQUEST)       || \
((x) == HTTP_REQUEST_TIME_OUT)    || \
((x) == HTTP_LENGTH_REQUIRED)     || \
((x) == HTTP_REQUEST_ENTITY_TOO_LARGE) || \
((x) == HTTP_REQUEST_URI_TOO_LARGE) || \
((x) == HTTP_INTERNAL_SERVER_ERROR) || \
((x) == HTTP_SERVICE_UNAVAILABLE) || \
((x) == HTTP_NOT_IMPLEMENTED))

HTTP_INFO

Is the status code (x) informational?

x >= 100 && x < 200

HTTP_SUCCESS

Is the status code (x) OK?

x >= 200 && x < 300

HTTP_REDIRECT

Is the status code (x) a redirect?

x >= 300 && x < 400

HTTP_ERROR

Is the status code (x) a error (client or server)?

x >= 400 && x < 600

HTTP_CLIENT_ERROR

Is the status code (x) a client error?

x >= 400 && x < 500

HTTP_SERVER_ERROR

Is the status code (x) a server error?

x >= 500 && x < 600

HTTP_VALID_RESPONSE

Is the status code (x) a (potentially) valid response code?

x >= 100 && x < 600

http_protocol.c

From http_protocol.c 

static const char * status_lines[RESPONSE_CODES] =
static const char * const status_lines[RESPONSE_CODES] =
"100 Continue",
"101 Switching Protocols",
"102 Processing",
"200 OK",
"201 Created",
"202 Accepted",
"203 Non-Authoritative Information",
"204 No Content",
"205 Reset Content",
"206 Partial Content",
"207 Multi-Status",
"300 Multiple Choices",
"301 Moved Permanently",
"302 Found",
"303 See Other",
"304 Not Modified",
"305 Use Proxy",
"306 unused",
"307 Temporary Redirect",
"400 Bad Request",
"401 Authorization Required",
"402 Payment Required",
"403 Forbidden",
"404 Not Found",
"405 Method Not Allowed",
"406 Not Acceptable",
"407 Proxy Authentication Required",
"408 Request Time-out",
"409 Conflict",
"410 Gone",
"411 Length Required",
"412 Precondition Failed",
"413 Request Entity Too Large",
"414 Request-URI Too Large",
"415 Unsupported Media Type",
"416 Requested Range Not Satisfiable",
"417 Expectation Failed",
"418 unused",
"419 unused",
"420 unused",
"421 unused",
"422 Unprocessable Entity",
"423 Locked",
"424 Failed Dependency",
"425 No code",
"426 Upgrade Required",
"500 Internal Server Error",
"501 Method Not Implemented",
"502 Bad Gateway",
"503 Service Temporarily Unavailable",
"504 Gateway Time-out",
"505 HTTP Version Not Supported",
"506 Variant Also Negotiates",
"507 Insufficient Storage",
"508 unused",
"509 unused",
"510 Not Extended"

IANA HTTP Status Code Registry

Value Description Reference
100 Continue [RFC2616] , Section 10.1.1
101 Switching Protocols [RFC2616] , Section 10.1.2
102 Processing [RFC2518] , Section 10.1
200 OK [RFC2616] , Section 10.2.1
201 Created [RFC2616] , Section 10.2.2
202 Accepted [RFC2616] , Section 10.2.3
203 Non-Authoritative Information [RFC2616] , Section 10.2.4
204 No Content [RFC2616] , Section 10.2.5
205 Reset Content [RFC2616] , Section 10.2.6
206 Partial Content [RFC2616] , Section 10.2.7
207 Multi-Status [RFC4918] , Section 11.1
226 IM Used [RFC3229] , Section 10.4.1
300 Multiple Choices [RFC2616] , Section 10.3.1
301 Moved Permanently [RFC2616] , Section 10.3.2
302 Found [RFC2616] , Section 10.3.3
303 See Other [RFC2616] , Section 10.3.4
304 Not Modified [RFC2616] , Section 10.3.5
305 Use Proxy [RFC2616] , Section 10.3.6
306 (Reserved) [RFC2616] , Section 10.3.7
307 Temporary Redirect [RFC2616] , Section 10.3.8
400 Bad Request [RFC2616] , Section 10.4.1
401 Unauthorized [RFC2616] , Section 10.4.2
402 Payment Required [RFC2616] , Section 10.4.3
403 Forbidden [RFC2616] , Section 10.4.4
404 Not Found [RFC2616] , Section 10.4.5
405 Method Not Allowed [RFC2616] , Section 10.4.6
406 Not Acceptable [RFC2616] , Section 10.4.7
407 Proxy Authentication Required [RFC2616] , Section 10.4.8
408 Request Timeout [RFC2616] , Section 10.4.9
409 Conflict [RFC2616] , Section 10.4.10
410 Gone [RFC2616] , Section 10.4.11
411 Length Required [RFC2616] , Section 10.4.12
412 Precondition Failed [RFC2616] , Section 10.4.13
413 Request Entity Too Large [RFC2616] , Section 10.4.14
414 Request-URI Too Long [RFC2616] , Section 10.4.15
415 Unsupported Media Type [RFC2616] , Section 10.4.16
416 Requested Range Not Satisfiable [RFC2616] , Section 10.4.17
417 Expectation Failed [RFC2616] , Section 10.4.18
422 Unprocessable Entity [RFC4918] , Section 11.2
423 Locked [RFC4918] , Section 11.3
424 Failed Dependency [RFC4918] , Section 11.4
426 Upgrade Required [RFC2817] , Section 6
500 Internal Server Error [RFC2616] , Section 10.5.1
501 Not Implemented [RFC2616] , Section 10.5.2
502 Bad Gateway [RFC2616] , Section 10.5.3
503 Service Unavailable [RFC2616] , Section 10.5.4
504 Gateway Timeout [RFC2616] , Section 10.5.5
505 HTTP Version Not Supported [RFC2616] , Section 10.5.6
506 Variant Also Negotiates [RFC2295] , Section 8.1
507 Insufficient Storage [RFC4918] , Section 11.5
510 Not Extended [RFC2774] , Section 7

WordPress 2.8 Changes

I just learned that my modification to the WordPress core was accepted and will be implemented for version 2.8! This may mean WordPress is the only 100% HTTP/1.1 compliant software on the net! Below is the new list (I thought someone out there could use the php array) and as you can see, unfortunately418 I’m a teapotdidn’t make it ;)

$wp_header_to_desc = array(
  100 => 'Continue',
  101 => 'Switching Protocols',
  102 => 'Processing',
 
  200 => 'OK',
  201 => 'Created',
  202 => 'Accepted',
  203 => 'Non-Authoritative Information',
  204 => 'No Content',
  205 => 'Reset Content',
  206 => 'Partial Content',
  207 => 'Multi-Status',
  226 => 'IM Used',
 
  300 => 'Multiple Choices',
  301 => 'Moved Permanently',
  302 => 'Found',
  303 => 'See Other',
  304 => 'Not Modified',
  305 => 'Use Proxy',
  306 => 'Reserved',
  307 => 'Temporary Redirect',
 
  400 => 'Bad Request',
  401 => 'Unauthorized',
  402 => 'Payment Required',
  403 => 'Forbidden',
  404 => 'Not Found',
  405 => 'Method Not Allowed',
  406 => 'Not Acceptable',
  407 => 'Proxy Authentication Required',
  408 => 'Request Timeout',
  409 => 'Conflict',
  410 => 'Gone',
  411 => 'Length Required',
  412 => 'Precondition Failed',
  413 => 'Request Entity Too Large',
  414 => 'Request-URI Too Long',
  415 => 'Unsupported Media Type',
  416 => 'Requested Range Not Satisfiable',
  417 => 'Expectation Failed',
  422 => 'Unprocessable Entity',
  423 => 'Locked',
  424 => 'Failed Dependency',
  426 => 'Upgrade Required',
 
  500 => 'Internal Server Error',
  501 => 'Not Implemented',
  502 => 'Bad Gateway',
  503 => 'Service Unavailable',
  504 => 'Gateway Timeout',
  505 => 'HTTP Version Not Supported',
  506 => 'Variant Also Negotiates',
  507 => 'Insufficient Storage',
  510 => 'Not Extended'
);

RIPE WHOIS

All the status codes are standard HTTP codes ( http://www.iana.org/assignments/http-status-codes ).

Clients should avoid any form of coupling with the the text/plain error message contained in response body since it may change between different releases of the API and is only intended as a starting point for indentifying the real causes of the exception event.

The following table gives a brief description of the mapping between standard Whois V.3 responses and the related REST services status codes. Consider this table as just an example of the error mapping strategy, it may change with future releases.

System Exception Whois Error HTTP Status Code
IllegalArgumentException Bad Request (400)
IllegalStateException Internal Server Error (500)
UnsupportedOperationException Bad Request (400)
ObjectNotFoundException Not Found (404)
IllegalStateException Bad Request (400)
IOException Bad Request (400)
SystemException Internal Server Error (500)
TooManyResultsException Internal Server Error (500)
WhoisServerException No Entries Found (101) Not Found (404)
WhoisServerException Unknown Source (102) Bad Request (400)
WhoisServerException Unknown Object Type (103) Bad Request (400)
WhoisServerException Unknown Attribute in Query (104) Bad Request (400)
WhoisServerException Attribute Is Not Inverse Searchable (105) Bad Request (400)
WhoisServerException No Search Key Specified (106) Bad Request (400)
WhoisServerException Access Denied (201) Forbidden (403)
WhoisServerException Access Control Limit Reached (202) Forbidden (403)
WhoisServerException Address Passing Not Allowed (203) Bad Request (400)
WhoisServerException Maximum Referral Lines Exceeded (204) Internal Server Error (500)
WhoisServerException Connection Has Been Closed(301) Internal Server Error (500)
WhoisServerException Referral Timeout (302) Internal Server Error (500)
WhoisServerException No Referral Host (303) Internal Server Error (500)
WhoisServerException Referral Host Not Responding (304) Internal Server Error (500)

Clients will have to define error messages generic enough to represent the four main error conditions, that are Bad Request, Forbidden, Not Found and Internal Server Error.

For example a possible mapping for client side error messages may be:

HTTP Status Code Error Message
Bad Request (400) The service is unable to understand and process the query.
Forbidden (403) Query limit exceeded.
Not Found (404) No results were found for Your search “Search term“
Internal Server Error (500) The server encountered an unexpected condition which prevented it from fulfilling the request.

Helpful HTTP Links

  1. IANA registry
  2. Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0)
  3. Adobe Flash status code definitions (ie 408)
  4. Microsoft Internet Information Server Status Codes and Sub-Codes
  5. httplint
  6. HTTP Headers, brief intro.
  7. Common User-Agent Issues
  8. [RFC2295] Holtman, K. and A.H. Mutz , ” Transparent Content Negotiation in HTTP “, RFC 2295, March 1998.
  9. [RFC2518] Goland, Y. , Whitehead, E. , Faizi, A. , Carter, S.R. , and D. Jensen , ” HTTP Extensions for Distributed Authoring — WEBDAV “, RFC 2518, February 1999.
  10. [RFC2616] Fielding, R. , Gettys, J. , Mogul, J. , Frystyk, H. , Masinter, L. , Leach, P. , and T. Berners-Lee , ” Hypertext Transfer Protocol — HTTP/1.1 “, RFC 2616, June 1999.
  11. [RFC2774] Nielsen, H. , Leach, P. , and S. Lawrence , ” An HTTP Extension Framework “, RFC 2774, February 2000.
  12. [RFC2817] Khare, R. and S. Lawrence, ” Upgrading to TLS Within HTTP/1.1 “, RFC 2817, May 2000.
  13. [RFC3229] Mogul, J., Krishnamurthy, B., Douglis, F., Feldmann, A., Goland, Y., van Hoff, A., and D. Hellerstein, ” Delta encoding in HTTP “, RFC 3229, January 2002.
  14. [RFC4918] Dusseault, L., Ed. , ” HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV) “, RFC 4918, June 2007.

HTTP Status Codes and .htaccess ErrorDocuments originally appeared on AskApache.com

]]> http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html/feed/ 22 Vetted – Top 3 WordPress Speed Plugins http://www.askapache.com/wordpress/fastest-caching-plugins.html http://www.askapache.com/wordpress/fastest-caching-plugins.html#comments Sun, 29 Nov 2009 19:14:16 +0000 AskApache http://www.askapache.com/?p=3701 There are so many WordPress plugins out there now that I wanted to post my favorite 3 plugins for speeding up a WP-Powered blog. These are the 3 plugins that I install for pretty much all of my WP-Powered sites, which I run about 300 now. They work together to provide a very optimized blog for speed.

DB-Cache Reloaded does something entirely different, it saves the mysql queries that are made to the WP-database, as well as the mysql results to static files, and then through php serves those cached-files instead of re-querying the mysql database. Most mysql databases are stored on separate servers, and although many are on the same local network there is a limit to how many queries, and how many connections can take place.

So DB-Cache Reloaded basically makes WP-Super Cache work alot faster when generating the cache files, and DB-Cache Reloaded helps in a number of areas un-related to WP-Super Cache, like in the admin panel. And DB-Cache without WP-Super-Cache is a joke because it still uses the application-level and php for everything. Gotta use both (or just WPSC).

]]> There are so many WordPress plugins out there now that I wanted to post my favorite 3 plugins for speeding up a WP-Powered blog (including one of my plugins). These are the 3 plugins that I install for pretty much all of my WP-Powered sites, which I run about 300 now. They work together to provide a very optimized blog for speed.

Top 3 WordPress Speed Plugins

  1. DB Cache Reloaded
  2. WP Super Cache
  3. AskApache Crazy Cache

WP Caching Overview

Each request to your blog has to fire up the php interpreter and query the mysql database to create the output that you see in your browser. Using a plugin like WP Super Cache simply saves the output of that request as a static HTML file and serves that to a request instead of using php and mysql every time. DB Cache Reloaded takes this a step further by optimizing the mysql queries. Finally, AskApache Crazy Cache is used to keep a cache fully primed and ready.

Why Caching?

If you have a private server, or you want to keep your MEMORY, BANDWIDTH, and CPU usage down for your server, these plugins will be dramatic. If you have a site that is updated maybe once a month and gets a very small amount of traffic, then the AskApache Crazy Cache would be redundant. That plugin is geared for the heaviest traffic sites.

Request and Response

Most HTTP communication is initiated by a user agent (browser) and consists of a request to a resource on some origin server. In the simplest case, this may be accomplished via a single connection (v) between the user agent (UA) and the origin server (O).

             request chain ------------------------>
          UA -------------------v------------------- O
             <----------------------- response chain

If a browser requests a WP-driven page, the server generates the response (the outputted html) by loading a php interpreter or module to read the WP php files and load all the settings. WP stores the settings in a mysql database and has to query the mysql database for all the data (like the content of your post that becomes html). Finally php sends the output through the server back to the browser. This is the norm for most PHP applications. Every time an interpreter is loaded additional CPU and Memory are used. And perhaps even more troublesome for shared-hosting using a virtual or network filesystem, each load causes many hard-disk accesses, additional processes, etc.

Cache PHP

By saving the php-generated output of a page to a static html file, your server can entirely skip loading a php interpreter or other process. Originally servers were needed and created to essentially open a static file on disk and send that file back to the requesting user. So servers are specialized for this as it’s their core function.

WP-Super cache does this for you by saving the output into a static html file, and by instructing the server to skip loading php. It basically only uses php for creating the cached version, in the same way that you can save this webpage as an html file, WP-Super cache saves all the WP-blog permalinks to static files.

DB-Cache Reloaded does something entirely different, it saves the mysql queries that are made to the WP-database, as well as the mysql results to static files, and then through php serves those cached-files instead of re-querying the mysql database. Most mysql databases are stored on separate servers, and although many are on the same local network there is a limit to how many queries, and how many connections can take place. But mysql is maybe the fastest thing I’ve seen, so your bottlenecks almost never happen there (if configured correctly).

So DB-Cache Reloaded basically makes WP-Super Cache work alot faster when generating the cache files, and DB-Cache Reloaded helps in a number of areas un-related to WP-Super Cache, like in the admin panel. And DB-Cache without WP-Super-Cache is a joke because it still uses the application-level and php for everything. Gotta use both (or just WPSC).

AskApache Crazy Cache is a plugin I wrote to do one thing very well, it runs at intervals via the WP-cron and forces WP-Super-Cache to create a static cache file for all the posts, pages, etc. on your site. Without this WPSC likes to do dumb things like try to manage it’s own cache with stale files and expired files, which equals a lot more php interpreters getting loaded instead of cached static files. For sites with more than a visit/page/10minutes this plugin keeps a full primed cache built by WPSC.

Compression and WP-Super Cache

Enabling compression in WP-Super Cache is almost always a great idea. I’ve never had a problem other than some php compat issues with not-updated php installations. This option basically lets WP-Super Cache compress the generated output of a page and save that to the static file. Normally Apache, Lighttpd, Nginx, etc. open the static file and compress it before it is sent to the browser, then the browser automatically decompresses it to view. This happens so fast because it is run by the server.

This lets WPSC instruct your server to send the compressed version to all browsers that accept compression, and send the uncompressed static file to any other browsers. So this is helpful because it eliminates your server having to do any transparent compressing, it can instead just focus on what it does best, serving static files.

PHP is an application so it requires memory, hard-drive access, and CPU time. Check out the protocol hierarchy:

       +------+ +-----+ +-----+       +-----+
       |Telnet| | FTP | |Voice|  ...  |     |  Application Level
       +------+ +-----+ +-----+       +-----+
             |   |         |             |
            +-----+     +-----+       +-----+
            | TCP |     | RTP |  ...  |     |  Host Level
            +-----+     +-----+       +-----+
               |           |             |
            +-------------------------------+
            |    Internet Protocol & ICMP   |  Gateway Level
            +-------------------------------+
                           |
              +---------------------------+
              |   Local Network Protocol  |    Network Level
              +---------------------------+

DB Cache Reloaded

DB Cache Reloaded Plugin Page – News

DB Cache ReloadedThis plugin caches every database query with given lifetime. It is much faster than other html caching plugins and uses less disk space for caching.

I think you’ve heard of WP-Cache or WP Super Cache, they are both top plugins for WordPress, which make your site faster and responsive. Forget about them – with DB Cache Reloaded your site will work much faster and will use less disk space for cached files. Your visitors will always get actual information in sidebars and server CPU loads will be as low as possible.

Why is DB Cache Reloaded better than WP Super Cache?

This plugin is based on a fundamentally different principle of caching queries to database instead of full pages, which optimises WordPress from the very beginning and uses less disk space for cache files because it saves only useful information. It saves information separately and also caches hidden requests to database.

WP Super Cache

WP Super Cache Plugin Page – News

WP Super CacheThis plugin generates static html files from your dynamic WordPress blog. After a html file is generated your webserver will serve that file instead of processing the comparatively heavier and more expensive WordPress PHP scripts.

The static html files will be served to the vast majority of your users, but because a user’s details are displayed in the comment form after they leave a comment those requests are handled by PHP. Static files are served to:

· Users who are not logged in.
· Users who have not left a comment on your blog.
· Or users who have not viewed a password protected post.

99% of your visitors will be served static html files. Those users who don’t see the static files will still benefit because they will see regular WP-Cache cached files and your server won’t be as busy as before. This plugin will help your server cope with a front page appearance on digg.com or other social networking site.

Why is WP-Super-Cache better than WP-Cache?

This plugin is based on the excellent WP-Cache plugin and therefore brings all the benefits of that plugin to WordPress. On top of that it creates copies of every page that is accessed on a blog in a form that is quickly served by the web server. It’s almost as quick as if the you had saved a page in your browser and uploaded it to replace your homepage.

Will the Super Cache compression slow down my server?

No, it will do the opposite in fact. Super Cache files are compressed and stored that way so the heavy compression is done only once. These files are generally much smaller and are sent to a visitor’s browser much more quickly than uncompressed html. As a result, your server spends less time talking over the network which saves CPU time and bandwidth, and can also serve the next request much more quickly.

AskApache Crazy Cache

AskApache Crazy Cache Plugin Page – News

AskApache Crazy CacheThis sweet little plugin does one thing very well. It caches all the posts on your entire blog at the same time, if you are using WP-Cache, WP-Super-Cache, or Hyper-Cache.

                              +---------+ ---------\      active OPEN
                              |  CLOSED |            \    -----------
                              +---------+<---------\   \   create TCB
                                |     ^              \   \  snd SYN
                   passive OPEN |     |   CLOSE        \   \
                   ------------ |     | ----------       \   \
                    create TCB  |     | delete TCB         \   \
                                V     |                      \   \
                              +---------+            CLOSE    |    \
                              |  LISTEN |          ---------- |     |
                              +---------+          delete TCB |     |
                   rcv SYN      |     |     SEND              |     |
                  -----------   |     |    -------            |     V
 +---------+      snd SYN,ACK  /       \   snd SYN          +---------+
 |         |<-----------------           ------------------>|         |
 |   SYN   |                    rcv SYN                     |   SYN   |
 |   RCVD  |<-----------------------------------------------|   SENT  |
 |         |                    snd ACK                     |         |
 |         |------------------           -------------------|         |
 +---------+   rcv ACK of SYN  \       /  rcv SYN,ACK       +---------+
   |           --------------   |     |   -----------
   |                  x         |     |     snd ACK
   |                            V     V
   |  CLOSE                   +---------+
   | -------                  |  ESTAB  |
   | snd FIN                  +---------+
   |                   CLOSE    |     |    rcv FIN
   V                  -------   |     |    -------
 +---------+          snd FIN  /       \   snd ACK          +---------+
 |  FIN    |<-----------------           ------------------>|  CLOSE  |
 | WAIT-1  |------------------                              |   WAIT  |
 +---------+          rcv FIN  \                            +---------+
   | rcv ACK of FIN   -------   |                            CLOSE  |
   | --------------   snd ACK   |                           ------- |
   V        x                   V                           snd FIN V
 +---------+                  +---------+                   +---------+
 |FINWAIT-2|                  | CLOSING |                   | LAST-ACK|
 +---------+                  +---------+                   +---------+
   |                rcv ACK of FIN |                 rcv ACK of FIN |
   |  rcv FIN       -------------- |    Timeout=2MSL -------------- |
   |  -------              x       V    ------------        x       V
    \ snd ACK                 +---------+delete TCB         +---------+
     ------------------------>|TIME WAIT|------------------>| CLOSED  |
                              +---------+                   +---------+
 
                      TCP Connection State Diagram

Vetted – Top 3 WordPress Speed Plugins originally appeared on AskApache.com

]]> http://www.askapache.com/wordpress/fastest-caching-plugins.html/feed/ 13 Optimizing Servers and Processes for Speed with ionice, nice, ulimit http://www.askapache.com/linux-unix/optimize-nice-ionice.html http://www.askapache.com/linux-unix/optimize-nice-ionice.html#comments Sat, 10 Oct 2009 05:41:28 +0000 AskApache http://www.askapache.com/?p=3167 To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.

]]> Ok, sup. I really felt I had to get this out of the way, because I have a whole stack of drafts waiting to be published, but I realized that not many people will benefit from all the advanced optimizations and tricks I’m writing unless they get a basic understanding of some of the tools I’m using. I decided to write a series of articles explaining how I optimize servers for speed because lately I’ve been getting a lot more people wanting to hire me to do that. I take on projects when I can but there is clearly a need out here on the net for some self-help. The momentum is swinging more and more towards VPS type of web hosting, and I would say that 99% of those customers are getting supremely ripped off, which goes against the foundation of the web.

Keep in mind that this blog and my research is only a hobby of mine, my job is primarily marketing and sales, so I’m not some licensed expert or anything, or even an unlicensed expert! haha. But it does bother me that those who are tech-savvy enough to run web-hosting companies are happily ripping people off. So this article details the main tools that are used to speed up and optimize your machine by delegating levels of priority to specific processes. Future articles will use these tools alot, so this is meant as an intro.

CPU and Disk I/O

As most of you are aware, there are 2 variables that determine any computer or programs speed. CPU and Disk I/O. CPU determines how fast you can process data, crunch numbers, etc. while disk I/O determines how fast your disks can read and write data to the hard-drive. Wouldn’t it be great if you could easily configure your server to give your httpd, php, and other processes both greater CPU processing and disk IO than your non-important processes like backup scripts, ftp daemons, etc.? We are talking about Linux in this article, so of course YES not only can you do that, you should!

RAM

RAM is like a hard-drive in that data is stored on it, and read/written to it. The difference is that RAM is somewhere around 30x faster than disk I/O, but the cost of that incredible speed is that the data stored on it is only temporary in the sense that it won’t be stored permanently, it is completely erased when your machine is rebooted. RAM is also expensive, and there is a limit to how much a server or machine can have due to hardware limits.

SWAP

SWAP takes off when you run out of RAM but you still want certain data to be read/write quickly. Basically when you start running out of RAM your machine starts supplementing RAM with SWAP storage. SWAP is usually a partition on a second hard-drive disk. There is an upper limit on how much I/O can occur on a disk at one time, and the more I/O takes place, the slower all I/O becomes, so SWAP works well on a separate hard-drive as it will have much faster I/O. On Windows they opted to copy the SWAP mechanism but instead use a file named pagefile.sys, and that is just one reason people in the know do not care for Windows.

CPU

So lets do this, think of your CPU (your processor) as having an amount of 100% processing available when not being used, 0% when its maxed out. CPU’s handle multiple processing tasks simultaneously, so what we will discuss in this article is how to specify HOW MUCH of that processing amount each of your programs (heretofore “processes”) are able to use. Yes, very very cool.

That is correct, you can easily configure your server to provide more of the available processing time to certain programs over others, like you can configure apache and php to utilize 50% of your CPU processing time by themselves, so that all other processes (proftpd, sshd, rsync, etc.) combined can only utilize 50%. The terminology is we can give certain specific processes (like php.cgi, httpd, fast-cgi.cgi) a specific priority, where -19 is the most priority, and +19 is the least amount of priority, or CPU processing time. I know it seems backwards..

The Tools

If you run Windows, you are in the right place… because the following advice will save your life: GET LINUX! Ok, now that that is out of the way, the following are the tools dicussed on this page. All of them are free, open-source, and wonderful. The basic idea of these tools is to control how much CPU is devoted to each process, and also how much Disk IO/Disk traffic is given to each process.

nice
run a program with modified scheduling priority
renice
alter priority of running processes
ionice
set or retrieve the I/O priority for a given pid or execute a new task with a given I/O priority.
iostat
Report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions.
ulimit
Ulimit provides control over the resources available to processes started by the shell, on systems that allow such control.
chrt
set or retrieve real-time scheduling parameters for a given pid or execute a new task under given scheduling parameters.
taskset
set or retrieve task CPU affinity for a given pid or execute a new task under a given affinity mask.

Part 1: Process Processes Faster

Ok so lets tackle figuring out how to give your response-intensive processes (like apache, php, ruby, perl, java) meaning a request to your server/machine requires a response. For instance, when you requested this page that you are reading at this very second, several things on my server had to happen for you to be able to read this.

First your computer sends out a request to see what server the www.askapache.com domain name is. DNS servers respond with my server IP, so for servers dedicated as nameservers, optimizing the DNS processes like bind would speed that up. Now that your computer knows how to reach my server it sends an HTTP GET request for this url. This request is received by the httpd process that is apache, and apache determines this url should be handled by my custom compiled php5.3.0 binary, because this page is WordPress generated. So the php binary loads up the WordPress /index.php file, which chain-loads several other php files, including wp-config.php containing my MySql database settings. Now php connects to my MySql Server to fetch this articles content, comments, title, tags, etc. and then generates the HTML and hands that back to Apache.

Finally, Apache generates a HTTP RESPONSE and sends the RESPONSE and CONTENT back to your Browser, which then in turn renders the page for your eyes with the necessary javascript, images, css, and other files included in the HTML response.

Too much Processing

Now you see why I’ve opted to write my own caching plugin that takes the php and mysql processes OUT of that equation. Both the php binary and the mysql instance consume CPU processing, and disk IO, to load all their library files, make various network requests and sockets, check permissions, and on and on. And that’s completely ok, the thing is, unless you configure these processes (Apache, PHP, MySQL) they will use the same amount of CPU processing that other processes use, other processes that have very little to do with you reading this sentence. Processes to run my mail server, my FTP server, my SSH server, my cronjobs, cleanup scripts, atd daemon, etc.. and they will get the same amount of CPU!

Another even simpler example is what got me to look into this myself. I wrote a shell script that created hourly, daily, weekly, and monthly backups for all of my websites and sql databases, and set it up to run by cronjob at those set intervals. Eventually I noticed my sites were slower, my php even slower, and sometimes I even saw 503 errors that my host throws up when my server is overloaded. The research that I pursued to prevent that from happening has been hugely eye-opening. What does a backup script do? Mine just created tar archives of all the files in my web root, then gzipped the tar archive saving to a backup server using scp (a file transfer using ssh). This resulted in the following huge problems that seem to have nothing to do with a faster server and speedier website, but they have everything to with it.

  1. CPU Bottleneck #1 – tar and gzip use compression algorithms at a low level to create a compressed version, and all that compressing uses a whole lot of crunching – CPU processing
  2. DISK IO Bottleneck – Tarring the whole web root directory was creating a ton of disk io, and remember the more disk io that is going on, the less is available for everything else.
  3. CPU Bottleneck #2 – Using scp to send my backups was security-smart, but these huge archive files had to be encrypted and sent over the net.

Breaking Bottles

I apologize for being a little long-winded there, but I think it’s important to make sure everyone understands those basic concepts, which are foreign to most people. Once you understand what is causing the bottlenecks, then you can understand the solutions, which actually are incredibly simple and even a novice linux user can easily do. Besides, the net gets a little bit faster every time someone implements this.

nice

NICE Levels ChartNice allows you to run a program with modified scheduling priority which specifies how much CPU is devoted to a particular process. Run COMMAND with an adjusted niceness, which affects process scheduling. With no COMMAND, print the current niceness.

Nicenesses range from -20 (most favorable scheduling) to 19 (least favorable). -n, --adjustment=N – add integer N to the niceness (default 10). nice +19 tasks get a HZ-independent 1.5%. Running a nice +10 and a nice +11 task means the first will get 55% of the CPU, the other 45%.

nice usage

nice [OPTION] [COMMAND [ARG]...]
 
-n, --adjustment=ADJUST   increment priority by ADJUST first

Examples of nice

Using nice to download a file

nice -n 17 curl -q -v -A 'Mozilla/5.0' -L -O http://wordpress.org/latest.zip

Unzipping a file with nice

nice -n 17 unzip latest.zip

Nice way to build from source

nice -n 2 ./configure
nice -n 2 make
nice -n 2 make install

It is sometimes useful to run non-interactive programs with reduced priority.

$ nice factor `echo '2^9 - 1'|bc`
511: 7 73

Since nice prints the current priority, we can invoke it through itself to demonstrate how it works: The default behavior is to reduce priority by 10.

 $ nice nice
10
$ nice -n 10 nice
10

The ADJUSTMENT is relative to the current priority. The first nice invocation runs the second one at priority 10, and it in turn runs the final one at a priority lowered by 3 more.

$ nice nice -n 3 nice
13

Specifying a priority larger than 19 is the same as specifying 19.

$ nice -n 30 nice
19

Only a privileged user may run a process with higher priority.

$ nice -n -1 nice
nice: cannot set priority: Permission denied
$ sudo nice -n -1 nice
-1

The new scheduler in v2.6.23 addresses all three types of complaints:

To address the first complaint (of nice levels being not “punchy” enough), the scheduler was decoupled from ‘time slice’ and HZ concepts (and granularity was made a separate concept from nice levels) and thus it was possible to implement better and more consistent nice +19 support: with the new scheduler nice +19 tasks get a HZ-independent 1.5%, instead of the variable 3%-5%-9% range they got in the old scheduler.

To address the second complaint (of nice levels not being consistent), the new scheduler makes nice(1) have the same CPU utilization effect on tasks, regardless of their absolute nice levels. So on the new scheduler, running a nice +10 and a nice 11 task has the same CPU utilization “split” between them as running a nice -5 and a nice -4 task. (one will get 55% of the CPU, the other 45%.) That is why nice levels were changed to be “multiplicative” (or exponential) – that way it does not matter which nice level you start out from, the ‘relative result’ will always be the same.

The third complaint (of negative nice levels not being “punchy” enough and forcing audio apps to run under the more dangerous SCHED_FIFO scheduling policy) is addressed by the new scheduler almost automatically: stronger negative nice levels are an automatic side-effect of the recalibrated dynamic range of nice levels.

renice

Renice is similar to the nice command, but it lets you modify the nice of a currently running process. This is nice for shell scripts where you can add this to the top of the script to nicify the whole script to 19.

renice usage

renice priority [ [ -p ] pids ] [ [ -g ] pgrps ] [ [ -u ] users ]
 
-g      Force who parameters to be interpreted as process group ID's.
-u      Force the who parameters to be interpreted as user names.
-p      Resets the who interpretation to be (the default) process ID's.

Examples of renice

From the shell, changes the priority of the shell and all children to 19. From a shell script, does the same but only for the script and its children.

renice 19 -p $$

This runs renice without any output

renice 19 -p $$ &>/dev/null

10 gets more CPU than 19

renice 10 -p $$

change the priority of process ID’s 987 and 32, and all processes owned by users daemon and root.

renice +1 987 -u daemon root -p 32

Part 2: Optimizing Disk I/O

Linux Scheduling Policies

The scheduler is the kernel component that decides which runnable process will be executed by the CPU next. Each process has an associated scheduling policy and a static scheduling priority, sched_priority

Processes scheduled under one of the real-time policies (SCHED_FIFO, SCHED_RR) have a sched_priority value in the range 1 (low) to 99 (high). (As the numbers imply, real-time processes always have higher priority than normal processes.) The following “real-time” policies are also supported, for special time-critical applications that need precise control over the way in which runnable processes are selected for execution:

Currently, Linux supports the following “normal” (i.e., non-real-time) scheduling policies:

SCHED_OTHER: Default Linux time-sharing scheduling
The standard round-robin time-sharing policy
SCHED_BATCH: Scheduling batch processes
This policy is useful for workloads that are non-interactive, but do not want to lower their nice value, and for workloads that want a deterministic scheduling policy without interactivity causing extra preemptions (between the workload’s tasks).
SCHED_IDLE: Scheduling very low priority jobs
This policy is intended for running jobs at extremely low priority (lower even than a +19 nice value with the SCHED_OTHER or SCHED_BATCH policies)
SCHED_FIFO: First In-First Out scheduling
A first-in, first-out policy
SCHED_RR: Round Robin scheduling
A round-robin policy.

Scheduling Classes

IOPRIO_CLASS_RT
This is the realtime io class. The RT scheduling class is given first access to the disk, regardless of what else is going on in the system. Thus the RT class needs to be used with some care, as it can starve other processes. As with the best effort class, 8 priority levels are defined denoting how big a time slice a given process will receive on each scheduling window. This scheduling class is given higher priority than any other in the system, processes from this class are given first access to the disk every time. Thus it needs to be used with some care, one io RT process can starve the entire system. Within the RT class, there are 8 levels of class data that determine exactly how much time this process needs the disk for on each service. In the future this might change to be more directly mappable to performance, by passing in a wanted data rate instead.
IOPRIO_CLASS_BE
This is the best-effort scheduling class, which is the default for any process that hasn’t set a specific io priority. This is the default scheduling class for any process that hasn’t asked for a specific io priority. Programs inherit the CPU nice setting for io priorities. This class takes a priority argument from 0-7, with lower number being higher priority. Programs running at the same best effort priority are served in a round-robin fashion. The class data determines how much io bandwidth the process will get, it’s directly mappable to the cpu nice levels just more coarsely implemented. 0 is the highest BE prio level, 7 is the lowest. The mapping between cpu nice level and io nice level is determined as: io_nice = (cpu_nice + 20) / 5.
IOPRIO_CLASS_IDLE
This is the idle scheduling class, processes running at this level only get io time when no one else needs the disk. A program running with idle io priority will only get disk time when no other program has asked for disk io for a defined grace period. The impact of idle io processes on normal system activity should be zero. This scheduling class does not take a priority argument. The idle class has no class data, since it doesn’t really apply here.

ionice

ionice – get/set program io scheduling class and priority. This program sets the io scheduling class and priority for a program. Since v3 (aka CFQ Time Sliced) CFQ implements I/O nice levels similar to those of CPU scheduling. These nice levels are grouped in three scheduling classes each one containing one or more priority levels:

ionice usage

If no arguments or just -p is given, ionice will query the current io scheduling class and priority for that process.

ionice [-c] [-n] [-p] [COMMAND [ARG...]]
  • -c – The scheduling class. 1 for real time, 2 for best-effort, 3 for idle.
  • -n – The scheduling class data. This defines the class data, if the class accepts an argument. For real time and best-effort, 0-7 is valid data.
  • -p – Pass in a process pid to change an already running process. If this argument is not given, ionice will run the listed program with the given parameters.

ionice Examples

Sets process with PID 89 as an idle io process.

ionice -c3 -p89

Runs ‘bash’ as a best-effort program with highest priority.

ionice -c2 -n0 bash

Returns the class and priority of the process with PID 89

ionice -p89

With the ionice command, you can set the IO priority for a process to one of three classes: Idle (3), Best Effort (2), and Real Time (1). The Idle class means that the process will only be able to read and write to the disk when all other processes are not using the disk. The Best Effort class is the default and has eight different priority levels from 0 (top priority) to 7 (lowest priority). The Real Time class results in the process having first access to the disk irregardless of other process and should never be used unless you know what you are doing.

If we wish to run the updatedb process in the background with an Idle IO class priority, we can run the following:

$ sudo date
$ sudo updatedb &
[1] 16324
$ sudo ionice -c3 -p16324

If we’d rather just lower the Best Effort class priority (defaults to 4) for the command so the process isn’t limited to idle IO periods, we can run the following:

$ sudo date
$ sudo updatedb &
[1] 16324
$ sudo ionice -c2 -n7 -p16324

Again, the Real Time class should not be used as it can prevent you from being able to interact with your system.

You may wonder where you can get the process ID if you don’t know it, can’t remember it, or didn’t start the process (an automatted script may have launched it). You can find process IDs with the ps command.

For example, if I had an updatedb program running in the background, and I wanted to find its process ID, I can run the following:

$ ps -C updatedb
PID TTY TIME CMD
4234 ? 00:00:42 updatedb

This tells me that the process’ process ID (PID) is 4234.

iostat

iostat Usage

iostat [ -c ] [ -d ] [ -N ] [ -n ] [ -h ] [ -k | -m ] [ -t ] [ -V ] [ -x ] [ -z ] [ <device> [...] | ALL ] [ -p [ <device> [,...] | ALL ] ] [ <interval> [ <count> ] ]

-c     The -c option is exclusive of the -d option and displays only the CPU usage report.
-d     The -d option is exclusive of the -c option and displays only the device utilization report.
-k     Display statistics in kilobytes per second instead of blocks per second.  Data displayed are valid only with kernels 2.4 and newer.
-m     Display statistics in megabytes per second instead of blocks or kilobytes per second.  Data displayed are valid only with kernels 2.4 and newer.
-n     Displays the NFS-directory statistic.  Data displayed are valid only with kernels 2.6.17 and newer.  This option is exclusive ot the -x option.
-h     Display the NFS report more human readable.
-p [ { device | ALL } ]   The  -p  option  is  exclusive  of  the -x option and displays statistics for block devices and all their partitions that are used by the system.
-t     Print the time for each report displayed.
-x     Display extended statistics.

iostat Examples

iostat -p ALL 2 1000
avg-cpu:  %user   %nice    %sys %iowait   %idle
            8.34    0.08    1.26    2.27   88.05

Display a single history since boot report for all CPU and Devices.

$ iostat

Display a continuous device report at two second intervals.

$ iostat -d 2

Display six reports at two second intervals for all devices.

$ iostat -d 2 6

Display six reports of extended statistics at two second intervals for devices hda and hdb.

$ iostat -x hda hdb 2 6

Display six reports at two second intervals for device sda and all its partitions (sda1, etc.)

$ iostat -p sda 2 6

Schedule Utils

These are the Linux scheduler utilities – schedutils for short. These programs take advantage of the scheduler family of syscalls that Linux implements across various kernels. These system calls implement interfaces for scheduler-related parameters such as CPU affinity and real-time attributes. The standard UNIX utilities do not provide support for these interfaces — thus this package.

The programs that are included in this package are chrt and taskset. Together with nice and renice (not included), they allow full control of process scheduling parameters. Suggestions for related utilities are welcome, although it is believed (barring new interfaces) that all scheduling interfaces are covered.

I’ve found that quite a few servers do not have this package installed, indicating to you that they might not know what they are doing. Here is how you can install this incredible package, for non-root users. Root users know how to do this, or they shouldn’t be root. Download and install in 1 line provided you have curl. Or just use the following commands.

mkdir -pv $HOME/{dist,source,bin,share/man/man1} && cd ~/dist && curl -O http://ftp.de.debian.org/debian/pool/main/s/schedutils/schedutils_1.5.0.orig.tar.gz && cd ~/source && tar -xvzf ~/dist/sch*z && cd sch* && sed -i -e 's,= /usr/local,=${HOME},g' Makefile && make && make install && make installdoc
mkdir -pv $HOME/{dist,source,bin,share/man/man1}
cd ~/dist && curl -O http://ftp.de.debian.org/debian/pool/main/s/schedutils/schedutils_1.5.0.orig.tar.gz
cd ~/source && tar -xvzf ~/dist/schedutils_1.5.0.orig.tar.gz
cd ~/source/schedutils-1.5.0 && sed -i -e 's,= /usr/local,=${HOME},g' Makefile
make || make -d && make install || make install -d && make installdoc || make installdoc -d

taskset

Taskset is used to set or retrieve the CPU affinity of a running process given its PID or to launch a new COMMAND with a given CPU affinity. CPU affinity is a scheduler property that “bonds” a process to a given set of CPUs on the system. The Linux scheduler will honor the given CPU affinity and the process will not run on any other CPUs. Note that the Linux scheduler also supports natural CPU affinity: the scheduler attempts to keep processes on the same CPU as long as practical for performance reasons. Therefore, forcing a specific CPU affinity is useful only in certain applications.

The CPU affinity is represented as a bitmask, with the lowest order bit corresponding to the first logical CPU and the highest order bit corresponding to the last logical CPU. Not all CPUs may exist on a given system but a mask may specify more CPUs than are present. A retrieved mask will reflect only the bits that correspond to CPUs physically on the system. If an invalid mask is given (i.e., one that corresponds to no valid CPUs on the current system) an error is returned. A user must possess CAP_SYS_NICE to change the CPU affinity of a process. Any user can retrieve the affinity mask.

taskset Usage

taskset [options] [mask | cpu-list] [pid | cmd [args...]]
 
-p, --pid            operate on existing given pid
-c, --cpu-list     display and specify cpus in list format

taskset-examples

The default behavior is to run a new command:

$ taskset 03 sshd -b 1024

You can retrieve the mask of an existing task or set it:

$ taskset -p 700
$ taskset -p 03 700

List format uses a comma-separated list instead of a mask:

$ taskset -pc 0,3,7-11 700

chrt

chrt sets or retrieves the real-time scheduling attributes of an existing PID or runs COMMAND with the given attributes. Both policy (one of SCHED_FIFO, SCHED_RR, or SCHED_OTHER) and priority can be set and retrieved. A user must possess CAP_SYS_NICE to change the scheduling attributes of a process. Any user can retrieve the scheduling information.

chrt Usage

chrt [options] [prio] [pid | cmd [args...]]
 
-p, --pid operate on an existing PID and do not launch a new task
-f, --fifo set scheduling policy to SCHED_FIFO
-m, --max show minimum and maximum valid priorities, then exit
-o, --other set policy scheduling policy to SCHED_OTHER
-r, --rr set scheduling policy to SCHED_RR (the default)

chrt Examples

The default behavior is to run a new command: chrt [prio] -- [command] [arguments]

You can also retrieve the real-time attributes of an existing task:

chrt -p [pid]

Or set them:

chrt -p [prio] [pid]

ulimit – get and set user limits

Ulimit provides control over the resources available to processes started by the shell, on systems that allow such control. One can set the resource limits of the shell using the built-in ulimit command. The shell’s resource limits are inherited by the processes that it creates to execute commands.

ulimit Usage

ulimit [-SHacdfilmnpqstuvx] [limit]
-S
use the `soft’ resource limit
-H
use the `hard’ resource limit
-a
all current limits are reported
-c
the maximum size of core files created
-d
the maximum size of a process’s data segment
-f
the maximum size of files created by the shell
-l
the maximum size a process may lock into memory
-m
the maximum resident set size
-n
the maximum number of open file descriptors
-p
the pipe buffer size
-s
the maximum stack size
-t
the maximum amount of cpu time in seconds
-u
the maximum number of user processes
-v
the size of virtual memory

If LIMIT is given, it is the new value of the specified resource; the special LIMIT values `soft’, `hard’, and `unlimited’ stand for the current soft limit, the current hard limit, and no limit, respectively. Otherwise, the current value of the specified resource is printed. If no option is given, then -f is assumed. Values are in 1024-byte increments, except for -t, which is in seconds, -p, which is in increments of 512 bytes, and -u, which is an unscaled number of processes.

RLIMIT_AS
The maximum size of the process’s virtual memory (address space) in bytes. This limit affects calls to brk(2), mmap(2) and mremap(2), which fail with the error ENOMEM upon exceeding this limit. Also automatic stack expansion will fail (and generate a SIGSEGV that kills the process if no alternate stack has been made available via sigaltstack(2)). Since the value is a long, on machines with a 32-bit long either this limit is at most 2 GiB, or this resource is unlimited.
RLIMIT_CORE
Maximum size of core file. When 0 no core dump files are created. When non-zero, larger dumps are truncated to this size.
RLIMIT_CPU CPU
time limit in seconds. When the process reaches the soft limit, it is sent a SIGXCPU signal. The default action for this signal is to terminate the process. However, the signal can be caught, and the handler can return control to the main program. If the process continues to consume CPU time, it will be sent SIGXCPU once per second until the hard limit is reached, at which time it is sent SIGKILL. (This latter point describes Linux 2.2 through 2.6 behavior. Implementations vary in how they treat processes which continue to consume CPU time after reaching the soft limit. Portable applications that need to catch this signal should perform an orderly termination upon first receipt of SIGXCPU.)
RLIMIT_DATA
The maximum size of the process’s data segment (initialized data, uninitialized data, and heap). This limit affects calls to brk(2) and sbrk(2), which fail with the error ENOMEM upon encountering the soft limit of this resource.
RLIMIT_FSIZE
The maximum size of files that the process may create. Attempts to extend a file beyond this limit result in delivery of a SIGXFSZ signal. By default, this signal terminates a process, but a process can catch this signal instead, in which case the relevant system call (e.g., write(2), truncate(2)) fails with the error EFBIG.
RLIMIT_LOCKS
(Early Linux 2.4 only) A limit on the combined number of flock(2) locks and fcntl(2) leases that this process may establish.
RLIMIT_MEMLOCK
The maximum number of bytes of memory that may be locked into RAM. In effect this limit is rounded down to the nearest multiple of the system page size. This limit affects mlock(2) and mlockall(2) and the mmap(2) MAP_LOCKED operation. Since Linux 2.6.9 it also affects the shmctl(2) SHM_LOCK operation, where it sets a maximum on the total bytes in shared memory segments (see shmget(2)) that may be locked by the real user ID of the calling process. The shmctl(2) SHM_LOCK locks are accounted for separately from the per-process memory locks established by mlock(2), mlockall(2), and mmap(2) MAP_LOCKED; a process can lock bytes up to this limit in each of these two categories. In Linux kernels before 2.6.9, this limit controlled the amount of memory that could be locked by a privileged process. Since Linux 2.6.9, no limits are placed on the amount of memory that a privileged process may lock, and this limit instead governs the amount of memory that an unprivileged process may lock.
RLIMIT_MSGQUEUE
(Since Linux 2.6.8) Specifies the limit on the number of bytes that can be allocated for POSIX message queues for the real user ID of the calling process. This limit is enforced for mq_open(3). Each message queue that the user creates counts (until it is removed) against this limit according to the formula: bytes = attr.mq_maxmsg * sizeof(struct msg_msg *) + attr.mq_maxmsg * attr.mq_msgsize where attr is the mq_attr structure specified as the fourth argument to mq_open(3). The first addend in the formula, which includes sizeof(struct msg_msg *) (4 bytes on Linux/i386), ensures that the user cannot create an unlimited number of zero-length messages (such messages nevertheless each consume some system memory for bookkeeping overhead).
RLIMIT_NICE
(since Linux 2.6.12, but see BUGS below) Specifies a ceiling to which the process’s nice value can be raised using setpriority(2) or nice(2). The actual ceiling for the nice value is calculated as 20 – rlim_cur. (This strangeness occurs because negative numbers cannot be specified as resource limit values, since they typically have special meanings. For example, RLIM_INFINITY typically is the same as -1.)
RLIMIT_NOFILE
Specifies a value one greater than the maximum file descriptor number that can be opened by this process. Attempts (open(2), pipe(2), dup(2), etc.) to exceed this limit yield the error EMFILE. (Historically, this limit was named RLIMIT_OFILE on BSD.)
RLIMIT_NPROC
The maximum number of processes (or, more precisely on Linux, threads) that can be created for the real user ID of the calling process. Upon encountering this limit, fork(2) fails with the error EAGAIN.
RLIMIT_RSS
Specifies the limit (in pages) of the process’s resident set (the number of virtual pages resident in RAM). This limit only has effect in Linux 2.4.x, x < 30, and there only affects calls to madvise(2) specifying MADV_WILLNEED.
RLIMIT_RTPRIO
(Since Linux 2.6.12, but see BUGS) Specifies a ceiling on the real-time priority that may be set for this process using sched_setscheduler(2) and sched_setparam(2).
RLIMIT_RTTIME
(Since Linux 2.6.25) Specifies a limit on the amount of CPU time that a process scheduled under a real-time scheduling policy may consume without making a blocking system call. For the purpose of this limit, each time a process makes a blocking system call, the count of its consumed CPU time is reset to zero. The CPU time count is not reset if the process continues trying to use the CPU but is preempted, its time slice expires, or it calls sched_yield(2). Upon reaching the soft limit, the process is sent a SIGXCPU signal. If the process catches or ignores this signal and continues consuming CPU time, then SIGXCPU will be generated once each second until the hard limit is reached, at which point the process is sent a SIGKILL signal. The intended use of this limit is to stop a runaway real-time process from locking up the system.
RLIMIT_SIGPENDING
(Since Linux 2.6.8) Specifies the limit on the number of signals that may be queued for the real user ID of the calling process. Both standard and real-time signals are counted for the purpose of checking this limit. However, the limit is only enforced for sigqueue(2); it is always possible to use kill(2) to queue one instance of any of the signals that are not already queued to the process.
RLIMIT_STACK
The maximum size of the process stack, in bytes. Upon reaching this limit, a SIGSEGV signal is generated. To handle this signal, a process must employ an alternate signal stack (sigaltstack(2)).

ulimit Examples

Turn off core dumps

ulimit -S -c 0

More Reading

Man Pages

  1. sched_setscheduler
  2. cpuset
  3. signal
  4. getrlimit
  5. ulimit
  6. ioprio_get
  7. ioprio_set

Kernel Documentation

Future Discussions:

IO Benchmarking: How, Why and With What

Optimizing Servers and Processes for Speed with ionice, nice, ulimit originally appeared on AskApache.com

]]> http://www.askapache.com/linux-unix/optimize-nice-ionice.html/feed/ 3 Protecting Files with Advanced Mod_Rewrite Anti-Hotlinking http://www.askapache.com/htaccess/anti-hotlinking-mod_rewrite.html http://www.askapache.com/htaccess/anti-hotlinking-mod_rewrite.html#comments Wed, 16 Sep 2009 05:09:11 +0000 AskApache http://www.askapache.com/?p=3287 If you have files on your site that you don't want indexed by malicious search engines, grabbed and leeched by malicious spammers, or stolen and made available elsewhere, you can use mod_rewrite to drastically reduce or totally reduce that activity.

]]>

If you have files on your site that you don’t want indexed by malicious search engines, grabbed and leeched by malicious spammers, or stolen and made available elsewhere, you can use mod_rewrite to drastically reduce or totally reduce that activity.

The Worst Kind of People

Spammers, and Leechers. They operate like this: Let’s say you have some mp3 files on a server, and SOMEWHERE on the web there is a link to that mp3 file’s location. This includes in javascript files, css files, robots.txt files, the spammers and leechers robots check all those files looking for the type of link they are looking for. Then they try to request that file usually utilizing a number of different types of requests to get access to the file. Then they use it for personal gain, at your peril.

Some robots perform valuable services for the world wide web community, and other leeching programming is pretty cool, so not all these activities are perpetrated by nefarious spammers.


Blocking by IP COOKIE

Ok so if a link exists to your file, it is going to be requested by a robot eventually, so the way to defeat them is by doing something on your site that modifies the way a user would request it. Robots for the most part are not javascript-capable, so the most-common advanced method is to set a cookie using javascript, and then we can check for that cookie in the request for the file using mod_rewrite.

So if your site sets a cookie named fspammers, and furthermore gives that cookie a value of 445, then this is what the request sent by an HTTP Client like Firefox looks like.

GET /hotlink/lovefreedom.mp3 HTTP/1.1
Host: static.askapache.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.askapache.com/wordpress/seo-in-wordpress.html
Cookie: fspammers=455

Mod_Rewrite HTTP Headers

The mod_rewrite module has access to ALL the HTTP Headers sent in a request, so for each of the HTTP Headers in the request example above, we can use mod_rewrite to validate.

Mod_Rewrite .htaccess Example

Finally, now that everyone is on the same page about what is really going on, here is the .htaccess code that blocks any requests for anything in the /hotlink/ folder.

Here are the triggers this code blocks access based on.

  1. Cookie: Checks if fspammers cookie is present, and that it has the value of 445.
  2. HTTP Protocol: Checks if HTTP 1.1 is being used (many robots use 1.0)
  3. Host: Checks that the HOST Requested was static.askapache.com
  4. Referer: Checks for Referring site is static.askapache.com or www.askapache.com
RewriteEngine On
RewriteBase /
 
RewriteCond %{HTTP_COOKIE} !^.*fspammers=445.*$ [NC,OR]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /(.*)\ HTTP/1\.1 [NC,OR]
RewriteCond %{HTTP_HOST} !^z\.askapache\.com$ [NC,OR]
RewriteCond %{HTTP_REFERER} !^http://(www|z)\.askapache\.com.*$ [NC]
RewriteRule ^hotlink/.*$ - [F]</p>

Protecting Files with Advanced Mod_Rewrite Anti-Hotlinking originally appeared on AskApache.com

]]> http://www.askapache.com/htaccess/anti-hotlinking-mod_rewrite.html/feed/ 1 Crazy Advanced Mod_Rewrite Tutorial http://www.askapache.com/htaccess/crazy-advanced-mod_rewrite-tutorial.html http://www.askapache.com/htaccess/crazy-advanced-mod_rewrite-tutorial.html#comments Fri, 11 Sep 2009 16:55:32 +0000 AskApache http://www.askapache.com/?p=1209 Note: Extremely ILL Content
Find the key to unlocking mod_rewrite and you WILL be sick.. sick with a diamond disease on your wrist!

]]> Are you an advanced mod_rewrite expert or guru? This article is for YOU too! Just make sure to read all the way to the bottom..

The following undocumented techniques and methods will allow you to utilize mod_rewrite at an “expert level” by showing you how to unlock its secrets.

Most if not all web developers and server administrators struggle with Apache mod_rewrite. It’s very tough and only gets a little easier with practice. Until Now! Get ready to explode your learning curve, I figured something out.

Why mod_rewrite is so tough

I have come to the conclusion, after many hours of zenful thought, that the reason mod_rewrite is so tough is pretty obvious, people are trying to apply regular-expressions to URLs and Variables that they don’t really understand. They understand what they want, but they don’t understand what the URLS and Variables are that they are trying to rewrite.

Hit-Or-Miss with mod_rewrite

A lot of the mod_rewrite “experts” and “gurus” floating around the net absolutely know their mod_rewrite, but what separates them from a beginner or novice is for the most part an understanding of what the URLS and Variables look like that are targeted by the regular expressions. Take this simple rewriterule that rewrites requests made without the www to www.

RewriteEngine On
RewriteBase /
 
RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC]
RewriteRule .+ http://www.askapache.com%{REQUEST_URI}

Pretty simple right? WRONG. Most people could not figure that out..

Why?

The reason intelligent people can’t figure that out is because they have no idea what HTTP_HOST or REQUEST_URI actually looks like. How can you write a rule for something if you don’t know what it looks like? You can’t.

When Not To Use Mod_Rewrite

Ok so heres an important concept that alot of people haven’t heard. You should only use mod_rewrite’s rewriterule when you use a rewritecond or if you are rewriting internally like my feedcount hack.

If you are simply redirecting one url to another, you should definately be using the much easier mod_alias’s redirect and redirectmatch, which is enabled on most Apache servers.

When To Use Mod_Rewrite

So then, you should only use mod_rewrite’s rewriterule when you are checking against one of the Apache Environment Variables to determine whether to rewrite or not. This is where the Apache Documentation is grossly lacking. They don’t tell you what those variables look like, leaving us completely incapable of creating rewrites based on them. Not anymore.

Mod_Rewrite Environment Variables (The Secret)

Here’s the variables I have found accessible by mod_rewrite (both documented and undocumented). A thing to note is that you can set these variables early in an .htaccess file using SetEnv, RewriteRule, Header, etc.. and they will be accessible at the end of the .htaccess file.

  • API_VERSION
  • AUTH_TYPE
  • CONTENT_LENGTH
  • CONTENT_TYPE
  • DOCUMENT_ROOT
  • GATEWAY_INTERFACE
  • HTTPS
  • HTTP_ACCEPT
  • HTTP_ACCEPT_CHARSET
  • HTTP_ACCEPT_ENCODING
  • HTTP_ACCEPT_LANGUAGE
  • HTTP_CACHE_CONTROL
  • HTTP_CONNECTION
  • HTTP_COOKIE
  • HTTP_FORWARDED
  • HTTP_HOST
  • HTTP_KEEP_ALIVE
  • HTTP_PROXY_CONNECTION
  • HTTP_REFERER
  • HTTP_USER_AGENT
  • IS_SUBREQ
  • ORIG_PATH_INFO
  • ORIG_PATH_TRANSLATED
  • ORIG_SCRIPT_FILENAME
  • ORIG_SCRIPT_NAME
  • PATH
  • PATH_INFO
  • PHP_SELF
  • QUERY_STRING
  • REDIRECT_QUERY_STRING
  • REDIRECT_REMOTE_USER
  • REDIRECT_STATUS
  • REDIRECT_URL
  • REMOTE_ADDR
  • REMOTE_HOST
  • REMOTE_IDENT
  • REMOTE_PORT
  • REMOTE_USER
  • REQUEST_FILENAME
  • REQUEST_METHOD
  • REQUEST_TIME
  • REQUEST_URI
  • SCRIPT_FILENAME
  • SCRIPT_GROUP
  • SCRIPT_NAME
  • SCRIPT_URI
  • SCRIPT_URL
  • SCRIPT_USER
  • SERVER_ADDR
  • SERVER_ADMIN
  • SERVER_NAME
  • SERVER_PORT
  • SERVER_PROTOCOL
  • SERVER_SIGNATURE
  • SERVER_SOFTWARE
  • THE_REQUEST
  • TIME
  • TIME_DAY
  • TIME_HOUR
  • TIME_MIN
  • TIME_MON
  • TIME_SEC
  • TIME_WDAY
  • TIME_YEAR
  • TZ
  • UNIQUE_ID

Decoding Mod_Rewrite Variables

So when I realized my problem was that I didn’t know the value of the variable being tested by the RewriteCond, I set out to try and discover how to view those variables.. Keep in mind you can also use RewriteLogging, but its only allowed for root users who can edit the httpd.conf, this is .htaccess.

Setting Environment Variables with RewriteRule

I discovered a multitude of methods to set and view apache environment variables, using various modules and some core tricks, but the method that allows me to view the most environment variables is RewriteRule.. I wanted to use SetEnvIf more, but its just not as powerful as mod_rewrite, due to programming.

This code sets the variable INFO_REQUEST_URI to have the value of REQUEST_URI.

RewriteEngine On
RewriteBase /
RewriteRule .* - [E=INFO_REQUEST_URI:%{REQUEST_URI},NE]

Saving the Apache Variable Values

Now the trick is how to view that environment variable… The method I came up with is nice… We will send the environment variable value in an HTTP Header, as there isn’t much data manipulation/validation so you get an accurate look at the actual value.. At first I tried adding the variable value to a redirection using the query_string.. but a HTTP_USER_AGENT value doesn’t play well as a query_string.

Using RequestHeader in .htaccess

This code takes advantage of the incredible mod_headers apache module to actually ADD a whole new header to YOUR request. Seriously one of the coolest tricks I’ve found yet.. Its almost the same as being able to spoof POST requests! Since Headers can be protected data… especially the HTTP_COOKIE header..

RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"

Viewing the Variable Values

Now you can use any kind of server-run interpreter like perl, php, ruby, etc., to view all the variable values. All cgi-script handlers like those are able to view request headers..

PHP Code to access Apache Variables

Works even in safe-mode… any interpreter can view HTTP Headers! Note that each of these variables are added as HTTP headers to the request for the script.. kinda confusing.. So each variable sent as a header is prefixed with HTTP_ to denote it was a header.

<?php
header("Content-Type: text/plain");
$INFO=$MISS=array();
foreach($_SERVER as $v=>$r)
{
  if(substr($v,0,9)=='HTTP_INFO')
  {
    if(!empty($r))$INFO[substr($v,10)]=$r;
    else $MISS[substr($v,10)]=$r;
  }
}
 
/* thanks Mike! */
ksort($INFO);
ksort($MISS);
ksort($_SERVER);
 
echo "Received These Variables:\n";
print_r($INFO);
 
echo "Missed These Variables:\n";
print_r($MISS);
 
echo "ALL Variables:\n";
print_r($_SERVER);
?>

Time to Get Crazy

Just create the above php file on your site as /test/index.php or whatever, then create /test/.htaccess which should contain the below .htaccess file snippet. Now just request /test/index.php and be amazed! If you’re looking for more general help check out this excellent mod_rewrite cheat sheet.

Ok, so I’ve prepared the .htaccess code you can use to view the values of all these variables. Just add it to a .htaccess file and make a request. For this test I created an index.php file that printed out all the $_SERVER variables, and made requests to it.

RewriteEngine On
RewriteBase /
RewriteRule .* - [E=INFO_API_VERSION:%{API_VERSION},NE]
RewriteRule .* - [E=INFO_AUTH_TYPE:%{AUTH_TYPE},NE]
RewriteRule .* - [E=INFO_CONTENT_LENGTH:%{CONTENT_LENGTH},NE]
RewriteRule .* - [E=INFO_CONTENT_TYPE:%{CONTENT_TYPE},NE]
RewriteRule .* - [E=INFO_DOCUMENT_ROOT:%{DOCUMENT_ROOT},NE]
RewriteRule .* - [E=INFO_GATEWAY_INTERFACE:%{GATEWAY_INTERFACE},NE]
RewriteRule .* - [E=INFO_HTTPS:%{HTTPS},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT:%{HTTP_ACCEPT},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_CHARSET:%{HTTP_ACCEPT_CHARSET},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_ENCODING:%{HTTP_ACCEPT_ENCODING},NE]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_LANGUAGE:%{HTTP_ACCEPT_LANGUAGE},NE]
RewriteRule .* - [E=INFO_HTTP_CACHE_CONTROL:%{HTTP_CACHE_CONTROL},NE]
RewriteRule .* - [E=INFO_HTTP_CONNECTION:%{HTTP_CONNECTION},NE]
RewriteRule .* - [E=INFO_HTTP_COOKIE:%{HTTP_COOKIE},NE]
RewriteRule .* - [E=INFO_HTTP_FORWARDED:%{HTTP_FORWARDED},NE]
RewriteRule .* - [E=INFO_HTTP_HOST:%{HTTP_HOST},NE]
RewriteRule .* - [E=INFO_HTTP_KEEP_ALIVE:%{HTTP_KEEP_ALIVE},NE]
RewriteRule .* - [E=INFO_HTTP_MOD_SECURITY_MESSAGE:%{HTTP_MOD_SECURITY_MESSAGE},NE]
RewriteRule .* - [E=INFO_HTTP_PROXY_CONNECTION:%{HTTP_PROXY_CONNECTION},NE]
RewriteRule .* - [E=INFO_HTTP_REFERER:%{HTTP_REFERER},NE]
RewriteRule .* - [E=INFO_HTTP_USER_AGENT:%{HTTP_USER_AGENT},NE]
RewriteRule .* - [E=INFO_IS_SUBREQ:%{IS_SUBREQ},NE]
RewriteRule .* - [E=INFO_ORIG_PATH_INFO:%{ORIG_PATH_INFO},NE]
RewriteRule .* - [E=INFO_ORIG_PATH_TRANSLATED:%{ORIG_PATH_TRANSLATED},NE]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_FILENAME:%{ORIG_SCRIPT_FILENAME},NE]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_NAME:%{ORIG_SCRIPT_NAME},NE]
RewriteRule .* - [E=INFO_PATH:%{PATH},NE]
RewriteRule .* - [E=INFO_PATH_INFO:%{PATH_INFO},NE]
RewriteRule .* - [E=INFO_PHP_SELF:%{PHP_SELF},NE]
RewriteRule .* - [E=INFO_QUERY_STRING:%{QUERY_STRING},NE]
RewriteRule .* - [E=INFO_REDIRECT_QUERY_STRING:%{REDIRECT_QUERY_STRING},NE]
RewriteRule .* - [E=INFO_REDIRECT_REMOTE_USER:%{REDIRECT_REMOTE_USER},NE]
RewriteRule .* - [E=INFO_REDIRECT_STATUS:%{REDIRECT_STATUS},NE]
RewriteRule .* - [E=INFO_REDIRECT_URL:%{REDIRECT_URL},NE]
RewriteRule .* - [E=INFO_REMOTE_ADDR:%{REMOTE_ADDR},NE]
RewriteRule .* - [E=INFO_REMOTE_HOST:%{REMOTE_HOST},NE]
RewriteRule .* - [E=INFO_REMOTE_IDENT:%{REMOTE_IDENT},NE]
RewriteRule .* - [E=INFO_REMOTE_PORT:%{REMOTE_PORT},NE]
RewriteRule .* - [E=INFO_REMOTE_USER:%{REMOTE_USER},NE]
RewriteRule .* - [E=INFO_REQUEST_FILENAME:%{REQUEST_FILENAME},NE]
RewriteRule .* - [E=INFO_REQUEST_METHOD:%{REQUEST_METHOD},NE]
RewriteRule .* - [E=INFO_REQUEST_TIME:%{REQUEST_TIME},NE]
RewriteRule .* - [E=INFO_REQUEST_URI:%{REQUEST_URI},NE]
RewriteRule .* - [E=INFO_SCRIPT_FILENAME:%{SCRIPT_FILENAME},NE]
RewriteRule .* - [E=INFO_SCRIPT_GROUP:%{SCRIPT_GROUP},NE]
RewriteRule .* - [E=INFO_SCRIPT_NAME:%{SCRIPT_NAME},NE]
RewriteRule .* - [E=INFO_SCRIPT_URI:%{SCRIPT_URI},NE]
RewriteRule .* - [E=INFO_SCRIPT_URL:%{SCRIPT_URL},NE]
RewriteRule .* - [E=INFO_SCRIPT_USER:%{SCRIPT_USER},NE]
RewriteRule .* - [E=INFO_SERVER_ADDR:%{SERVER_ADDR},NE]
RewriteRule .* - [E=INFO_SERVER_ADMIN:%{SERVER_ADMIN},NE]
RewriteRule .* - [E=INFO_SERVER_NAME:%{SERVER_NAME},NE]
RewriteRule .* - [E=INFO_SERVER_PORT:%{SERVER_PORT},NE]
RewriteRule .* - [E=INFO_SERVER_PROTOCOL:%{SERVER_PROTOCOL},NE]
RewriteRule .* - [E=INFO_SERVER_SIGNATURE:%{SERVER_SIGNATURE},NE]
RewriteRule .* - [E=INFO_SERVER_SOFTWARE:%{SERVER_SOFTWARE},NE]
RewriteRule .* - [E=INFO_THE_REQUEST:%{THE_REQUEST},NE]
RewriteRule .* - [E=INFO_TIME:%{TIME},NE]
RewriteRule .* - [E=INFO_TIME_DAY:%{TIME_DAY},NE]
RewriteRule .* - [E=INFO_TIME_HOUR:%{TIME_HOUR},NE]
RewriteRule .* - [E=INFO_TIME_MIN:%{TIME_MIN},NE]
RewriteRule .* - [E=INFO_TIME_MON:%{TIME_MON},NE]
RewriteRule .* - [E=INFO_TIME_SEC:%{TIME_SEC},NE]
RewriteRule .* - [E=INFO_TIME_WDAY:%{TIME_WDAY},NE]
RewriteRule .* - [E=INFO_TIME_YEAR:%{TIME_YEAR},NE]
RewriteRule .* - [E=INFO_TZ:%{TZ},NE]
RewriteRule .* - [E=INFO_UNIQUE_ID:%{UNIQUE_ID},NE]
 
RequestHeader set INFO_API_VERSION "%{INFO_API_VERSION}e"
RequestHeader set INFO_AUTH_TYPE "%{INFO_AUTH_TYPE}e"
RequestHeader set INFO_CONTENT_LENGTH "%{INFO_CONTENT_LENGTH}e"
RequestHeader set INFO_CONTENT_TYPE "%{INFO_CONTENT_TYPE}e"
RequestHeader set INFO_DOCUMENT_ROOT "%{INFO_DOCUMENT_ROOT}e"
RequestHeader set INFO_GATEWAY_INTERFACE "%{INFO_GATEWAY_INTERFACE}e"
RequestHeader set INFO_HTTPS "%{INFO_HTTPS}e"
RequestHeader set INFO_HTTP_ACCEPT "%{INFO_HTTP_ACCEPT}e"
RequestHeader set INFO_HTTP_ACCEPT_CHARSET "%{INFO_HTTP_ACCEPT_CHARSET}e"
RequestHeader set INFO_HTTP_ACCEPT_ENCODING "%{INFO_HTTP_ACCEPT_ENCODING}e"
RequestHeader set INFO_HTTP_ACCEPT_LANGUAGE "%{INFO_HTTP_ACCEPT_LANGUAGE}e"
RequestHeader set INFO_HTTP_CACHE_CONTROL "%{INFO_HTTP_CACHE_CONTROL}e"
RequestHeader set INFO_HTTP_CONNECTION "%{INFO_HTTP_CONNECTION}e"
RequestHeader set INFO_HTTP_COOKIE "%{INFO_HTTP_COOKIE}e"
RequestHeader set INFO_HTTP_FORWARDED "%{INFO_HTTP_FORWARDED}e"
RequestHeader set INFO_HTTP_HOST "%{INFO_HTTP_HOST}e"
RequestHeader set INFO_HTTP_KEEP_ALIVE "%{INFO_HTTP_KEEP_ALIVE}e"
RequestHeader set INFO_HTTP_MOD_SECURITY_MESSAGE "%{INFO_HTTP_MOD_SECURITY_MESSAGE}e"
RequestHeader set INFO_HTTP_PROXY_CONNECTION "%{INFO_HTTP_PROXY_CONNECTION}e"
RequestHeader set INFO_HTTP_REFERER "%{INFO_HTTP_REFERER}e"
RequestHeader set INFO_HTTP_USER_AGENT "%{INFO_HTTP_USER_AGENT}e"
RequestHeader set INFO_IS_SUBREQ "%{INFO_IS_SUBREQ}e"
RequestHeader set INFO_ORIG_PATH_INFO "%{INFO_ORIG_PATH_INFO}e"
RequestHeader set INFO_ORIG_PATH_TRANSLATED "%{INFO_ORIG_PATH_TRANSLATED}e"
RequestHeader set INFO_ORIG_SCRIPT_FILENAME "%{INFO_ORIG_SCRIPT_FILENAME}e"
RequestHeader set INFO_ORIG_SCRIPT_NAME "%{INFO_ORIG_SCRIPT_NAME}e"
RequestHeader set INFO_PATH "%{INFO_PATH}e"
RequestHeader set INFO_PATH_INFO "%{INFO_PATH_INFO}e"
RequestHeader set INFO_PHP_SELF "%{INFO_PHP_SELF}e"
RequestHeader set INFO_QUERY_STRING "%{INFO_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_QUERY_STRING "%{INFO_REDIRECT_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_REMOTE_USER "%{INFO_REDIRECT_REMOTE_USER}e"
RequestHeader set INFO_REDIRECT_STATUS "%{INFO_REDIRECT_STATUS}e"
RequestHeader set INFO_REDIRECT_URL "%{INFO_REDIRECT_URL}e"
RequestHeader set INFO_REMOTE_ADDR "%{INFO_REMOTE_ADDR}e"
RequestHeader set INFO_REMOTE_HOST "%{INFO_REMOTE_HOST}e"
RequestHeader set INFO_REMOTE_IDENT "%{INFO_REMOTE_IDENT}e"
RequestHeader set INFO_REMOTE_PORT "%{INFO_REMOTE_PORT}e"
RequestHeader set INFO_REMOTE_USER "%{INFO_REMOTE_USER}e"
RequestHeader set INFO_REQUEST_FILENAME "%{INFO_REQUEST_FILENAME}e"
RequestHeader set INFO_REQUEST_METHOD "%{INFO_REQUEST_METHOD}e"
RequestHeader set INFO_REQUEST_TIME "%{INFO_REQUEST_TIME}e"
RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"
RequestHeader set INFO_SCRIPT_FILENAME "%{INFO_SCRIPT_FILENAME}e"
RequestHeader set INFO_SCRIPT_GROUP "%{INFO_SCRIPT_GROUP}e"
RequestHeader set INFO_SCRIPT_NAME "%{INFO_SCRIPT_NAME}e"
RequestHeader set INFO_SCRIPT_URI "%{INFO_SCRIPT_URI}e"
RequestHeader set INFO_SCRIPT_URL "%{INFO_SCRIPT_URL}e"
RequestHeader set INFO_SCRIPT_USER "%{INFO_SCRIPT_USER}e"
RequestHeader set INFO_SERVER_ADDR "%{INFO_SERVER_ADDR}e"
RequestHeader set INFO_SERVER_ADMIN "%{INFO_SERVER_ADMIN}e"
RequestHeader set INFO_SERVER_NAME "%{INFO_SERVER_NAME}e"
RequestHeader set INFO_SERVER_PORT "%{INFO_SERVER_PORT}e"
RequestHeader set INFO_SERVER_PROTOCOL "%{INFO_SERVER_PROTOCOL}e"
RequestHeader set INFO_SERVER_SIGNATURE "%{INFO_SERVER_SIGNATURE}e"
RequestHeader set INFO_SERVER_SOFTWARE "%{INFO_SERVER_SOFTWARE}e"
RequestHeader set INFO_THE_REQUEST "%{INFO_THE_REQUEST}e"
RequestHeader set INFO_TIME "%{INFO_TIME}e"
RequestHeader set INFO_TIME_DAY "%{INFO_TIME_DAY}e"
RequestHeader set INFO_TIME_HOUR "%{INFO_TIME_HOUR}e"
RequestHeader set INFO_TIME_MIN "%{INFO_TIME_MIN}e"
RequestHeader set INFO_TIME_MON "%{INFO_TIME_MON}e"
RequestHeader set INFO_TIME_SEC "%{INFO_TIME_SEC}e"
RequestHeader set INFO_TIME_WDAY "%{INFO_TIME_WDAY}e"
RequestHeader set INFO_TIME_YEAR "%{INFO_TIME_YEAR}e"
RequestHeader set INFO_TZ "%{INFO_TZ}e"
RequestHeader set INFO_UNIQUE_ID "%{INFO_UNIQUE_ID}e"

Mod_Rewrite Variables Decoded!

[API_VERSION] => 20020903:12
[AUTH_TYPE] => Digest
[DOCUMENT_ROOT] => /home/user/www_root/askapache.com
[HTTPS] => off
[HTTP_ACCEPT] => text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
[HTTP_COOKIE] => PHPSESSID=752ee6d56e15f305233e30045987e5ce568c034; __qca=1176541225-59967328-5223185;
[HTTP_HOST] => www.askapache.com
[HTTP_REFERER] => http://www.askapache.com/protest/index.php?askapache=awesomeness&you=rock
[HTTP_USER_AGENT] => Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16
[IS_SUBREQ] => false
[QUERY_STRING] => e=404
[REMOTE_ADDR] => 22.162.144.211
[REMOTE_HOST] => 22.162.144.211
[REMOTE_PORT] => 4511
[REMOTE_USER] => administrator
[REQUEST_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[REQUEST_METHOD] => GET
[REQUEST_URI] => /protest/index.php
[SCRIPT_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[SCRIPT_GROUP] => daemonu
[SCRIPT_USER] => askapache
[SERVER_ADDR] => 208.113.134.190
[SERVER_ADMIN] => webmaster@askapache.com
[SERVER_NAME] => www.askapache.com
[SERVER_PORT] => 80
[SERVER_PROTOCOL] => HTTP/1.1
[SERVER_SOFTWARE] => Apache/2.0.61 (Unix) PHP/4.4.7 mod_ssl/2.0.61 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2
[THE_REQUEST] => GET /protest/adf HTTP/1.1
[TIME] => 20080820014309
[TIME_DAY] => 20
[TIME_HOUR] => 01
[TIME_MIN] => 43
[TIME_MON] => 08
[TIME_SEC] => 09
[TIME_WDAY] => 3
[TIME_YEAR] => 2008

Request using HTTPS

[API_VERSION] => 20020903:12
[AUTH_TYPE] => Digest
[DOCUMENT_ROOT] => /home/user/www_root/askapache.com
[HTTPS] => on
[HTTP_ACCEPT] => text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
[HTTP_COOKIE] => PHPSESSID=752ee6d56e15f305233e30045987e5ce568c034; __qca=1176541225-59967328-5223185;
[HTTP_HOST] => www.askapache.com
[HTTP_REFERER] => http://www.askapache.com/protest/index.php?askapache=awesomeness&you=rock
[HTTP_USER_AGENT] => Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16
[IS_SUBREQ] => false
[QUERY_STRING] => hi=you&whats=&amp;you
[REMOTE_ADDR] => 22.162.144.211
[REMOTE_HOST] => 22.162.144.211
[REMOTE_PORT] => 4605
[REMOTE_USER] => administrator
[REQUEST_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[REQUEST_METHOD] => GET
[REQUEST_URI] => /protest/index.php
[SCRIPT_FILENAME] => /home/user/www_root/askapache.com/protest/index.php
[SCRIPT_GROUP] => daemonu
[SCRIPT_USER] => askapache
[SERVER_ADDR] => 208.113.134.190
[SERVER_ADMIN] => webmaster@askapache.com
[SERVER_NAME] => www.askapache.com
[SERVER_PORT] => 443
[SERVER_PROTOCOL] => HTTP/1.1
[SERVER_SOFTWARE] => Apache/2.0.61 (Unix) PHP/4.4.7 mod_ssl/2.0.61 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2
[THE_REQUEST] => GET /protest/index.php?hi=you&whats=&amp;you HTTP/1.1
[TIME] => 20080820015016
[TIME_DAY] => 20
[TIME_HOUR] => 01
[TIME_MIN] => 50
[TIME_MON] => 08
[TIME_SEC] => 16
[TIME_WDAY] => 3
[TIME_YEAR] => 2008

Emulating ErrorDocuments with Mod_Rewrite

The ErrorDocument directive is helpful because an errordocument is called differently then a normal file, and it contains special variables to help an admin debug.

I’ve wanted to use a RewriteCond + a RewriteRule to cause an Apache ErrorDocument to be displayed for a long time… I finally figured it out. Simply use the HTTP STATUS CODE trick in combination with a simple RewriteRule to trigger an Apache ErrorDocument.

This code emulates the internal 404 process Apache goes through.. If the file is not found it requests the /test/trigger-error/404 internally which triggers the 404 ErrorDocument.

ErrorDocument 404 /test/errordocument/404.html
Redirect 404 /test/trigger-error/404
 
RewriteEngine On
RewriteBase /
RewriteCond %{ENV:REDIRECT_STATUS} !=404
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .* /test/trigger-error/404 [L]

Big Deal.. you might say… well consider that this works with any status code, and using this method you now have the power to trigger any errordocument page based on any kind of rewritecond. I’ll be writing about some practical uses for this powerful method in the coming weeks, but heres a good example now so you can see how it can be used.

This bit of code Triggers the 505 HTTP Version Not Supported When a request is made to the server with a protocol other than 1.1.

ErrorDocument 505 /test/errordocument/505.html
Redirect 505 /test/trigger-error/505
 
RewriteEngine On
RewriteBase /
 
RewriteCond %{ENV:REDIRECT_STATUS} !=505
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /.*\ HTTP/(0\.9|1\.0|1\.1) [NC]
RewriteRule .* /test/trigger-error/505 [L]

YES! I realize I didn’t explain that very well, I didn’t realize it was that complicated.. I wanted to go into how to use these advanced tricks and methods to achieve some really cool stuff, but explaining just this little bit took me awhile and I’m out of page space!

So play around with this and I’ll post back some of the untapped sicknesses you can give a website with such powerful methods at your disposal.

Ralf S. Engelschall

/*
 *  URL Rewriting Module
 *
 *  This module uses a rule-based rewriting engine (based on a
 *  regular-expression parser) to rewrite requested URLs on the fly.
 *
 *  It supports an unlimited number of additional rule conditions (which can
 *  operate on a lot of variables, even on HTTP headers) for granular
 *  matching and even external database lookups (either via plain text
 *  tables, DBM hash files or even external processes) for advanced URL
 *  substitution.
 *
 *  It operates on the full URLs (including the PATH_INFO part) both in
 *  per-server context (httpd.conf) and per-dir context (.htaccess) and even
 *  can generate QUERY_STRING parts on result.   The rewriting result finally
 *  can lead to internal subprocessing, external request redirection or even
 *  to internal proxy throughput.
 *
 *  This module was originally written in April 1996 and
 *  gifted exclusively to the The Apache Software Foundation in July 1997 by
 *
 *      Ralf S. Engelschall
 *      rse engelschall.com
 *      www.engelschall.com
 */

Crazy Advanced Mod_Rewrite Tutorial originally appeared on AskApache.com

]]> http://www.askapache.com/htaccess/crazy-advanced-mod_rewrite-tutorial.html/feed/ 22 mod_rewrite Fix for Caching Updated Files http://www.askapache.com/htaccess/mod_rewrite-fix-for-caching-updated-files.html http://www.askapache.com/htaccess/mod_rewrite-fix-for-caching-updated-files.html#comments Sun, 30 Aug 2009 13:20:59 +0000 AskApache http://www.askapache.com/htaccess/mod_rewrite-code-to-fix-caching-updated-files.html Web Developers sometimes use file.ext?v=004 as a version control system to force visitors to use updated files. This is terrible. Instead link to apache-003.css and set it to be cached forever. When you change the file you just change the links to apache-004.css. That eliminates millions of bandwidth and resource robbing If-Modified-Since requests. You only need Apache with mod_rewrite, and 1-10 minutes!

]]> Web Developers sometimes use file.ext?v=137 as a version control system so they can force visitors to use an updated file. This is so terrible. Instead link to apache-003.css and set it to be cached forever. When you change the file you just change the links to apache-004.css.

Raw Speed Benefit

This eliminates millions of bandwidth and resource robbing 304 If-Modified-Since requests.

Renaming links vs. Renaming files

On the server my files are named apache.css and apache.js, but in the xhtml I point to them using the names apache-113.css and apache-113.js, after I change the file I just add 1 to the number, and the new file is cached. They are internally redirected to apache.css and apache.js (invisible to the user) The concept is similar to a “shortcut” in windows or a symlink in BSD. The trick is that I never actually rename the files on the server. I just rename them in the html. That means apache-135.css is served from the file apache.css but the browser/cache only see and know about apache-135.css.

XHTML

NOTE: You can do your own investigating of this sites source code and HTTP headers to see this whole system in action

<link href="http://static.askapache.com/z/c/apache-0031.css" rel="stylesheet" type="text/css" />
<script src="http://static.askapache.com/z/j/apache-0031.js" type="text/javascript"></script>

mod_rewrite code for htaccess or httpd.conf

Updated: 10/20/2008

RewriteEngine On
RewriteBase /
RewriteCond %{THE_REQUEST} ^(GET|HEAD)\ /z/([a-z]+)/(.+)-([0-9]+)\.(js|css).*\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .+ /z/%2/%3.%5 [NC,L]

Alternate mod_rewrite code

RewriteEngine On
RewriteBase /
RewriteRule ^([cij]+)(/?[a-z]*)/([a-z]+)-([0-9]+)\.([a-z]+)$ /$1$2/$3.$5 [L]

Ideal Caching Scheme

Ok so you want the xhtml to be the only file that isn’t cached without being validated, its simple to setup your own caching scheme.

Bad Cache information for a file with the ?v=foo hack

This object will be fresh for 1 week. It can be validated with Last-Modified. This object requests that a Cookie be set; this makes it and other pages affected automatically stale; clients must check them upon every request. Because it contains a query (‘?’) in the URL, many caches will not keep this object.

mod_rewrite Fix for Caching Updated Files originally appeared on AskApache.com

]]> http://www.askapache.com/htaccess/mod_rewrite-fix-for-caching-updated-files.html/feed/ 2 An AskApache Plugin Upgrade to Rule them All http://www.askapache.com/wordpress/an-askapache-plugin-upgrade-to-rule-them-all.html http://www.askapache.com/wordpress/an-askapache-plugin-upgrade-to-rule-them-all.html#comments Wed, 29 Jul 2009 17:59:07 +0000 AskApache http://www.askapache.com/?p=3062 apache-server-statusSo my blog as been rather quiet for almost a year now, and very few updates if any have been released for my Password Protection PLugin, my Google 404 Plugin, and definately not for my AskApache CrazyCache plugin, which I will be releasing last... So for all of you who've helped me out by sending me suggestions and notifying me of errors and sticking with it... Just wanted to say sorry about that, and thanks for all the great ideas.. Well, I've been sticking with it as well believe it our not. I manage to get free days once in a while, and then its time to jam.

]]> An AskApache Plugin Upgrade to Rule them AllSo my blog as been rather quiet for almost a year now, and very few updates if any have been released for my Password Protection PLugin, my Google 404 Plugin, and definately not for my AskApache CrazyCache plugin, which I will be releasing last… So for all of you who’ve helped me out by sending me suggestions and notifying me of errors and sticking with it… Just wanted to say sorry about that, and thanks for all the great ideas.. Well, I’ve been sticking with it as well believe it our not. I manage to get free days once in a while, and then its time to jam.

I’ve used just about every CMS/Blog/Forum/Trac/Gallery/etc) and really didn’t like a lot of the way they coded… I could use php but I didn’t KNOW php.. so I’ve had to learn php also, and it was tough to learn the advanced class usage and all the other language specific (but similar) constructs for php. It was especially difficult (but fun and challenging) to program so as to be compatible with php4 and php5 (Such is WordPress). But I kept at it, and soon you can decide for yourself what to make of it.

I can code in plenty of languages (bash, lua, windows .bat and vbs, ocaml, big fan of all things shell) and can work my way through C and even sorta somewhat with assembly. Assembly is the hardest, by far, I’m into easy and powerful languages like Python, Javascript, perl, php, ruby, and CGI. I’ve used PHP for a long time to do various things, but never to build software projects like this. Once I noticed WordPress’s core .php files and the excellent programming I wanted to try and learn hot to do it. The WordPress code is some of the best I’ve seen. It appears the way they built it was planned, and not just dreamt up while typing that I can’t help but do. Every time I read through the core code I learn a new trick or very nice way to do something. Those guys are really good, and I think WordPress is going to dominate for a long long time.

The Strategy

The Password Protection (passpro) plugin has a lot of complex stuff going on, especially for a newbie to PHP and WordPress like me, so after refactoring the whole thing at least 5 times I decided to modify my approach, and wrote the AskApache Google 404 Plugin as a way to practice on a simpler piece of code, while at the same time providing a plugin of value. Eventually I stopped thinking I could just code the whole thing in one sit-down with a stream-of-consciousness, and had to instead modularize the code and focus in on each part before moving to the next (I go without a plan because its fun, just not the most productive, but again, I’m not a programmer in the scientific sense.).

So I decided I had to really learn how WordPress Plugins work, filters, hooks, actions, and basically comfortability at reverse-engineering code, (Im a beginner for the last time), and so with the upcoming release of the AskApache Google 404 Plugin I have succeeded in making an incredibly stable plugin. That way I only have to worry about what the aapasspro plugin is doing, instead of trying to fit it into a framework.

AskApache Google 404 Upgrade

I think its rather unusual to develop a nice plugin like this 404 handler merely for the purpose of improving upon another plugin, but hey it worked. As of 08/03/2009 14:06PM EST I have about 1 hour left of finishing touches to release this upgrade. But as you cantell by my badly edited posts, I don’t have a lot of time to myself. An hour here and there is about it. So it could be up to 2 weeks before I actually have the time to commit the release to the repo. On a sidenote, have you checked out Windows 7 News? I’ve been contracted to do some technical work for them and thought they had an excellent site.

An AskApache Plugin Upgrade to Rule them All

An AskApache Plugin Upgrade to Rule them All

But keep in mind, the 404 PLugin is just where I practice for the passpro plugin, which truly does have features that no other software like it has ever had. I understand the technology behind this plugin, and know it would really have a great impact on improving the Web (esp. WordPress) for all of us, I’ve just had to learn how to make it.

AskApache Password Protection

Probably still a couple weeks away, this plugin is the ultimate culmination of apache hackers dreams, at least those on shared servers (who may be interested in learning how to bypass security of said servers).. So this is something I have much too fun with doing what I like to do.. network/protocol-level security. I’ve examined the source code for many software packages that I use or have used to audit a server’s security, and this simple php plugin in most instances can enumerate with accuraccy most of the server’s setup in about a minute. The catch (and the file permission problems I had to find a workaround too) is that this software is launched on the server, not remotely against the server.

Some of the software I examined was whiskers, nessus, nmap, hping, mozilla source, wireshark, ncftp, netcat, etc.. The closest comparison to the socket-level class I’ve hacked together to those is wireshark. Except that wireshark only interprets (captures) the data passing over the wire, while this class does that and in fact sends and receives the data like netcat or nmap. Its really more similar to metasploit, and can easily be used to send hex, binary, ascii, or any type of payload to the remote or local host.

The Upgrades Begin

Well I started working on them a long time ago. Both the Password Protection plugin and the Google 404 plugin needed serious work. And I finally have it all figured out. Essentially I would work on one and finish an upgrade, but I just wasn’t happy with it and I wold start all over again, refactoring the code. So as I put the finishing touches on those 2 plugins keep an eye out. They are major upgrades. I was able to meet all the goals I had for them, and came up with a lot of more improvements during the process.One of the main things I needed was a socket-level class to perform all kinds of checks and tests on. I need this also for my crazy cache plugin, which my blog is currently using , and I have a 2 more really nice pplugins I use that also needed access to a network class. I wrote about what I was doing with fsockopen, and I’ve been improving on that example ever since. I use this class to do some really powerful and exciting stuff, but you’ll see it soon enough. As an indication of ‘getting it right’ for the Password Protection plugin, the plugin will now work on Windows, Apache, IIS, Lighthttpd, and will even work running on a blackberry web server. So now everyone using wordpress can at least get some security()

Many of the the other improvements focus on using the fsockopen class and .htaccess tricks to basically enumerate and discover all the different capabilities of your particular server; That way you can learn about all the features and security that are possible for your specific server, and the securty modules wi8ll be geared for that as well. FINALLY this plugin is going to be stable, and I just cant wait to see how people react when they learn all great capability their Apache-based Server has that they didn’t have a clue about. Its amazing in that sense, and hackers will love theh way it works.. but your server admins will love it even more because its entirely 100% focused on helping you to set your site up (if you have Apache) to keep spammers out, to keep virii-serving robots and their log-hogging exploit requests and CPU/Mem robiing 404 errors off of your servers for real. This will have a noticeable affect to whoever is running the server. As you can tell.. I am pumped!

Apache is easy to configure and use, but only when you have root access. Most people on shared and private hosting aren’t even able to view the main config file, let alone execute the Apache binaries to see what features are available and what configuration is being used.

Apache can only be influenced by the main server configs and by .htaccess files. Not by php, not by perl, and the main configs are almost never accessible to the masses. But .htaccess files are. And many hosting providers allow and enable .htaccess files, a configuration file for your web server. The advanced features and capabilities of Apache were out of reach for most of us, it just wasn’t possible to enumerate or access, and most hosting providers are infamous for their lack of .htaccess (customer) support. This plugin goes around those problems to give the power back to the people.

y creating custom .htaccess files containing unpublished .htaccess tricks and techniques and combining that with the use of socket-level networking from WordPress (PHP) using fsockopen, we can effectively enumerate and discover an incredible amount of features and settings you will be able to control and use with this plugin.

Here are a few examples of the capabilities of this plugin, some of which I believe no other software can do.. (Open source free to copy!).

  1. Current Version of Apache (Down to the API Version)
  2. List of ALL Modules currently enabled by Apache (Such as Mod_Rewrite)
  3. List of ALL Directives enabled by EACH enabled Module.
  4. Enumerate .htaccess Overrides, Context Permissions
  5. Test for any builtin Handlers (like the status handler screenshot)
  6. Configure SSI (http://www.askapache.com/htaccess/advanced-htaccess-ssi.html#htaccess-ssi-security)

March 1, 2009
I would focus on the method that WordPress uses. The code they have now (2.8 bleeding-edge) still isn’t where it needs to be, but this is some difficult stuff and they have a brilliant start, it’ll work.. just a question of when.

Apache Security Model - In ColorThe main issue with the password protection plugin working for some people and not others is due to file permission configurations. The plugin attempts to write/modify files in your blog’s root directory.

November 05, 2008
To make a long story short, I downloaded each major release of the apache httpd source code starting at version 1.3.0 and finishing with version 2.2.11, I then compiled each version and built a HTTPD from source for all these apache versions.

  • 1.3.0
  • 1.3.1
  • 1.3.11
  • 1.3.12
  • 1.3.14
  • 1.3.17
  • 1.3.19
  • 1.3.2
  • 1.3.20
  • 1.3.22
  • 1.3.23
  • 1.3.24
  • 1.3.27
  • 1.3.28
  • 1.3.29
  • 1.3.3
  • 1.3.31
  • 1.3.32
  • 1.3.33
  • 1.3.34
  • 1.3.35
  • 1.3.36
  • 1.3.37
  • 1.3.39
  • 1.3.4
  • 1.3.41
  • 1.3.6
  • 1.3.9
  • 2.0.35
  • 2.0.36
  • 2.0.39
  • 2.0.40
  • 2.0.42
  • 2.0.43
  • 2.0.44
  • 2.0.45
  • 2.0.46
  • 2.0.47
  • 2.0.48
  • 2.0.49
  • 2.0.50
  • 2.0.51
  • 2.0.52
  • 2.0.53
  • 2.0.54
  • 2.0.55
  • 2.0.58
  • 2.0.59
  • 2.0.61
  • 2.0.63
  • 2.1.3-beta
  • 2.1.6-alpha
  • 2.1.7-beta
  • 2.1.8-beta
  • 2.1.9-beta
  • 2.2.0
  • 2.2.10
  • 2.2.2
  • 2.2.3
  • 2.2.4
  • 2.2.6
  • 2.2.8
  • 2.2.9
  • 2.2.10
  • 2.2.11


Then I went through each version and determined the compatible modules for that version, and I’m pretty confident that I was also able to find each and every directive allowed by the compatible modules for that version (including core directives). See .htaccess directive list. Basically I can now test a server using a variety of methods and determine almost 100% accurately what version of Apache (down to the API) is running, what modules (and versions) are enabled, and each and every directive that is allowed or disallowed for that version. So this is so awesome because now we can enable all sorts of additional security features.

Htaccess enabled Modules

Here are most of the modules that come with Apache. Each one can have new commands that can be used in .htaccess file scopes.

mod_actions, mod_alias, mod_asis, mod_auth_basic, mod_auth_digest, mod_authn_anon, mod_authn_dbd, mod_authn_dbm, mod_authn_default, mod_authn_file, mod_authz_dbm, mod_authz_default, mod_authz_groupfile, mod_authz_host, mod_authz_owner, mod_authz_user, mod_autoindex, mod_cache, mod_cern_meta, mod_cgi, mod_dav, mod_dav_fs, mod_dbd, mod_deflate, mod_dir, mod_disk_cache, mod_dumpio, mod_env, mod_expires, mod_ext_filter, mod_file_cache, mod_filter, mod_headers, mod_ident, mod_imagemap, mod_include, mod_info, mod_log_config, mod_log_forensic, mod_logio, mod_mem_cache, mod_mime, mod_mime_magic, mod_negotiation, mod_proxy, mod_proxy_ajp, mod_proxy_balancer, mod_proxy_connect, mod_proxy_ftp, mod_proxy_http, mod_rewrite, mod_setenvif, mod_speling, mod_ssl, mod_status, mod_substitute, mod_unique_id, mod_userdir, mod_usertrack, mod_version, mod_vhost_alias

Debugging HTTP protocol

Check this out! I’m particularly happy about this feature, which outputs an exact trace of any requests made by the plugin (such as during the testing phase) by saving the actual raw data sent out on the wire using fsockopen, RX and TX. This is useful for a number of reasons, viewing your headers, finding Redirect Loops, testing RewriteRules, and following the request hop-by-hop for debugging. The below example shows 2 requests for 2 URIs. The first URI is protected using Digest Authentication, the 2nd shows Basic.

 ______________
|  RAW TRACE   |
==================================================================================================================================
GET /htaccess/index.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
 
HTTP/1.1 401 Authorization Required
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
WWW-Authenticate: Digest realm="do or die", nonce="03328f3ec7c7b", algorithm=MD5, domain="/", qop="auth"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 882
Connection: close
Content-Type: text/html; charset=UTF-8
 
GET /htaccess/index.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
Authorization: Digest username="test",realm="do or die",nonce="03328f3ec7c7b",uri="/htaccess/index.txt?testing=query",
cnonce="82d057852a9dc497",nc=00000001,algorithm=MD5,response="9d476e9ea3",qop="auth"
 
HTTP/1.1 200 OK
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
Authentication-Info: rspauth="9051b01ee26dd62b3e2b40dada694f45", cnonce="82d057852a9dc497", nc=00000001, qop=auth
Last-Modified: Tue, 21 Jul 2009 23:56:00 GMT
Accept-Ranges: bytes
Cache-Control: max-age=3600
Expires: Wed, 22 Jul 2009 07:29:58 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 27
Connection: close
Content-Type: text/plain; charset=UTF-8
```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````
 
 ______________
|  RAW TRACE   |
==================================================================================================================================
GET /htaccess/po.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
 
HTTP/1.1 401 Authorization Required
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
WWW-Authenticate: Basic realm="Po Pimping"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 878
Connection: close
Content-Type: text/html; charset=UTF-8
 
GET /htaccess/po.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
Authorization: Basic adfAGAltcA==
 
HTTP/1.1 200 OK
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
Last-Modified: Wed, 22 Jul 2009 05:54:39 GMT
Accept-Ranges: bytes
Cache-Control: max-age=3600
Expires: Wed, 22 Jul 2009 07:29:58 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 27
Connection: close
Content-Type: text/plain; charset=UTF-8
```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````

.htaccess Directives

AcceptFilter, AcceptMutex, AcceptPathInfo, AccessFileName, Action, AddAlt, AddAltByEncoding, AddAltByType, AddCharset, AddDefaultCharset, AddDescription, AddEncoding, AddHandler, AddIcon, AddIconByEncoding, AddIconByType, AddInputFilter, AddLanguage, AddModuleInfo, AddOutputFilter, AddOutputFilterByType, AddType, Alias, AliasMatch, AllowCONNECT, AllowEncodedSlashes, AllowOverride, Anonymous, Anonymous_Authoritative, Anonymous_LogEmail, Anonymous_MustGiveEmail, Anonymous_NoUserID, Anonymous_NoUserId, Anonymous_VerifyEmail, AssignUserId, AuthAuthoritative, AuthBasicAuthoritative, AuthBasicProvider, AuthDBDUserPWQuery, AuthDBDUserRealmQuery, AuthDBM, AuthDBMAuthoritative, AuthDBMGroupFile, AuthDBMType, AuthDBMUserFile, AuthDefaultAuthoritative, AuthDigestAlgorithm, AuthDigestDomain, AuthDigestFile, AuthDigestGroupFile, AuthDigestNcCheck, AuthDigestNonceFormat, AuthDigestNonceLifetime, AuthDigestProvider, AuthDigestQop, AuthDigestShmemSize, AuthGroupFile, AuthLDAPAuthzEnabled, AuthLDAPBindDN, AuthLDAPBindON, AuthLDAPBindPassword, AuthLDAPCharsetConfig, AuthLDAPCompareDNOnServer, AuthLDAPDereferenceAliases, AuthLDAPEnabled, AuthLDAPFrontPageHack, AuthLDAPGroupAttribute, AuthLDAPGroupAttributeIsDN, AuthLDAPRemoteUserAttribute, AuthLDAPRemoteUserIsDN, AuthLDAPStartTLS, AuthLDAPURL, AuthLDAPUrl, AuthName, AuthType, AuthUserFile, AuthzDBMAuthoritative, AuthzDBMType, AuthzDefaultAuthoritative, AuthzGroupFileAuthoritative, AuthzLDAPAuthoritative, AuthzOwnerAuthoritative, AuthzUserAuthoritative, BS2000Account, BalancerMember, BrowserMatch, BrowserMatchNoCase, BufferedLogs, CGIMapExtension, CacheDefaultExpire, CacheDirLength, CacheDirLevels, CacheDisable, CacheEnable, CacheExpiryCheck, CacheFile, CacheForceCompletion, CacheGcClean, CacheGcDaily, CacheGcInterval, CacheGcMemUsage, CacheGcUnused, CacheIgnoreCacheControl, CacheIgnoreHeaders, CacheIgnoreNoLastMod, CacheLastModifiedFactor, CacheMaxExpire, CacheMaxFileSize, CacheMaxStreamingBuffer, CacheMinFileSize, CacheNegotiatedDocs, CacheRoot, CacheSize, CacheStoreNoStore, CacheStorePrivate, CacheTimeMargin, CharsetDefault, CharsetOptions, CharsetSourceEnc, CheckCaseOnly, CheckSpelling, ChildPerUserId, ContentDigest, CookieDomain, CookieExpires, CookieLog, CookieName, CookieStyle, CookieTracking, CoreDumpDirectory, CustomLog, DAV, DAVDepthInfinity, DAVGenericLockDB, DAVMinTimeout, DBDExptime, DBDKeep, DBDMax, DBDMin, DBDParams, DBDPersist, DBDPrepareSQL, DBDriver, Dav, DavDepthInfinity, DavGenericLockDB, DavLockDB, DavMinTimeout, DefaultIcon, DefaultLanguage, DefaultType, DeflateBufferSize, DeflateCompressionLevel, DeflateFilterNote, DeflateMemLevel, DeflateWindowSize, Directory, DirectoryIndex, DirectoryMatch, DirectorySlash, DocumentRoot, DumpIOInput, DumpIOOutput, EnableExceptionHook, EnableMMAP, EnableSendfile, ErrorDocument, ErrorLog, Example, ExpiresActive, ExpiresByType, ExpiresDefault, ExtFilterDefine, ExtFilterOptions, ExtendedStatus, FancyIndexing, FileETag, Files, FilesMatch, FilterChain, FilterDeclare, FilterProtocol, FilterProvider, FilterTrace, ForceLanguagePriority, ForceType, ForensicLog, GprofDir, GracefulShutdownTimeout, Group, Header, HeaderName, HostNameLookups, HostnameLookups, ISAIPFakeAsync, ISAPIAppendLogToErrors, ISAPIAppendLogToQuery, ISAPICacheFile, ISAPIFakeAsync, ISAPILogNotSupported, ISAPIReadAheadBuffer, IdentityCheck, IdentityCheckTimeout, IfDefine, IfModule, IfVersion, ImapBase, ImapDefault, ImapMenu, Include, IndexIgnore, IndexOptions, IndexOrderDefault, IndexStyleSheet, KeepAlive, KeepAliveTimeout, LDAPCacheEntries, LDAPCacheTTL, LDAPCertDBPath, LDAPConnectionTimeout, LDAPOpCacheEntries, LDAPOpCacheTTL, LDAPSharedCacheFile, LDAPSharedCacheSize, LDAPTrustedClientCert, LDAPTrustedGlobalCert, LDAPTrustedMode, LDAPVerifyServerCert, LanguagePriority, Limit, LimitExcept, LimitInternalRecursion, LimitRequestBody, LimitRequestFields, LimitRequestFieldsize, LimitRequestLine, LimitXMLRequestBody, Listen, ListenBacklog, LoadFile, LoadModule, Location, LocationMatch, LockFile, LogFormat, LogLevel, MCacheMaxObjectCount, MCacheMaxObjectSize, MCacheMaxStreamingBuffer, MCacheMinObjectSize, MCacheRemovalAlgorithm, MCacheSize, MMapFile, MaxClients, MaxKeepAliveRequests, MaxMemFree, MaxRequestsPerChild, MaxSpareServers, MaxSpareThreads, MaxSpareThreadsPerChild, MaxThreads, MetaDir, MetaFiles, MetaSuffix, MimeMagicFile, MinSpareServers, MinSpareThreads, ModMimeUsePathInfo, MultiviewsMatch, NWSSLTrustedCerts, NWSSLUpgradeable, NameVirtualHost, NoProxy, NumServers, Options, PassEnv, PerlAccessHandler, PerlAuthenHandler, PerlAuthzHandler, PerlChildExitHandler, PerlChildInitHandler, PerlCleanupHandler, PerlDispatchHandler, PerlFixupHandler, PerlFreshRestart, PerlHandler, PerlHeaderParserHandler, PerlInitHandler, PerlLogHandler, PerlModule, PerlPassEnv, PerlPostReadRequestHandler, PerlRequire, PerlRestartHandler, PerlSendHeader, PerlSetEnv, PerlSetVar, PerlSetupEnv, PerlTaintCheck, PerlTransHandler, PerlTypeHandler, PerlWarn, PidFile, Port, Protocol, ProtocolEcho, Proxy, ProxyBadHeader, ProxyBlock, ProxyDomain, ProxyErrorOverride, ProxyFtpDirCharset, ProxyIOBufferSize, ProxyMatch, ProxyMaxForwards, ProxyPass, ProxyPassInterpolateEnv, ProxyPassMatch, ProxyPassReverse, ProxyPassReverseCookieDomain, ProxyPassReverseCookiePath, ProxyPreserveHost, ProxyReceiveBufferSize, ProxyRemote, ProxyRemoteMatch, ProxyRequests, ProxySet, ProxyStatus, ProxyTimeout, ProxyVia, RLimitCPU, RLimitMEM, RLimitNPROC, ReadmeName, Redirect, RedirectMatch, RedirectPermanent, RedirectTemp, RemoveCharset, RemoveEncoding, RemoveHandler, RemoveInputFilter, RemoveLanguage, RemoveOutputFilter, RemoveType, RequestHeader, Require, RewriteBase, RewriteCond, RewriteEngine, RewriteLock, RewriteLog, RewriteLogLevel, RewriteMap, RewriteOptions, RewriteRule, SSIAccessEnable, SSIEndTag, SSIErrorMsg, SSIStartTag, SSITimeFormat, SSIUndefinedEcho, SSLCACertificateFile, SSLCACertificatePath, SSLCADNRequestFile, SSLCADNRequestPath, SSLCARevocationFile, SSLCARevocationPath, SSLCertificateChainFile, SSLCertificateFile, SSLCertificateKeyFile, SSLCipherSuite, SSLCryptoDevice, SSLEngine, SSLHonorCipherOrder, SSLLog, SSLLogLevel, SSLMutex, SSLOptions, SSLPassPhraseDialog, SSLProtocol, SSLProxyCACertificateFile, SSLProxyCACertificatePath, SSLProxyCARevocationFile, SSLProxyCARevocationPath, SSLProxyCipherSuite, SSLProxyEngine, SSLProxyMachineCertificateFile, SSLProxyMachineCertificatePath, SSLProxyProtocol, SSLProxyVerify, SSLProxyVerifyDepth, SSLRandomSeed, SSLRequire, SSLRequireSSL, SSLSessionCache, SSLSessionCacheTimeout, SSLUserName, SSLVerifyClient, SSLVerifyDepth, Satisfy, ScoreBoardFile, Script, ScriptAlias, ScriptAliasMatch, ScriptInterpreterSource, ScriptLog, ScriptLogBuffer, ScriptLogLength, ScriptStock, SecureListen, SendBufferSize, ServerAdmin, ServerAlias, ServerLimit, ServerName, ServerPath, ServerRoot, ServerSignature, ServerTokens, SetEnv, SetEnvIf, SetEnvIfNoCase, SetHandler, SetInputFilter, SetOutputFilter, StartServers, StartThreads, Substitute, SuexecUserGroup, ThreadLimit, ThreadStackSize, ThreadsPerChild, TimeOut, Timeout, TraceEnable, TransferLog, TypeAuthDBMUserFile, TypesConfig, UnsetEnv, UseCanonicalName, UseCanonicalPhysicalPort, User, UserDir, VirtualDocumentRoot, VirtualDocumentRootIP, VirtualHost, VirtualScriptAlias, VirtualScriptAliasIP, Win32DisableAcceptEx, XBitHack, allow, deny, order, php_admin_flag, php_admin_value, php_flag, php_value

You can view the plugins home page, old, or view it on the wordpress.org site.

An AskApache Plugin Upgrade to Rule them All originally appeared on AskApache.com

]]> http://www.askapache.com/wordpress/an-askapache-plugin-upgrade-to-rule-them-all.html/feed/ 1 DNS Round Robin Configuration using Rsync over SSH http://www.askapache.com/linux-unix/dns-round-robin-rsync-over-ssh.html http://www.askapache.com/linux-unix/dns-round-robin-rsync-over-ssh.html#comments Tue, 14 Apr 2009 05:46:20 +0000 AskApache http://www.askapache.com/?p=2540 The goal is to add the HostGator server to be an exact mirror of the static.askapache.com domain, then to add that server as a 2nd A record to my DNS zone. That way half the visitors to the size will be taking up resources and bandwidth on the HostGator server instead of mine.

Round Robin A records in DNS are intended to evenly distribute queries between each host of the same name. Using some tricks straight out of a hackers toolbox we can verify if the distribution is taking place. (It is.)

]]> My traffic is growing, alot, and I need to plan how I’m going to maintain scalability, high availability, and redundancy. Scalability is an application’s ability to support a growing number of users. High availability can be defined as redundancy or speed. I decided to setup Round Robin DNS for static.askapache.com, which is the “static” subdomain of AskApache that serves all the static assets like images, javascript, css, etc.. (BTW, the z stands for ZAP).. All I needed to attempt setting this up was another hosting account on a separate server. I have hosting accounts with around 10 different companies from working with various clients over the years, like Powweb and I don’t use them because they suck in terms of the unix environment. Many of these web hosts are actually very fast bandwidth-wise..

Round Robin Concept

Round Robin DNSA few months ago I was given a free hosting account on HostGator to evaluate and tempt me away from DreamHost to become a Gator. I get a lot of these types of offers from time to time from the smaller Web companies who read AskApache.. but when I noticed the SSH access was jailed and saw how restrictive the shell was I felt like I was on a windows box.. I want a shell, cpanel sucks. I compile and run everything from the shell so thats was a deal-breaker and I sorta forgot all about it.

The goal is to add the HostGator server to be an exact mirror of the static.askapache.com domain, then to add that server as a 2nd A record to my DNS zone. That way half the visitors to the size will be taking up resources and bandwidth on the HostGator server instead of mine.

Round Robin A records in DNS are intended to evenly distribute queries between each host of the same name. Using some tricks straight out of a hackers toolbox we can verify if the distribution is taking place. (It is.)

DNS – Domain Name System

The Domain Name System is often analogized as a “phone book” for the Internet because it translates human-friendly computer hostnames into IP addresses. For example, www.askapache.com translates to 208.113.134.190. Every request for a human-friendly hostname first needs to be translated to the IP before the server can be queried. Normally each hostname corresponds to exactly 1 IP address. But in a Round Robin setup, the idea is to have the hostname correspond to multiple IP addresses, which are different servers that house the exact same content, resulting in some hosts requesting files from one server, and another host requesting files from the other server, resulting in less CPU resources and bandwidth.

Here is an the zone for static.askapache.com Round Robin records:

QUERY TTL CLASS TYPE ANSWER
static.askapache.com 13084 IN A 64.111.114.111
static.askapache.com 13084 IN A 69.56.174.114

Round Robin DNS

Round robin DNS is a technique of load distribution, load balancing, or fault-tolerance provisioning multiple, redundant Internet Protocol service hosts, e.g., Web servers, FTP servers, by managing the Domain Name System’s (DNS) responses to address requests from client computers according to an appropriate statistical model.

In its simplest implementation Round-robin DNS works by responding to DNS requests not only with a single IP address, but a list of IP addresses of several servers that host identical services. The order in which IP addresses from the list are returned is the basis for the term round robin. With each DNS response, the IP address sequence in the list is permuted. Usually, basic IP clients attempt connections with the first address returned from a DNS query so that on different connection attempts clients would receive service from different providers, thus distributing the overall load among servers.

Round robin DNS is often used for balancing the load of geographically-distributed Web servers. For example, a company has one domain name and three identical web sites residing on three servers with three different IP addresses. When one user accesses the home page it will be sent to the first IP address. The second user who accesses the home page will be sent to the next IP address, and the third user will be sent to the third IP address. In each case, once the IP address is given out, it goes to the end of the list. The fourth user, therefore, will be sent to the first IP address, and so forth.

Although easy to implement, round robin DNS has problematic drawbacks, such as those arising from record caching in the DNS hierarchy itself, as well as client-side address caching and reuse, the combination of which can be difficult to manage. Round robin DNS should not solely be relied upon for service availability. If a service at one of the addresses in the list fails, the DNS will continue to hand out that address and clients will still attempt to reach the inoperable service.

Does Round Robin Work

Definately. I can look at the access logs for both servers and see that they are splitting the requests nicely. It is definately not an exact split however, look at these statistics and you’ll see what I mean.

$ dig @ns1.dreamhost.com +authority +all static.askapache.com
 
;; ANSWER SECTION:
static.askapache.com.        14400   IN      A       69.56.174.114
static.askapache.com.        14400   IN      A       64.111.114.111
 
$ dig @ns1.dreamhost.com +authority +all static.askapache.com
 
;; ANSWER SECTION:
static.askapache.com.        14400   IN      A       64.111.114.111
static.askapache.com.        14400   IN      A       69.56.174.114

The effects of caching will distort the effectiveness of any IP address allocation algorithm unless a 0 TTL is used which has the effect of significantly increasing the load on the DNS (and is not always implemented consistently). In this case the cure may be worse than the disease Good news we have good load balancing on our web servers. Bad news we need 17 more DNS servers!. Intuitively, and without running any experiments to verify, we would suggest that given a normal TTL (12 hours or more) and ANY IP allocation algorithm other than a single static list, loads should be reasonably balanced .

Full root server query

Tracing to static.askapache.com[a] via A.ROOT-SERVERS.NET, maximum of 3 retries
A.ROOT-SERVERS.NET [.] (198.41.0.4)
 |\___ L.GTLD-SERVERS.NET [com] (192.41.162.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ D.GTLD-SERVERS.NET [com] (192.31.80.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ J.GTLD-SERVERS.NET [com] (192.48.79.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ H.GTLD-SERVERS.NET [com] (192.54.112.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ E.GTLD-SERVERS.NET [com] (192.12.94.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ F.GTLD-SERVERS.NET [com] (192.35.51.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ I.GTLD-SERVERS.NET [com] (192.43.172.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ G.GTLD-SERVERS.NET [com] (192.42.93.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ B.GTLD-SERVERS.NET [com] (2001:0503:231d:0000:0000:0000:0002:0030) Not queried
 |\___ B.GTLD-SERVERS.NET [com] (192.33.14.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ A.GTLD-SERVERS.NET [com] (2001:0503:a83e:0000:0000:0000:0002:0030) Not queried
 |\___ A.GTLD-SERVERS.NET [com] (192.5.6.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ C.GTLD-SERVERS.NET [com] (192.26.92.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ M.GTLD-SERVERS.NET [com] (192.55.83.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
  \___ K.GTLD-SERVERS.NET [com] (192.52.178.30)
       |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
       |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
        \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114

Live Online DNS Tools

More Reading

RFC’s related to DNS

  • RFC 920: Specified original TLDs: .arpa, .com, .edu, .org, .gov, .mil and two-character country codes
  • RFC 1032: Domain administrators guide
  • RFC 1033: Domain administrators operations guide
  • RFC 1034: Domain Names – Concepts and Facilities.
  • RFC 1035: Domain Names – Implementation and Specification
  • RFC 1101: DNS Encodings of Network Names and Other Types
  • RFC 1123: Requirements for Internet Hosts — Application and Support
  • RFC 1912: Common DNS Operational and Configuration Errors
  • RFC 1995: Incremental Zone Transfer in DNS
  • RFC 1996: A Mechanism for Prompt Notification of Zone Changes (DNS NOTIFY)
  • RFC 2136: Dynamic Updates in the domain name system (DNS UPDATE)
  • RFC 2181: Clarifications to the DNS Specification
  • RFC 2182: Selection and Operation of Secondary DNS Servers
  • RFC 2308: Negative Caching of DNS Queries (DNS NCACHE)
  • RFC 2317: Classless IN-ADDR.ARPA delegation
  • RFC 2671: Extension Mechanisms for DNS (EDNS0)
  • RFC 3597: Handling of Unknown DNS Resource Record (RR) Types
  • RFC 3696: Application Techniques for Checking and Transformation of Names
  • RFC 4343: Domain Name System (DNS) Case Insensitivity Clarification
  • RFC 4592: The Role of Wildcards in the Domain Name System
  • RFC 4892: Requirements for a Mechanism Identifying a Name Server Instance
  • RFC 5001: DNS Name Server Identifier Option (NSID)
  • RFC 5395: Domain Name System (DNS) IANA Considerations

This page contains content by Author of Article from Wikipedia and is licensed under the GNU FDL.

Table of Contents

DNS Round Robin Configuration using Rsync over SSH originally appeared on AskApache.com

]]> http://www.askapache.com/linux-unix/dns-round-robin-rsync-over-ssh.html/feed/ 5