WARNING! This is for advanced users of the shell, this is NOT a howto or tutorial. The thing is, I googled how to uninstall cpanel and for once in my life I came up empty.. And certainly the cpanel official sites themselves don't provide any instructions other than to say "Dont uninstall it, reinstall your entire operating system without it.".. they sure don't seem confident that they know exactly what their code is doing. Cpanel is great for most people, perfect for many situations, so don't get me wrong.. this is just for fun.

Why Uninstall?

Why? Because I have always built my servers, php installations, perl installs, ruby, iptables, everything from source. I read the INSTALL/README docs, I read the man pages, and I read the info pages as well. I google for configuration advice, I google for tips, and I don't need a web-based perl script messing my stuff up!

The main problems I had with cpanel, which really is a great bit of software for millions of website developers, is that it was incredibly sneaky! I used it for about 6 months and spent that entire time trying to figure out what the heck it was doing. A couple issues that I really disliked, it takes over your bind install, it takes over your apache install, it takes over your php install. And although it does let you configure some things (very few) for custom configurations and the like, I just don't need any of that. By removing the darn thing I am saving GIGS of space on my server, tons of bandwidth, and most importantly to me I am saving CPU and processing time along with RAM and IO speed.

Anything Else

Please.. I could go on for DAYS! Another reason I have wanted to be rid of cpanel is that I like my servers to be as lean and mean as possible. This means I like as few files and processes as possible. There are many benefits to this, like it's much easier for my integrity checking software and rootkit/antivirus software to run and drastically reduces the false positives. And there is that glaring security issue of constantly having cpanel run it's own software to create the WHM/Cpanel web interface, which is accessible online. I haven't researched cpanel security at all, it's possible that it never has security problems that are published, but for me, why take the chance?

Warning - Caution!

The following is just a quick reference of some of the commands I used to uninstall cpanel. This is for advanced users of the shell. If you aren't that advanced and you run a single one of these commands without fully understanding it, you will probably kill your server, probably lose everything on it permanently, probably not have a website or email for weeks.. So backup all your data FIRST. Also, if you aren't 100% sure you won't run into problems, you should contact your hosts technical support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for hosts, and the last thing tech support wants is to fix a server broken by someone who doesn't know what they are doing.

Last Warning!

Unless you understand what theses commands do and the purpose they serve, do not try any of this. These are not the exact commands I used verbatim, they are also not in order. I only put them up here because I was so amazed that google didn't have any uninstall cpanel intructions. Hopefully it's not a conspiracy that will get my site taken down.. ;)

That said and out of the way, it really only took me about 10 minutes to uninstall cpanel completely. But keep in mind I have been closely monitoring and debugging cpanel for 6 months, so I knew what I was doing. And finally, I do apologize for not having better instructions.. but hey, if you don't get this then you have no business trying to figure out how to uninstall cpanel! It's great software and shouldn't be removed unless you are fully capable of managing email/dns/www/ftp and any/all other servers and services on your machine by hand.

Do This First

I have a few drafts I'm working on at the moment with specifics, but for now you will have to figure it out with google. Basically you want to make sure you don't totally knock your machine offline without being able to reconnect. What I do is compile a static version of openssh and a few other security-type shell tools, and configure this binary sshd to run by using inittab, which is the file run by init (pid 1) and makes sure if it dies it is restarted. Here is my /etc/inittab to run the static sshd binary:

hh:12345:respawn:/failover/os/sbin/aassh -D -q -u0 -f /failover/os/etc/aassh

Another trick is to keep a detached screen logged in to root. That way if you mess up your sudoers or securetty or pam or whatever, you can just reattach and fix it.

Finally, you may want to setup your syslog to start earlier than usual, and set up more than normal verbosity. ( I take it to the max ). Then you should setup a 2nd server or machine somewhere to act as a syslog server. Lastly, configure your web server syslog to copy all messages to the remote syslog you set up. I use a reverse ssh tunnel to encrypt the syslog packets, but when I do something serious like reboot after uninstalling cpanel, I prepare for it by adding additional networking routes on my machine to make sure I will get some logs even if sshd cant start or even if my network addresses aren't brought up correctly.

If that sounds easy to you, please continue. If you are saying: Wha??? Continue in read-only mode.

Also, you can't just uninstall cpanel, I have replaced a lot of cpanel already, like building my own bind, apache, php, syslog and making sure they work and aren't being tampered with by cpanel. Basically cpanel runs everything on your server in most cases, so you should prepare by creating your own static software to replace cpanel, and make sure it works.

Find files Accessing /var/cpanel

More than likely these will need to be killed.

lsof +w -Rg -nP +c15 -x f +D /var/cpanel
lsof +w -Rg -nP +c15 -x f +D /usr/local/cpanel

Killing cpanel

Just an example, your machine may have a lot more than these, I have been slowly taking control of my machine back from cpanel for 6 months, so it was easier for me.

for P in tailwatchd queueprocd cpanellogd exim; do pkill -9 $P; done

Commands and Shortcuts

alias NF='nice find $PWD -mount -depth ! -type d'
alias NFF='nice find $PWD -mount -depth ! -type d | xargs -IF87 file F87'
alias NA='nice find $PWD -mount -depth'
alias NAF='nice find $PWD -mount -depth | xargs -IF87 file F87'

Watch out for crontab

An example of the sneakiness (from my POV, from most it's called builtin robustness) that cpanel does is automagically adding crontab entries that make it behave similarly to a self-propagating virus. If you don't disable the cronjobs and kill the right processes within a short period of time, be prepared for a magic resurrection.

Here's my awesome crontab information function, you will need to check every file, it lists the default crons on my box, and every users crontab, but it can't account for other cron software like at and other crons.

function askapache_crontab()
{
  local GG i;
  for i in `getent passwd|cut -d ":" -f1`;
  do
    GG=$(sudo crontab -u $i -l 2>$N6 | tr -s '\n\000' | sed '/^#/d');
    [[ ${#GG} -gt 3 ]] && sleep 1 && echo -e "$i \n\n${GG}"
  done;
   sleep 4;
   ls -aLls1ch --color=always /etc/cron.{hourly,daily,weekly,monthly,d} | sed '/^total/d; /\ drwxr-xr-x/d';
}

grep -ir /var/spool cpan

#6 3 * * * /scripts/upcp
#0 1 * * * /scripts/cpbackup
#0 2 * * * /scripts/mailman_chown_archives
#35 * * * * /usr/bin/test -x /usr/local/cpanel/bin/tail-check && /usr/local/cpanel/bin/tail-check
#11,26,41,56 * * * * /usr/local/cpanel/whostmgr/bin/dnsqueue > /dev/null 2>&1
#30 */4 * * * /usr/bin/test -x /scripts/update_db_cache && /scripts/update_db_cache
#45 */8 * * * /usr/bin/test -x /usr/local/cpanel/bin/optimizefs && /usr/local/cpanel/bin/optimizefs
#*/5 * * * * /usr/local/cpanel/bin/dcpumon >/dev/null 2>&1
#25 1 * * * /usr/local/cpanel/whostmgr/docroot/cgi/cpaddons_report.pl --notify

Delete Crontabs

sudo crontab -u mailman -r

Find INIT scripts with cpanel

This is the main startup script: /usr/local/cpanel/etc/init/startup

I had no idea ruby-on-rails was being controlled by cpanel.. sneaky bugger. You can tell by all of these advanced unix commands just how difficult it would be to uninstall cpanel, its totally like the Alien!

(1:3744)# find . ! -type d -print0|xargs -0 -I'F87' grep -Hi "cpan\|tailwat\|chkser" F87
./fastmail:# Author:       cPanel, Inc. <nick@cpanel.net>
./httpd:        HTTPD=/usr/local/cpanel/bin/chroothttpd
./cpanel:# cpanel8       Start Cpanel Services
./cpanel:# Author:       cPanel, Inc. <nick@cpanel.net>
./cpanel:# description: This is the cpanel webserver and chat.
./cpanel:# processname: cpaneld
./cpanel:# pidfile: /var/run/cpanel.pid
./cpanel:[ -f /usr/local/cpanel/etc/init/startup ] || exit 0
./cpanel:       if [ -f "/var/cpanel/smtpgidonlytweak" ]; then
./cpanel:       echo -n "Starting cPanel services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpsrvd
./cpanel:       echo -n "Starting cPanel brute force detector services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcphulkd
./cpanel:    echo -n "Starting cPanel dav services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpdavd
./cpanel:               daemon /usr/local/cpanel/etc/init/startcppop
./cpanel:       echo -n "Starting cPanel Chat services: "
./cpanel:               daemon /usr/local/cpanel/entropychat/entropychat
./cpanel:               daemon /usr/local/cpanel/bin/startmelange
./cpanel:                       /usr/local/cpanel/bin/startinterchange
./cpanel:       echo -n "Starting cPanel ssl services: "
./cpanel:       daemon /usr/local/cpanel/startstunnel
./cpanel:    echo -n "Starting cPanel Queue services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startqueueprocd
./cpanel:    echo -n "Starting tailwatchd: "
./cpanel:    daemon /usr/local/cpanel/libexec/tailwatchd --start
./cpanel:       echo -n "Starting cPanel Log services: "
./cpanel:       daemon /usr/local/cpanel/cpanellogd
./cpanel:    action "Starting mailman services: " /usr/local/cpanel/etc/init/startmailman
./cpanel:    action "Stopping tailwatchd: " /usr/local/cpanel/libexec/tailwatchd --stop
./cpanel:    action "Stopping cPanel services: " /usr/local/cpanel/etc/init/stopcpsrvd
./cpanel:       action "Stopping cPanel dav services: " /usr/local/cpanel/etc/init/stopcpdavd
./cpanel:       action "Stopping cPanel queue services: " /usr/local/cpanel/etc/init/stopqueueprocd
./cpanel:       action "Stopping cPanel brute force detector services: " /usr/local/cpanel/etc/init/stopcphulkd
./cpanel:               action "Stopping pop3 services: " /usr/local/cpanel/etc/init/stopcppop
./cpanel:       echo -n "Stopping cPanel log services: "
./cpanel:       killproc cpanellogd
./cpanel:       echo -n "Stopping cPanel Chat services: "
./cpanel:       action "Stopping cPanel ssl services: " /usr/local/cpanel/etc/init/stopstunnel
./cpanel:       action "Stopping mailman services: " /usr/local/cpanel/etc/init/stopmailman
./cpanel:       if [ -e "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl" ]; then
./exim:if [ -e "/etc/chkserv.d" ]; then
./exim:        for file in `ls /etc/chkserv.d`
./exim:            if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./exim:    if [ -x "/usr/local/cpanel/etc/init/startspamd" ]; then
./exim:        /usr/local/cpanel/etc/init/startspamd
./exim:        if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --statusall
./securetmp:# Author:       cPanel, Inc. <copyright@cpanel.net>

Turn off cpanel services

You should remove the below delete command and start by just disabling the inits by turning them off. Then reboot. Then delete. If your machine won't reboot, I told you so, Cpanel told you so, and likely your host told you so.

for S in cpanel ror securetmp fastmail exim; do R=$(command chkconfig --level 123456 $S off ||echo); R=$(command chkconfig --del $S ||echo); done

HTTPD

If you are running chrooted httpd then you'll need to make sure you don't delete your entire webserver on accident. Here's a relevant part from the /etc/init.d/httpd script.

# the path to your httpd binary, including options if necessary
if [ -e "/etc/chroothttpd" ]; then
        HTTPD=/usr/local/cpanel/bin/chroothttpd
else
        HTTPD=/usr/local/apache/bin/httpd
fi

Finding files owned by cpanel

Some super cool bash commands in this post.. let's start with one to find all the files and folders on your machine owned by cpanel. Check your /etc/passwd file for your machines specific usernames and groups. This command saves all the filenames to ~/cpanel-files-backup.txt, which is used by tar next to create a backup of all of them.

{ find / -mount -depth -maxdepth 150 \( -group cpanel -o   -group cpanel-phpmyadmin -o -group cpanel-phppgadmin   -o -group cpanelphpmyadmin   -o -group cpanelphppgadmin   -o -group cpanelhorde   -o -group cpanelroundcube \) -print; find / -mount -depth -maxdepth 150 \( -user cpanel -o   -user cpanel-phpmyadmin -o -user cpanel-phppgadmin   -o -user cpanelphpmyadmin   -o -user cpanelphppgadmin   -o -user cpanelhorde   -o -user cpanelroundcube \) -print; } > ~/cpanel-files-backup.txt

Here's another way to search directories.

 grep --color=always -Hir cpanel /var

Create the Backup

Note that you must have the latest version of tar for this exact command, also you should backup /var/cpanel and /usr/local/cpanel and /etc and heck the whole machine why dontcha!

tar -T ~/cpanel-files-backup.txt -cvz --checkpoint=1000 --checkpoint-action="ttyout=\rHit %s checkpoint #%u" -f /cpanel-files-backup.tgz --totals

Remove Files

Once you do this your upstream without a paddle, you better make sure you know what you're doing with this. This removes all those files.

cat ~/cpanel-files-backup.txt | xargs -I'F87' rm -vfr F87

Additionally you will want to remove /usr/local/cpanel and /var/cpanel - What I always do when running as root is alias my rm command to instead simply move the files to a .trash folder. That way if something goes bork you have a better chance at fixing it.

Find Group-Owned Files

find / -mount -depth -maxdepth 150 \
\(  -group cpanel -o \
  -group cpanel-phpmyadmin \
  -o -group cpanel-phppgadmin \
  -o -group cpanelphpmyadmin \
  -o -group cpanelphppgadmin \
  -o -group mailman \
  -o -group cpanelhorde \
  -o -group cpanelroundcube \
\) -fprintf /root/cpanel-group-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'

Find User-Owned Files

find / -mount -depth -maxdepth 150 \(
  -user cpanel \
  -o -user cpanel-phpmyadmin \
  -o -user cpanel-phppgadmin \
  -o -user cpanelphpmyadmin \
  -o -user cpanelphppgadmin \
  -o -user mailman \
  -o -user cpanelhorde \
  -o -user cpanelroundcube
\) -fprintf /root/cpanel-users-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'

       4  0755  drwxr-xr-x     cpanel:cpanel     32002:32004 /var/cpanel/userhomes/cpanel cpanel d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/tmp tmp d reiserfs
       4  0644  -rw-r--r-- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin cpanel-phpmyadmin d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/sessions sessions d reiserfs
       4  0644  -rw-r--r-- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin cpanelphppgadmin d reiserfs
       4  0750  drwxr-x--- cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/mail mail d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/sessions sessions d reiserfs
       4  0711  drwx--x--x cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube cpanelroundcube d reiserfs
       4  0644  -rw-r--r--     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches caches d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel .cpanel d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/tmp tmp d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/log log d reiserfs

Find Permissions

cat ~/cpanel-group-files.log ~/cpanel-users-files.log |tr -s '\000 \t'|cut -d' ' -f3|sort -u

Find files tailwatchd

(1:3732)# $NICE find ${1:-`pwd`} -mount -name '*tailwatch*'
/usr/local/cpanel/libexec/tailwatchd
/usr/local/cpanel/libexec/tailwatch
/usr/local/cpanel/libexec/tailwatch/tailwatchd
/usr/local/cpanel/etc/init/scripts/freebsd/tailwatchd.sh
/usr/local/cpanel/etc/init/scripts/trustix/tailwatchd
/usr/local/cpanel/etc/init/scripts/centos/tailwatchd
/usr/local/cpanel/etc/init/scripts/suse/tailwatchd
/usr/local/cpanel/etc/init/scripts/caos/tailwatchd
/usr/local/cpanel/etc/init/scripts/whitebox/tailwatchd
/usr/local/cpanel/etc/init/scripts/mandrake/tailwatchd
/usr/local/cpanel/etc/init/scripts/debian/tailwatchd
/usr/local/cpanel/etc/init/scripts/redhat/tailwatchd
/usr/local/cpanel/etc/init/scripts/fedora/tailwatchd
/usr/local/cpanel/etc/init/stoptailwatchd
/usr/local/cpanel/etc/init/starttailwatchd
/usr/local/cpanel/bin/tailwatchd
/usr/local/cpanel/logs/tailwatchd_log
/var/log/cpanel/tailwatchd_log
/var/cpanel/log_rotation/cp_tailwatchd_log.cpanellogd
/var/cpanel/tailwatch.positions
/var/run/tailwatchd.pid
/etc/chkserv.d/tailwatchd
/scripts/restartsrv_tailwatchd

Delete cpanel Users/Groups

for U in cpanel-phpmyadmin cpanel-phppgadmin cpanelphpmyadmin cpanelphppgadmin cpanelhorde cpanelroundcube machbuild; do userdel -fr $U; groupdel $U; done

Check for broken symlinks

find / -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'

Especially check /etc

$ find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'
/etc/ftpd-rsa.pem                   broken symbolic link to `/var/cpanel/ssl/ftp/ftpd-rsa.pem'
/etc/rc.d/rc1.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc1.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc1.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc3.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc6.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc5.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc2.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc2.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc2.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc4.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc4.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc4.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc0.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/authlib/authProg               broken symbolic link to `/usr/local/cpanel/bin/courier-auth'

And delete if you are sure

find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p' |cut -d' ' -f1|xargs -I'F87' rm -rvf 'F87'

Reinstall CSF

The only thing I actually used that came with cpanel is the CSF/LFD Firewall package, which is a fantastic piece of software. I had to reinstall this, and to get it working without cpanel add the following line to the csf.conf

GENERIC = "1"

Thats It

Now once you've cleaned up everything, you should try everything conceivable to get an error before rebooting. Like you should start and stop every service in /etc/init.d/, you should use telinit to check various runlevels (which keeps your sshd connection still live). Go all out, should take at least a full hour.

Another thing I like to do is rebuild alot of my source-built software again in case anything got messed up. I upgrade perl from cpanels 5.8.8 to 5.10, which is pretty thorough, and you know, reinstall anything else I think I might need. One of the benefits of compiling your own software is all I have to do is cd to the source directory and type make -B && ( { make test || make check || make checks || make tests; } || echo ) && sudo make install and that's it. The tests/checks are optional of course.

If anyone actually ever reads this and does it, please share your advice here.. everybody knows we need it! Good Luck

HOWTO: Uninstall CPANEL over SSH originally appeared on AskApache.com

Lightning Strikes Twice!

LOL..

3rd Dimension Broken with Bash!

So 3D... it'll hurt you eyes! lol.. Man I am cracking up here. Ha but seriously those really do look 3D for terminal though.. I am actually really impressed. Sweet. Here is the little function I wrote to output that grey marble.

Just a word to the wise, start learning and going over some of these concepts, especially the code used in functions, I will be back in a followup that details actually using this stu.

This function is one of my favorites because it is so fast and useful. Like when designing a 256color prompt.

aa_256 ()
{
    local o x=`tput op` y=`printf %$((${COLUMNS}-6))s`;
    for i in {0..256};
    do
        o=00$i;
        echo -e ${o:${#o}-3:3} `tputm "setaf $i" "setab $i"`${y// /=}$x;
    done
}

Ya this is actually not very helpful or useful, but there you have it.

tputm ()
{
    local a;
    for a in "$@";
    do
        echo -en "${a}\n";
    done | tput -S
}

Some people call this function the grey bringer of death. Not really.

a256 ()
{
    ( x=`tput op` y=`printf %$((${COLUMNS}-6))slocal `;
    for i in {242..232} 232 232;
    do
        echo -en "`tput setaf $i;tput setab $i`${y}${x}`tput op`";
    done )
}

Helpful Ncurses Programs

ALWAYS check out my functions!

This function will print out the terminal, show it's colors, etc.. I have some really nice ones in this article that I use for tmux, screen, and that sort of thing.

c ()
{
    tput clear;
    pm "$TERM: [colors:`tput colors`/`tput pairs`]";
    RC=`tput op` L1=$(L '=' $(( ${COLUMNS} - 25 )));
    for i in `seq ${1:-0} ${2:-16}`;
    do
        o="  $i";
        echo -e " ${o:${#o}-3:3} `tput setaf $i;tput setab $i`${L1}${RC}";
    done
}

Standard Capabilities

X.364 and iBCS2

• \033c - RIS - full reset
• \0337 - SC - save cursor
• \0338 - RC - restore cursor
• \033[r - RSR - not an X.364 mnemonic
• \033[m - SGR0 - not an X.364 mnemonic
• \033[2J - ED2 - clear page

Specified by ISO 2022

• \033(0 - ISO DEC G0 - enable DEC graphics for G0
• \033(A - ISO UK G0 - enable UK chars for G0
• \033(B - ISO US G0 - enable US chars for G0
• \033)0 - ISO DEC G1 - enable DEC graphics for G1
• \033)A - ISO UK G1 - enable UK chars for G1
• \033)B - ISO US G1 - enable US chars for G1

ISO 2022 charset switching:

• ISO/IEC 2022:1994
• "Character code structure and extension techniques"
• ECMA-035
• code structure
  • =00..=1F C0 set of control characters
  • =20..=7F G0 set of 94 or 96 graphic characters
  • =80..=9F C1 set of control characters "C1 controls"
  • =A0..=FF G1 set of 94 or 96 graphic characters
  • ... G2 and G3
• registration authority
  • ISO/IEC 2375:1985 Procedure for registration of escape sequences
  • "ECMA registry"
  • ISO-IR registration number
  • ftp://dkuug.dk/i18n/iso2375reg.txt
  • Internaltional register of coded character sets to be used with escape sequences
  • ISO/IEC JTC1/SC2/WG3
• ISO-2022 terminals from Siemens
• recode iso-2022..utf-8 not yet implemented
• CJK charsets
  • ISO-2022-JP
  • ISO-2022-JP-2
  • ISO-2022-KR
  • ISO-2022-CN
  • ISO-2022-CN-EXT

DEC private controls widely supported by emulators

• \033= - DECPAM - application keypad mode
• \033> - DECPNM - normal keypad mode
• \033< - DECANSI - enter ANSI mode
• \033[!p - DECSTR - soft reset
• \033 F - S7C1T - 7-bit controls

ECMA modes

ISO 2022

• 2 - AM - keyboard action mode
• 4 - IRM - insert/replace mode
• 12 - SRM - send/receive mode
• 20 - LNM - linefeed mode

DEC modes

• 1 - CKM - application cursor keys
• 2 - ANM - set VT52 mode
• 3 - COLM - 132-column mode
• 4 - SCLM - smooth scroll
• 5 - SCNM - reverse video mode
• 6 - OM - origin mode
• 7 - AWM - wraparound mode
• 8 - ARM - auto-repeat mode

ECMA attribute sequences

• 0 - NORMAL - normal
• 1 - +BOLD - bold on
• 2 - +DIM - dim on
• 3 - +ITALIC - italic on
• 4 - +UNDERLINE - underline on
• 5 - +BLINK - blink on
• 6 - +FASTBLINK - fastblink on
• 7 - +REVERSE - reverse on
• 8 - +INVISIBLE - invisible on
• 9 - +DELETED - deleted on
• 10 - MAIN-FONT - select primary font
• 11 - ALT-FONT-1 - select alternate font 1
• 12 - ALT-FONT-2 - select alternate font 2
• 13 - ALT-FONT-3 - select alternate font 3
• 14 - ALT-FONT-4 - select alternate font 4
• 15 - ALT-FONT-5 - select alternate font 5
• 16 - ALT-FONT-6 - select alternate font 6
• 17 - ALT-FONT-7 - select alternate font 7
• 18 - ALT-FONT-1 - select alternate font 1
• 19 - ALT-FONT-1 - select alternate font 1
• 20 - FRAKTUR - Fraktur font
• 21 - DOUBLEUNDER - double underline
• 22 - -DIM - dim off
• 23 - -ITALIC - italic off
• 24 - -UNDERLINE - underline off
• 25 - -BLINK - blink off
• 26 - -FASTBLINK - fastblink off
• 27 - -REVERSE - reverse off
• 28 - -INVISIBLE - invisible off
• 29 - -DELETED - deleted off

Init strings

Cap strings

Variable and Function Index

For stty

• BC: tgoto
• ospeed: Output Padding
• PC: Output Padding
• tgetent: Find
• tgetflag: Interrogate
• tgetnum: Interrogate
• tgetstr: Interrogate
• tgoto: tgoto
• tparam: tparam
• tputs: Output Padding
• UP: tgoto

Summary of Capability Names

Here are all the terminal capability names in alphabetical order with a brief description of each. For cross references to their definitions, see the index of capability names.

Variable and Function Index

• BC: tgoto
• ospeed: Output Padding
• PC: Output Padding
• tgetent: Find
• tgetflag: Interrogate
• tgetnum: Interrogate
• tgetstr: Interrogate
• tgoto: tgoto
• tparam: tparam
• tputs: Output Padding
• UP: tgoto

• ae: Standout
• AL: Insdel Line
• al: Insdel Line
• am: Wrapping
• as: Standout
• bc: Cursor Motion
• bl: Bell
• bs: Cursor Motion
• bt: Cursor Motion
• bw: Cursor Motion
• CC: Basic
• cd: Clearing
• ce: Clearing
• ch: Cursor Motion
• cl: Clearing
• CM: Cursor Motion
• cm: Cursor Motion
• co: Screen Size
• cr: Cursor Motion
• cS: Scrolling
• cs: Scrolling
• ct: Initialization
• cv: Cursor Motion
• da: Scrolling
• dB: Pad Specs
• db: Scrolling
• dC: Pad Specs
• DC: Insdel Char
• dc: Insdel Char
• dF: Pad Specs
• DL: Insdel Line
• dl: Insdel Line
• dm: Insdel Char
• dN: Pad Specs
• DO: Cursor Motion
• do: Cursor Motion
• ds: Status Line
• dT: Pad Specs
• ec: Clearing
• ed: Insdel Char
• ei: Insdel Char
• eo: Basic
• es: Status Line
• ff: Cursor Motion
• fs: Status Line
• gn: Basic
• hc: Basic
• hd: Half-Line
• ho: Cursor Motion
• hs: Status Line
• hu: Half-Line
• hz: Basic
• i1: Initialization
• i3: Initialization
• IC: Insdel Char
• ic: Insdel Char
• if: Initialization
• im: Insdel Char
• in: Insdel Char
• ip: Insdel Char
• is: Initialization
• it: Initialization
• K1...K5: Keypad
• k1...k9: Keypad
• kA...kT: Keypad
• ka...ku: Keypad
• km: Meta Key
• l0...l9: Keypad
• LE: Cursor Motion
• le: Cursor Motion
• li: Screen Size
• ll: Cursor Motion
• lm: Scrolling
• LP: Wrapping
• mb: Standout
• md: Standout
• me: Standout
• mh: Standout
• mi: Insdel Char
• mk: Standout
• mm: Meta Key
• mo: Meta Key
• mp: Standout
• mr: Standout
• ms: Underlining
• ms: Standout
• nc: Cursor Motion
• nd: Cursor Motion
• NF: Initialization
• nl: Cursor Motion
• ns: Scrolling
• nw: Cursor Motion
• os: Basic
• pb: Pad Specs
• pc: Pad Specs
• pf: Printer
• pO: Printer
• po: Printer
• ps: Printer
• rc: Cursor Motion
• RI: Cursor Motion
• rp: Basic
• rs: Initialization
• sa: Standout
• sc: Cursor Motion
• se: Standout
• SF: Scrolling
• sf: Scrolling
• sg: Standout
• so: Standout
• SR: Scrolling
• sr: Scrolling
• st: Initialization
• ta: Cursor Motion
• te: Initialization
• ti: Initialization
• ts: Status Line
• uc: Underlining
• ue: Underlining
• ug: Underlining
• ul: Underlining
• UP: Cursor Motion
• up: Cursor Motion
• us: Underlining
• vb: Bell
• ve: Cursor Visibility
• vi: Cursor Visibility
• vs: Cursor Visibility
• wi: Windows
• ws: Status Line
• xb: Basic
• xn: Wrapping
• xs: Standout
• xt: Standout
• xt: Cursor Motion

• Basic: Basic characteristics.
• Screen Size: Screen size, and what happens when it changes.
• Cursor Motion: Various ways to move the cursor.
• Wrapping: What happens if you write a character in the last column.
• Scrolling: Pushing text up and down on the screen.
• Windows: Limiting the part of the window that output affects.
• Clearing: Erasing one or many lines.
• Insdel Line: Making new blank lines in mid-screen; deleting lines.
• Insdel Char: Inserting and deleting characters within a line.
• Standout: Highlighting some of the text.
• Underlining: Underlining some of the text.
• Cursor Visibility: Making the cursor more or less easy to spot.
• Bell: Attracts user's attention; not localized on the screen.
• Keypad: Recognizing when function keys or arrows are typed.
• Meta Key: META acts like an extra shift key.
• Initialization: Commands used to initialize or reset the terminal.
• Pad Specs: Info for the kernel on how much padding is needed.
• Status Line: A status line displays background information.
• Half-Line: Moving by half-lines, for superscripts and subscripts.
• Printer: Controlling auxiliary printers of display terminals.

Translated Term Capabilities

This is helpful so you can understand what a term cap does, by looking at the long name.

List of All Terminals

You can view this list with the following command:

$ toe -a|sort -d

VT100 keypad Diagram

VT100 LINK

Here's a diagram of the VT100 keypad keys with their bindings. The top line is the name of the key (some DEC keyboards have the keys labelled somewhat differently, like GOLD instead of PF1, but this is the most "official" name). The second line is the escape sequence it generates in Application Keypad mode (where "$" means the ESC character). The third line contains two items, first the mapping of the key in terminfo, and then in termcap.

   _______________________________________
  |   PF1   |   PF2   |   PF3   |   PF4   |
  |   $OP   |   $OQ   |   $OR   |   $OS   |
  |_kf1__k1_|_kf2__k2_|_kf3__k3_|_kf4__k4_|
  |    7         8         9         -    |
  |   $Ow   |   $Ox   |   $Oy   |   $Om   |
  |_kf9__k9_|_kf10_k;_|_kf0__k0_|_________|
  |    4    |    5    |    6    |    ,    |
  |   $Ot   |   $Ou   |   $Ov   |   $Ol   |
  |_kf5__k5_|_kf6__k6_|_kf7__k7_|_kf8__k8_|
  |    1    |    2    |    3    |         |
  |   $Oq   |   $Or   |   $Os   |  enter  |
  |_ka1__K1_|_kb2__K2_|_ka3__K3_|  $OM    |
  |         0         |   .     |         |
  |        $Op        |  $On    |         |
  |___kc1_______K4____|_kc3__K5_|_kent_@8_|

A better adaptation to modern keyboards such as the PC's, which have a dozen function keys and the keypad 2,4,6,8 keys are labeled with arrows keys, is to use the 5-key arrangement to model the arrow keys as suggested in the terminfo guidelines:

   _______________________________________
  |   PF1   |   PF2   |   PF3   |   PF4   |
  |   $OP   |   $OQ   |   $OR   |   $OS   |
  |_kf1__k1_|_kf2__k2_|_kf3__k3_|_kf4__k4_|
  |    7         8         9         -    |
  |   $Ow   |   $Ox   |   $Oy   |   $Om   |
  |_ka1__K1_|_________|_ka3__K3_|_________|
  |    4    |    5    |    6    |    ,    |
  |   $Ot   |   $Ou   |   $Ov   |   $Ol   |
  |_________|_kb2__K2_|_________|_________|
  |    1    |    2    |    3    |         |
  |   $Oq   |   $Or   |   $Os   |  enter  |
  |_kc1__K4_|_________|_kc3__K5_|  $OM    |
  |         0         |   .     |         |
  |        $Op        |  $On    |         |
  |___________________|_________|_kent_@8_|

SGR parameter values

Sequence     Sequence                             Parameter   or
Mnemonic     Name              Sequence           Value      Mode   terminfo
-----------------------------------------------------------------------------
APC  Applicatn Program Command \E _                -         Delim  -
BEL  Bell *                    ^G                  -         -      bel
BPH  Break Permitted Here *    \E B                -         *      -
BS   Backpace *                ^H                  -         EF     -
CAN  Cancel *                  ^X                  -         -      -   (A)
CBT  Cursor Backward Tab       \E [ Pn Z           1         eF     cbt
CCH  Cancel Previous Character \E T                -         -      -
CHA  Cursor Horizntal Absolute \E [ Pn G           1         eF     hpa (B)
CHT  Cursor Horizontal Tab     \E [ Pn I           1         eF     tab (C)
CMD  Coding Method Delimiter * \E
CNL  Cursor Next Line          \E [ Pn E           1         eF     nel (D)
CPL  Cursor Preceding Line     \E [ Pn F           1         eF     -
CPR  Cursor Position Report    \E [ Pn ; Pn R      1, 1      -      -   (E)
CSI  Control Sequence Intro    \E [                -         Intro  -
CTC  Cursor Tabulation Control \E [ Ps W           0         eF     -   (F)
CUB  Cursor Backward           \E [ Pn D           1         eF     cub
CUD  Cursor Down               \E [ Pn B           1         eF     cud
CUF  Cursor Forward            \E [ Pn C           1         eF     cuf
CUP  Cursor Position           \E [ Pn ; Pn H      1, 1      eF     cup (G)
CUU  Cursor Up                 \E [ Pn A           1         eF     cuu
CVT  Cursor Vertical Tab       \E [ Pn Y           -         eF     -   (H)
DA   Device Attributes         \E [ Pn c           0         -      -
DAQ  Define Area Qualification \E [ Ps o           0         -      -
DCH  Delete Character          \E [ Pn P           1         eF     dch
DCS  Device Control String     \E P                -         Delim  -
DL   Delete Line               \E [ Pn M           1         eF     dl
DLE  Data Link Escape *        ^P                  -         -      -
DMI  Disable Manual Input      \E \                -         Fs     -
DSR  Device Status Report      \E [ Ps n           0         -      -   (I)
DTA  Dimension Text Area *     \E [ Pn ; Pn SPC T  -         PC     -
EA   Erase in Area             \E [ Ps O           0         eF     -   (J)
ECH  Erase Character           \E [ Pn X           1         eF     ech
ED   Erase in Display          \E [ Ps J           0         eF     ed  (J)
EF   Erase in Field            \E [ Ps N           0         eF     -
EL   Erase in Line             \E [ Ps K           0         eF     el  (J)
EM   End of Medium *           ^Y                  -         -      -
EMI  Enable Manual Input       \E b                          Fs     -
ENQ  Enquire                   ^E                  -         -      -
EOT  End Of Transmission       ^D                  -         *      -
EPA  End of Protected Area     \E W                -         -      -   (K)
ESA  End of Selected Area      \E G                -         -      -
ESC  Escape                    ^[                  -         -      -
ETB  End Transmission Block    ^W                  -         -      -
ETX  End of Text               ^C                  -         -      -
FF   Form Feed                 ^L                  -         -      -
FNK  Function Key *            \E [ Pn SPC W       -         -      -
GCC  Graphic Char Combination* \E [ Pn ; Pn SPC B  -         -      -
FNT  Font Selection            \E [ Pn ; Pn SPC D  0, 0      FE     -
GSM  Graphic Size Modify       \E [ Pn ; Pn SPC B  100, 100  FE     -   (L)
GSS  Graphic Size Selection    \E [ Pn SPC C       none      FE     -
HPA  Horz Position Absolute    \E [ Pn `           1         FE     -   (B)
HPB  Char Position Backward    \E [ j              1         FE     -
HPR  Horz Position Relative    \E [ Pn a           1         FE     -   (M)
HT   Horizontal Tab *          ^I                  -         FE     -   (N)
HTJ  Horz Tab w/Justification  \E I                -         FE     -
HTS  Horizontal Tab Set        \E H                -         FE     hts
HVP  Horz & Vertical Position  \E [ Pn ; Pn f      1, 1      FE     -   (G)
ICH  Insert Character          \E [ Pn @           1         eF     ich
IDCS ID Device Control String  \E [ SPC O          -         *      -
IGS  ID Graphic Subrepertoire  \E [ SPC M          -         *      -
IL   Insert Line               \E [ Pn L           1         eF     il
IND  Index                     \E D                -         FE     -
INT  Interrupt                 \E a                -         Fs     -
JFY  Justify                   \E [ Ps SPC F       0         FE     -
IS1  Info Separator #1 *       ^_                  -         *      -
IS2  Info Separator #1 *       ^^                  -         *      -
IS3  Info Separator #1 *       ^]                  -         *      -
IS4  Info Separator #1 *       ^\                  -         *      -
LF   Line Feed                 ^J                  -         -      -
LS1R Locking Shift Right 1 *   \E ~                -         -      -
LS2  Locking Shift 2 *         \E n                -         -      -
LS2R Locking Shift Right 2 *   \E }                -         -      -
LS3  Locking Shift 3 *         \E o                -         -      -
LS3R Locking Shift Right 3 *   \E |                -         -      -
MC   Media Copy                \E [ Ps i           0         -      -   (S)
MW   Message Waiting           \E U                -         -      -
NAK  Negative Acknowledge *    ^U                  -         *      -
NBH  No Break Here *           \E C                -         -      -
NEL  Next Line                 \E E                -         FE     nel (D)
NP   Next Page                 \E [ Pn U           1         eF     -
NUL  Null *                    ^@                  -         -      -
OSC  Operating System Command  \E ]                -         Delim  -
PEC  Pres. Expand/Contract *   \E Pn SPC Z         0         -      -
PFS  Page Format Selection *   \E Pn SPC J         0         -      -
PLD  Partial Line Down         \E K                -         FE     -   (T)
PLU  Partial Line Up           \E L                -         FE     -   (U)
PM   Privacy Message           \E ^                -         Delim  -
PP   Preceding Page            \E [ Pn V           1         eF     -
PPA  Page Position Absolute *  \E [ Pn SPC P       1         FE     -
PPB  Page Position Backward *  \E [ Pn SPC R       1         FE     -
PPR  Page Position Forward *   \E [ Pn SPC Q       1         FE     -
PTX  Parallel Texts *          \E [ \              -         -      -
PU1  Private Use 1             \E Q                -         -      -
PU2  Private Use 2             \E R                -         -      -
QUAD Typographic Quadding      \E [ Ps SPC H       0         FE     -
REP  Repeat Char or Control    \E [ Pn b           1         -      rep
RI   Reverse Index             \E M                -         FE     -   (V)
RIS  Reset to Initial State    \E c                -         Fs     -
RM   Reset Mode *              \E [ Ps l           -         -      -   (W)
SACS Set Add. Char. Sep. *     \E [ Pn SPC /       0         -      -
SAPV Sel. Alt. Present. Var. * \E [ Ps SPC ]       0         -      -   (X)
SCI  Single-Char Introducer    \E Z                -         -      -
SCO  Sel. Char. Orientation *  \E [ Pn ; Pn SPC k  -         -      -
SCS  Set Char. Spacing *       \E [ Pn SPC g       -         -      -
SD   Scroll Down               \E [ Pn T           1         eF     rin
SDS  Start Directed String *   \E [ Pn ]           1         -      -
SEE  Select Editing Extent     \E [ Ps Q           0         -      -   (Y)
SEF  Sheet Eject & Feed *  \E [ Ps ; Ps SPC Y  0,0       -      -
SGR  Select Graphic Rendition  \E [ Ps m           0         FE     sgr (O)
SHS  Select Char. Spacing *    \E [ Ps SPC K       0         -      -
SI   Shift In                  ^O                  -         -      -   (P)
SIMD Sel. Imp. Move Direct. *  \E [ Ps ^           -         -      -
SL   Scroll Left               \E [ Pn SPC @       1         eF     -
SLH  Set Line Home *           \E [ Pn SPC U       -         -      -
SLL  Set Line Limit *          \E [ Pn SPC V       -         -      -
SLS  Set Line Spacing *        \E [ Pn SPC h       -         -      -
SM   Select Mode               \E [ Ps h           none      -      -   (W)
SO   Shift Out                 ^N                  -         -      -   (Q)
SOH  Start Of Heading *        ^A                  -         -      -
SOS  Start of String *         \E X                -         -      -
SPA  Start of Protected Area   \E V                -         -      -   (Z)
SPD  Select Pres. Direction *  \E [ Ps ; Ps SPC S  0,0       -      -
SPH  Set Page Home *           \E [ Ps SPC G       -         -      -
SPI  Spacing Increment         \E [ Pn ; Pn SPC G  none      FE     -
SPL  Set Page Limit *          \E [ Ps SPC j       -         -      -
SPQR Set Pr. Qual. & Rapid. *  \E [ Ps SPC X       0         -      -
SR   Scroll Right              \E [ Pn SPC A       1         eF     -
SRCS Set Reduced Char. Sep. *  \E [ Pn SPC f       0         -      -
SRS  Start Reversed String *   \E [ Ps [           0         -      -
SSA  Start of Selected Area    \E F                -         -      -
SSU  Select Size Unit *        \E [ Pn SPC I       0         -      -
SSW  Set Space Width *         \E [ Pn SPC [       none      -      -
SS2  Single Shift 2 (G2 set)   \E N                -         Intro  -
SS3  Single Shift 3 (G3 set)   \E O                -         Intro  -
ST   String Terminator         \E \                -         Delim  -
STAB Selective Tabulation *    \E [ Pn SPC ^       -         -      -
STS  Set Transmit State        \E S                -         -      -
STX  Start pf Text *           ^B                  -         -      -
SU   Scroll Up                 \E [ Pn S           1         eF     indn
SUB  Substitute *              ^Z                  -         -      -
SVS  Select Line Spacing *     \E [ Pn SPC \       1         -      -
SYN  Synchronous Idle *        ^F                  -         -      -
TAC  Tabul. Aligned Centered * \E [ Pn SPC b       -         -      -
TALE Tabul. Al. Leading Edge * \E [ Pn SPC a       -         -      -
TATE Tabul. Al. Trailing Edge* \E [ Pn SPC `       -         -      -
TBC  Tab Clear                 \E [ Ps g           0         FE     tbc
TCC  Tabul. Centered on Char * \E [ Pn SPC c       -         -      -
TSR  Tabulation Stop Remove  * \E [ Pn SPC d       -         FE     -
TSS  Thin Space Specification  \E [ Pn SC E        none      FE     -
VPA  Vert. Position Absolute   \E [ Pn d           1         FE     vpa
VPB  Line Position Backward *  \E [ Pn k           1         FE     -
VPR  Vert. Position Relative   \E [ Pn e           1         FE     -   (R)
VPR  Vert. Position Relative   \E [ Pn e           1         FE     -   (R)
VT   Vertical Tabulation *     ^K                  -         FE     -
VTS  Vertical Tabulation Set   \E J                -         FE     -

Basic Characteristics

This section documents the capabilities that describe the basic and nature of the terminal, and also those that are relevant to the output of graphic characters.

7bit vs. 8bit

These recommendations are optional. IBCS2 allows the leading escape to # be either the 7-bit \E[ or 8-bit \0233 introducer, in accordance with # the ANSI X.364/ISO 6429/ECMA-48 standard.

1.6.1 Describing the Encoding

For example, the cm string for a standard ANSI terminal is written as \E[%i%d;%dH. (\E stands for .) cm by convention always requires two parameters, the vertical and horizontal goal positions, so this string specifies the encoding of two parameters. Here %i increments the two values supplied, and each %d encodes one of the values in decimal. If the cursor position values 20,58 are encoded with this string, the result is \E[21;59H.

First, here are the %-sequences that generate output. Except for %%, each of them encodes one parameter and advances the pointer to the following parameter.

• %% Output a single %. This is the only way to represent a literal % in a terminal command with parameters. %% does not use up a parameter.
• %d As in printf, output the next parameter in decimal.
• %2 Like %02d in printf: output the next parameter in decimal, and always use at least two digits.
• %3 Like %03d in printf: output the next parameter in decimal, and always use at least three digits. Note that %4 and so on are _not_ defined.
• %. Output the next parameter as a single character whose ASCII code is the parameter value. Like %c in printf.
• %+CHAR Add the next parameter to the character CHAR, and output the resulting character. For example, %+ represents 0 as a space, 1 as !, etc.

The following %-sequences specify alteration of the parameters (their values, or their order) rather than encoding a parameter for output. They generate no output; they are used only for their side effects on the parameters. Also, they do not advance the "next parameter" pointer except as explicitly stated. Only %i, %r and %> are defined in standard Unix termcap. The others are GNU extensions.

• %i Increment the next two parameters. This is used for terminals that expect cursor positions in origin 1. For example, %i%d,%d would output two parameters with 1 for 0, 2 for 1, etc.
• %r Interchange the next two parameters. This is used for terminals whose cursor positioning command expects the horizontal position first.
• %s Skip the next parameter. Do not output anything.
• %b Back up one parameter. The last parameter used will become once again the next parameter to be output, and the next output command will use it. Using %b more than once, you can back up any number of parameters, and you can refer to each parameter any number of times.
• %>C1C2 Conditionally increment the next parameter. Here C1 and C2 are characters which stand for their ASCII codes as numbers. If the next parameter is greater than the ASCII code of C1, the ASCII code of C2 is added to it.
• %a OP TYPE POS Perform arithmetic on the next parameter, do not use it up, and do not output anything. Here OP specifies the arithmetic operation, while TYPE and POS together specify the other operand.

Spaces are used above to separate the operands for clarity; the spaces don't appear in the data base, where this sequence is exactly five characters long. The character OP says what kind of arithmetic operation to perform. It can be any of these characters:

• = - assign a value to the next parameter, ignoring its old value. The new value comes from the other operand.
• + - add the other operand to the next parameter.
• - - subtract the other operand from the next parameter.
• * - multiply the next parameter by the other operand.
• / - divide the next parameter by the other operand.

The "other operand" may be another parameter's value or a constant; the character TYPE says which. It can be:

• p - Use another parameter. The character POS says which parameter to use. Subtract 64 from its ASCII code to get the position of the desired parameter relative to this one. Thus, the character A as POS means the parameter after the next one; the character ? means the parameter before the next one.
• c - Use a constant value. The character POS specifies the value of the constant. The 0200 bit is cleared out, so that 0200 can be used to represent zero.

The following %-sequences are special purpose hacks to compensate for the weird designs of obscure terminals. They modify the next parameter or the next two parameters but do not generate output and do not use up any parameters. %m is a GNU extension; the others are defined in standard Unix termcap.

• %n - Exclusive-or the next parameter with 0140, and likewise the parameter after next.
• %m - Complement all the bits of the next parameter and the parameter after next.
• %B - Encode the next parameter in BCD. It alters the value of the parameter by adding six times the quotient of the parameter by ten. Here is a C statement that shows how the new value is computed: PARM = (PARM / 10) * 16 + PARM % 10;
• %D - Transform the next parameter as needed by Delta Data terminals. This involves subtracting twice the remainder of the parameter by 16. PARM -= 2 * (PARM % 16);

Terminal Type Name Conventions

Here is a list of standard suffixes and their conventional meanings:

• -w - Short for "wide". This is a mode that gives the terminal more columns than usual. This is normally a user option.
• -am - "Automatic margins". This is an alternate description for use when the terminal's margin-wrap switch is on; it contains the am flag. The implication is that normally the switch is off and the usual description for the terminal says that the switch is off.
• -nam - "No automatic margins". The opposite of -am, this names an alternative description which lacks the am flag. This implies that the terminal is normally operated with the margin-wrap switch turned on, and the normal description of the terminal says so.
• -na - "No arrows". This terminal description initializes the terminal to keep its arrow keys in local mode. This is a user option.
• -rv - "Reverse video". This terminal description causes text output for normal video to appear as reverse, and text output for reverse video to come out as normal. Often this description differs from the usual one by interchanging the two strings which turn reverse video on and off.

This is a user option; you can choose either the "reverse video" variant terminal type or the normal terminal type, and termcap will obey.

• -s - "Status". Says to enable use of a status line which ordinary output does not touch (*note Status Line::).

Some terminals have a special line that is used only as a status line. For these terminals, there is no need for an -s variant; the status line commands should be defined by default. On other terminals, enabling a status line means removing one screen line from ordinary use and reducing the effective screen height. For these terminals, the user can choose the -s variant type to request use of a status line.

• -NLINES - Says to operate with NLINES lines on the screen, for terminals such as the Ambassador which provide this as an option. Normally this is a user option; by choosing the terminal type, you control how many lines termcap will use.
• -NPAGESp - Says that the terminal has NPAGES pages worth of screen memory, for terminals where this is a hardware option.
• -unk - Says that description is not for direct use, but only for reference in tc capabilities. Such a description is a kind of subroutine, because it describes the common characteristics of several variant descriptions that would use other suffixes in place of -unk.

MS-DOS ANSI.SYS

Here is a description of the color and attribute controls supported in the ANSI.SYS driver under MS-DOS. Most console drivers and ANSI terminal emulators for Intel boxes obey these. They are a proper subset of the ECMA-48 escapes.

Bright black becomes gray, bright brown becomes yellow. These coincide with the prescriptions of the ISO 6429/ECMA-48 standard. * If the 5 attribute is on and you set a background color (40-47) it is supposed to enable bright background.

0        all attributes off
1        foreground bright
4        underscore on
5        blink on/background bright (not reliable with brown)
7        reverse-video
8        set blank (non-display)
10       set primary font
11       set first alternate font (on PCs, display ROM characters 1-31)
12       set second alternate font (on PCs, display IBM high-half chars)
3n       set foreground color       / 0=black, 1=red,     2=green, 3=brown,
4n       set background color       \ 4=blue,  5=magenta, 6=cyan,  7=white

STTY

Stty Control Settings

Stty Input settings

Stty Output settings

These arguments specify output-related operations.

Stty Local settings

Stty Combination settings

cread -ignbrk brkint -inlcr -igncr icrnl -ixoff -iuclc -ixany imaxbel opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0 isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke

-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -opost -isig -icanon -xcase min 1 time 0

Special characters

The special characters' default values vary from system to system. They are set with the syntax name value, where the names are listed below and the value can be given either literally, in hat notation (^C), or as an integer which may start with 0x to indicate hexadecimal, 0 to indicate octal, or any other digit to indicate decimal.

For GNU stty, giving a value of ^- or undef disables that special character. (This is incompatible with Ultrix stty, which uses a value of u to disable a special character. GNU stty treats a value u like any other, namely to set that special character to .)

Special settings

• min N - Set the minimum number of characters that will satisfy a read until the time value has expired, when -icanon is set.
• time N - Set the number of tenths of a second before reads time out if the minimum number of characters have not been read, when -icanon is set.
• ispeed N - Set the input speed to N.
• ospeed N - Set the output speed to N.
• rows N - Tell the tty kernel driver that the terminal has N rows. Non-POSIX.
• cols N, columns N - Tell the kernel that the terminal has N columns. Non-POSIX.
• size - Print the number of rows and columns that the kernel thinks the terminal has. (Systems that don't support rows and columns in the kernel typically use the environment variables LINES and COLUMNS instead; however, GNU stty does not know anything about them.) Non-POSIX.
• line N - Use line discipline N. Non-POSIX.
• speed - Print the terminal speed.
• N - Set the input and output speeds to N. N can be one of: 0 50 75 110 134 134.5 150 200 300 600 1200 1800 2400 4800 9600 19200 38400 exta extb. exta is the same as 19200; extb is the same as 38400. 0 hangs up the line if -clocal is set.

Color Terminals

color256 ()
{
    local L=$(sed 's/[0-9]//g; s/./#/g' <<<`seq -s+0 $(($COLUMNS - 10))`);
    for i in `seq 0 $(tput colors)`;
    do printf " %.3d `tput setab $i`${L}${R} %.3d `tput setab 0;tput setaf 7`\n" $i $i | tr '#' ' ';
    done
}

for F in `find ~/.terminfo/ -type f | sed 's/^.*\///' | xargs -P0 -I'TT' sh -c 'echo "$(tput -T TT colors): TT"'|sed '/^-/d'| grep ^16:|cut -d: -f2`; do
echo -e "\n\n";
pm $F; export TERM=$F; tput -T $F initc; color256; done

find ~/.terminfo/ -type f | sed 's/^.*\///' | xargs -I'TT' sh -c 'echo "$(tput -T TT colors): TT"'|sed '/^-/d'

Terminals with 16 Colors

• aixterm-16color
• amiga-vnc
• hp2397
• hp2397a
• hp+color
• ibm+16color
• konsole-16color
• nsterm-16color
• nsterm-7-c
• nsterm-7-c-s
• nsterm-acs-c
• nsterm-acs-c-s
• nsterm-c
• nsterm+c
• nsterm-c-7
• nsterm-c-acs
• nsterm-c-s
• nsterm-c-s-7
• nsterm-c-s-acs
• rxvt-16color
• screen-16color
• screen-16color-bce
• screen-16color-bce-s
• screen-16color-s
• xterm-16color

Terminals with 52 Colors

• d430c-dg-ccc
• d430c-unix-25-ccc
• d430c-unix-ccc
• d430c-unix-s-ccc
• d430c-unix-sr-ccc
• d430c-unix-w-ccc
• d430-dg-ccc
• d430-unix-25-ccc
• d430-unix-ccc
• d430-unix-s-ccc
• d430-unix-sr-ccc
• d430-unix-w-ccc
• dg+ccc
• dgunix+ccc

Terminals with 64 Colors

• hpterm-color
• wy370
• wy370-101k
• wy370-105k
• wy370-EPC
• wy370-nk
• wy370-rv
• wy370-vb
• wy370-w
• wy370-wvb
• wyse370
• Eterm-88color
• rxvt-88color
• xterm-88color
• xterm+88color

Terminals with 256 colors

• Eterm-256color
• gnome-256color
• konsole-256color
• putty-256color
• rxvt-256color
• screen-256color
• screen-256color-bce
• screen-256color-bce-s
• screen-256color-s
• xterm-256color
• xterm+256color

View All tput capabilities

http://blogs.msdn.com/adioltean/articles/271063.aspx

function tputs(){ infocmp -1|sed 's/^[   ]*//; /=/!d; s/=.*//g; /[#\|]/d'; }

Some common terminfo parameter sequences, their termcap equivalents, and some terminal types which commonly have such sequences, are:

terminfo                    termcap   Representative Terminals
---------------------------------------------------------------
%p1%c                       %.        adm
%p1%d                       %d        hp, ANSI standard, vt100
%p1%+%c                 %+x       concept
%i                          %iq       ANSI standard, vt100
%p1%?%>%t%p1%+%;    %>xy      concept
%p2 is printed before %p1   %r        hp

Installing your Own Terminfo

MostLike, or if that is offline I uploaded a copy: mostlike - manpages with color looking like most

rm -rvf ~/.terminfo
mkdir -pv ~/.terminfo
curl -o terminfo.master http://nion.modprobe.de/mostlike.txt; curl -Ss ftp://invisible-island.net/ncurses/terminfo.src.gz
export TERMINFO=~/.terminfo;
sudo tic -e screen-256color,screen-256color-s,screen-256color-bce,screen-256color-bce-s,xterm+256color,xterm,xterm-pcolor terminfo.src

256 Colors in Terminal

256 Colors (Background)

W=`tput setaf 7` RC="\E[0;0;0m" L=$(sed 's/[0-9]//g; s/./ /g' <<<`seq -s+0 $(($COLUMNS/2))`);
 
for i in `seq 0 256`;
do
  printf "${W}\n%.3d `tput setab $i`${L}${RC}" $i;
done

256 Colors (Foreground)

W=`tput setaf 7` WB=`tput setab 0` RC="\E[0;0;0m" L=$(sed 's/[0-9]//g; s/./#/g' <<<`seq -s+0 $(($COLUMNS/2))`);
for i in `seq 0 256`;
do
  printf "${WB}\n%.3d `tput setaf $i`${L}${RC}" $i;
done

Terminal Capabilities - Infocmp

infocmp -1Lq|grep -v "$TERM\|#"|tr -d '  '

DEC/ANSI special sequences

List of the DEC/ANSI special sequences recognized:

Term Sorted

./MKtermsort.sh |sed 's/.*\/\*  \(.*\)  \*\//\1/g'|sort -u|grep ^[a-zA-Z]|sortwc|grep -v ^static|sort -d|tr " " ','|sed 's/\([^,]\{1,\}\),/<var>\1<\/var>,/g'

AB,ac,acs_btee,acsc,acs_chars,acs_hline,acs_llcorner,acs_lrcorner,acs_ltee,acs_plus,acs_rtee,acs_ttee,acs_ulcorner,acs_urcorner,acs_vline,ae,AF,al,AL,alt_scancode_esc,am,apstr,arrow_key_map,as,auto_left_margin,auto_right_margin,back_color_erase,backspace_delay,backspace_if_not_bs,backspaces_with_bs,back_tab,batt1,batt2,bc,bce,bel,bell,bicr,binel,birep,bit_image_carriage_return,bit_image_entwining,bit_image_newline,bit_image_repeat,bit_image_type,bitwin,bitype,bl,blink,bold,box1,box2,box_chars_1,bs,bt,BT,btml,btns,buffer_capacity,bufsz,buttons,bw,bx,can_change,carriage_return,carriage_return_delay,cb,cbt,cc,CC,ccc,cd,ce,ceol_standout_glitch,ch,change_char_pitch,change_line_pitch,change_res_horz,change_res_vert,change_scroll_region,char_padding,char_set_names,chr,chts,ci,civis,cl,clear,clear_all_tabs,clear_margins,clear_screen,clr_bol,clr_eol,clr_eos,cm,CM,cmdch,cnorm,co,Co,code_set_init,col_addr_glitch,colb0,colb1,colb2,colb3,colb4,colb5,colb6,colb7,colf0,colf1,colf2,colf3,colf4,colf5,colf6,colf7,color_names,colornm,colors,cols,column_address,columns,command_character,cpi,cpi_changes_res,cpix,cps,cr,cr_cancels_micro_mode,create_window,crt_no_scrolling,crxm,cs,csin,csnm,csr,ct,cub,cub1,cud,cud1,cuf,cuf1,cup,cursor_address,cursor_down,cursor_home,cursor_invisible,cursor_left,cursor_mem_address,cursor_normal,cursor_right,cursor_to_ll,cursor_up,cursor_visible,cuu,cuu1,cv,cvr,cvvis,CW,cwin,da,daisy,db,dB,dc,dC,DC,dch,dch1,dclk,defbi,defc,define_bit_image_region,define_char,delete_character,delete_line,dest_tabs_magic_smso,device_type,devt,DI,dial,dial_phone,dim,dispc,display_clock,display_pc_char,dis_status_line,DK,dl,DL,dl1,dm,dN,do,DO,docr,dot_horz_spacing,dot_vert_spacing,down_half_line,ds,dsl,dT,dv,eA,eat_newline_glitch,ec,ech,ed,ehhlm,ei,el,el1,elhlm,elohlm,ena_acs,enacs,endbi,end_bit_image_region,enter_alt_charset_mode,enter_am_mode,enter_blink_mode,enter_bold_mode,enter_ca_mode,enter_delete_mode,enter_dim_mode,enter_doublewide_mode,enter_draft_quality,enter_horizontal_hl_mode,enter_insert_mode,enter_italics_mode,enter_left_hl_mode,enter_leftward_mode,enter_low_hl_mode,enter_micro_mode,enter_near_letter_quality,enter_normal_quality,enter_pc_charset_mode,enter_protected_mode,enter_reverse_mode,enter_right_hl_mode,enter_scancode_mode,enter_secure_mode,enter_shadow_mode,enter_standout_mode,enter_subscript_mode,enter_superscript_mode,enter_top_hl_mode,enter_underline_mode,enter_upward_mode,enter_vertical_hl_mode,enter_xon_mode,eo,erase_chars,erase_overstrike,erhlm,es,eslok,ethlm,evhlm,exit_alt_charset_mode,exit_am_mode,exit_attribute_mode,exit_ca_mode,exit_delete_mode,exit_doublewide_mode,exit_insert_mode,exit_italics_mode,exit_leftward_mode,exit_micro_mode,exit_pc_charset_mode,exit_scancode_mode,exit_shadow_mode,exit_standout_mode,exit_subscript_mode,exit_superscript_mode,exit_underline_mode,exit_upward_mode,exit_xon_mode,F1,F2,F3,F4,F5,F6,F7,F8,F9,Fa,FA,Fb,FB,Fc,FC,Fd,FD,Fe,FE,ff,Ff,FF,Fg,FG,fh,Fh,FH,Fi,FI,fixed_pause,Fj,FJ,Fk,FK,Fl,FL,flash,flash_hook,flash_screen,fln,Fm,FM,Fn,FN,Fo,FO,font0,font1,font2,font3,font4,font5,font6,font7,form_feed,Fp,FP,Fq,FQ,Fr,FR,from_status_line,fs,FS,fsl,FT,FU,FV,FW,FX,FY,FZ,G1,G2,G3,G4,GC,GD,generic_type,getm,get_mouse,GH,GL,Gm,gn,gnu_has_meta_key,goto_window,GR,GU,GV,hangup,hard_copy,hard_cursor,has_hardware_tabs,has_meta_key,has_print_wheel,has_status_line,hc,HC,hd,hl,hls,ho,home,hook,horizontal_tab_delay,hpa,hs,ht,hts,hu,HU,hue_lightness_saturation,hup,hz,i1,i2,i3,ic,Ic,IC,ich,ich1,if,il,il1,im,in,ind,indn,init_1string,init_2string,init_3string,initc,init_file,initialize_color,initial ize_pair,initp,init_prog,init_tabs,insert_character,insert_line,insert_null_glitch,insert_padding,invis,ip,iP,Ip,iprog,is,is1,is2,is3,it,k;,k0,k1,K1,k2,K2,k3,K3,k4,K4,k5,K5,k6,k7,k8,k9,ka,kA,ka1,ka3,kact,kb,kB,kb2,kbeg,kBEG,kbs,kbtab,kC,kc1,kc3,kcan,kCAN,kcbt,kclo,kclr,kcmd,kCMD,kcpn,kcpy,kCPY,kcrt,kCRT,kctab,kcub1,kcud1,kcuf1,kcuu1,kd,kD,kDC,kdch1,kDL,kdl1,kdo,ke,kE,ked,kel,kend,kEND,kent,kEOL,kext,kEXT,key_a1,key_a3,key_b2,key_backspace,key_beg,key_btab,key_c1,key_c3,key_cancel,key_catab,key_clear,key_close,key_command,key_copy,key_create,key_ctab,key_dc,key_dl,key_down,key_eic,key_end,key_enter,key_eol,key_eos,key_exit,key_f0,key_f1,key_f10,key_f11,key_f12,key_f13,key_f14,key_f15,key_f16,key_f17,key_f18,key_f19,key_f2,key_f20,key_f21,key_f22,key_f23,key_f24,key_f25,key_f26,key_f27,key_f28,key_f29,key_f3,key_f30,key_f31,key_f32,key_f33,key_f34,key_f35,key_f36,key_f37,key_f38,key_f39,key_f4,key_f40,key_f41,key_f42,key_f43,key_f44,key_f45,key_f46,key_f47,key_f48,key_f49,key_f5,key_f50,key_f51,key_f52,key_f53,key_f54,key_f55,key_f56,key_f57,key_f58,key_f59,key_f6,key_f60,key_f61,key_f62,key_f63,key_f7,key_f8,key_f9,key_find,key_help,key_home,key_ic,key_il,key_left,key_ll,key_mark,key_message,key_mouse,key_move,key_next,key_npage,key_open,key_options,keypad_local,keypad_xmit,key_ppage,key_previous,key_print,key_redo,key_reference,key_refresh,key_replace,key_restart,key_resume,key_right,key_save,key_sbeg,key_scancel,key_scommand,key_scopy,key_screate,key_sdc,key_sdl,key_select,key_send,key_seol,key_sexit,key_sf,key_sfind,key_shelp,key_shome,key_sic,key_sleft,key_smessage,key_smove,key_snext,key_soptions,key_sprevious,key_sprint,key_sr,key_sredo,key_sreplace,key_sright,key_srsume,key_ssave,key_ssuspend,key_stab,key_sundo,key_suspend,key_undo,key_up,kF,kf0,kf1,kf10,kf11,kf12,kf13,kf14,kf15,kf16,kf17,kf18,kf19,kf2,kf20,kf21,kf22,kf23,kf24,kf25,kf26,kf27,kf28,kf29,kf3,kf30,kf31,kf32,kf33,kf34,kf35,kf36,kf37,kf38,kf39,kf4,kf40,kf41,kf42,kf43,kf44,kf45,kf46,kf47,kf48,kf49,kf5,kf50,kf51,kf52,kf53,kf54,kf55,kf56,kf57,kf58,kf59,kf6,kf60,kf61,kf62,kf63,kf7,kf8,kf9,kfnd,kFND,kh,kH,khlp,kHLP,kHOM,khome,khts,kI,kIC,kich1,kil1,kind,kl,kL,kLFT,kll,km,kM,Km,kmous,kmov,kMOV,kmpf1,kmpf2,kmpf3,kmpf4,kmpf5,kmpf6,kmpf7,kmpf8,kmpf9,kmpt1,kmpt2,kmpt3,kmpt4,kmpt5,kmpt6,kmpt7,kmpt8,kmpt9,kmrk,kmsg,kMSG,kn,kN,knl,knp,knpn,knxt,kNXT,ko,kopn,kopt,kOPT,kP,kpp,kppn,kprt,kPRT,kprv,kPRV,kquit,kr,kR,krdo,kRDO,kref,kres,kRES,krfr,kri,kRIT,krmir,krpl,kRPL,krst,ks,kS,ksav,kSAV,kscl,kscr,ksel,ksf1,ksf10,ksf2,ksf3,ksf4,ksf5,ksf6,ksf7,ksf8,ksf9,kslt,kspd,kSPD,kt,kT,ktab,ktbc,ku,kund,kUND,l0,l1,l2,l3,l4,l5,l6,l7,l8,l9,la,label_format,label_height,label_off,label_on,label_width,lab_f0,lab_f1,lab_f10,lab_f2,lab_f3,lab_f4,lab_f5,lab_f6,lab_f7,lab_f8,lab_f9,le,LE,Lf,LF,lf0,lf1,lf10,lf2,lf3,lf4,lf5,lf6,lf7,lf8,lf9,lh,li,linefeed_if_not_lf,linefeed_is_newline,lines,lines_of_memory,ll,lm,LO,lpi,lpi_changes_res,lpix,lvert,lw,ma,maddr,magic_cookie_glitch,magic_cookie_glitch_ul,max_attributes,max_colors,maximum_windows,max_micro_address,max_micro_jump,max_pairs,mb,MC,mc0,mc4,mc5,mc5i,mc5p,mcs,mcub,mcub1,mcud,mcud1,mcuf,mcuf1,mcuu,mcuu1,md,me,meml,memory_above,memory_below,memory_lock,memory_unlock,memu,meta_off,meta_on,mgc,mh,mhpa,mi,Mi,micro_col_size,micro_column_address,micro_down,micro_left,micro_line_size,micro_right,micro_row_address,micro_up,minfo,mir,mjump,mk,ml,ML,mls,mm,mo,mouse_info,move_insert_mode,move_standout_mode,mp,mr,MR,mrcup,ms,msgr,MT,mu,mvpa,MW,nc,NC,ncv,nd,ND,ndscr,needs_xon_xoff,nel,newline,new_line_delay,nl,Nl,NL,nlab,no_color_video,no_correctly_working_cr,no_esc_ctlc,non_dest_scroll_region,non_rev_rmcup,no_pad_char,NP,npc,npins,NR,nrrmc,ns,number_of_function_keys,number_of_pins,num_labels,nw,nx,nxon,oc,op,orc,order_of_pins,orhi,orig_colors,orig_pair,orl,orvi,os,OTbc,OTbs,OTdB,OTdC,OTdN,OTdT,OTG1,OTG2,OTG3,OTG4,OTGC,OTGD,OTGH,OTGL,OTGR,OTGU,OTGV,other_non_function_keys,OTi2,OTkn,OTko,OTma,OTMT,OTnc,OTnl,OTNL,OTns,OTpt,OTrs,OTug,OTxr,output_res_char,output_res_horz_inch,output_res_line,output_res_vert_inch,over_strike,pa,PA,pad,pad_char,padding_baud_rate,pairs,parm_dch,parm_delete_line,parm_down_cursor,parm_down_micro,parm_ich,parm_index,parm_insert_line,parm_left_cursor,parm_left_micro,parm_right_cursor,parm_right_micro,parm_rindex,parm_up_cursor,parm_up_micro,pause,pb,pc,pc_term_options,pctrm,pf,pfkey,pfloc,pfx,pfxl,pk,pkey_key,pkey_local,pkey_plab,pkey_xmit,pl,plab_norm,pln,pn,po,pO,porder,print_rate,print_screen,prot,prtr_non,prtr_off,prtr_on,prtr_silent,ps,pt,PU,pulse,px,QD,qdial,quick_dial,r1,r2,r3,RA,rbim,rc,RC,rcsd,remove_clock,rep,repeat_char,req_for_input,req_mouse_pos,reqmp,reset_1string,reset_2string,reset_3string,reset_file,restore_cursor,return_does_clr_eol,rev,rf,RF,rfi,ri,RI,rin,ritm,rlm,rmacs,rmam,rmclk,rmcup,rmdc,rmicm,rmir,rmkx,rmln,rmm,rmp,rmpch,rmsc,rmso,rmul,rmxon,row_address,row_addr_glitch,rp,rP,RQ,rs,rs1,rs2,rs3,rshm,rsubm,rsupm,rum,rvert,rwidm,RX,s0,s0ds,s1,S1,s1ds,s2,S2,s2ds,s3,S3,s3ds,S4,S5,S6,S7,S8,sa,sA,SA,sam,save_cursor,Sb,sbim,sc,SC,scancode_escape,scesa,scesc,sclk,scp,scroll_forward,scroll_reverse,scs,scsd,sdrfq,se,select_char_set,semi_auto_right_margin,set0_des_seq,set1_des_seq,set2_des_seq,set3_des_seq,set_a_attributes,setab,set_a_background,setaf,set_a_foreground,set_attributes,setb,set_background,set_bottom_margin,set_bottom_margin_parm,set_clock,setcolor,set_color_band,set_color_pair,setf,set_foreground,set_left_margin,set_left_margin_parm,set_lr_margin,set_page_length,set_pglen_inch,set_right_margin,set_right_margin_parm,set_tab,set_tb_margin,set_top_margin,set_top_margin_parm,set_window,sf,Sf,SF,sg,sgr,sgr0,sgr1,sitm,sL,slength,slines,slm,smacs,smam,smcup,smdc,smgb,smgbp,smgl,smglp,smglr,smgr,smgrp,smgt,smgtb,smgtp,smicm,smir,smkx,smln,smm,smpch,smsc,smso,smul,smxon,snlq,snrmq,so,sp,spinh,spinv,sr,SR,sshm,ssubm,ssupm,st,start_bit_image,start_char_set_def,status_line_esc_ok,stop_bit_image,stop_char_set_def,subcs,subscript_characters,sum,supcs,superscript_characters,swidm,SX,ta,tab,tbc,te,termcap_init2,termcap_reset,these_cause_cr,ti,tilde_glitch,TO,tone,topl,to_status_line,transparent_underline,ts,tsl,u0,u1,u2,u3,u4,u5,u6,u7,u8,u9,uc,ue,ug,ul,underline_char,up,UP,up_half_line,us,user0,user1,user2,user3,user4,user5,user6,user7,user8,user9,ut,vb,ve,vi,virtual_terminal,vpa,vs,vt,WA,wait,wait_tone,WG,wi,widcs,wide_char_size,width_status_line,wind,wingo,wnum,ws,wsl,xb,xenl,XF,Xh,xhp,xhpa,xl,Xl,xmc,xn,XN,xo,Xo,xoffc,xoff_character,xon,xonc,xon_character,xon_xoff,xr,Xr,xs,xsb,xt,Xt,Xv,xvpa,Xy,Ya,YA,Yb,YB,Yc,YC,Yd,YD,Ye,YE,Yf,YF,Yg,YG,Yh,Yi,Yj,Yk,Yl,Ym,Yn,Yo,Yp,Yv,Yw,Yx,Yy,Yz,YZ,Za,ZA,Zb,ZB,Zc,ZC,Zd,ZD,Ze,ZE,zerom,zero_motion,Zf,ZF,Zg,ZG,Zh,ZH,Zi,ZI,Zj,ZJ,Zk,ZK,Zl,ZL,Zm,ZM,Zn,ZN,Zo,ZO,Zp,ZP,Zq,ZQ,Zr,ZR,Zs,ZS,Zt,ZT,Zu,ZU,Zv,ZV,Zw,ZW,Zx,ZX,Zy,ZY,Zz,ZZ

TERMINAL TYPE DESCRIPTIONS SOURCE FILE

ANSI, UNIX CONSOLE, AND SPECIAL TYPES
Specials
ANSI.SYS/ISO 6429/ECMA-48 Capabilities
ANSI/ECMA-48 terminals and terminal emulators
DOS ANSI.SYS variants
ANSI console types
BeOS
Linux consoles
Mach
OSF Unix
QNX
NetBSD consoles
FreeBSD console entries
386BSD and BSD/OS Consoles
DEC VT52
DEC VT100 and compatibles
VT100 emulations
X terminal emulators
MGR
UNIX VIRTUAL TERMINALS, VIRTUAL CONSOLES, AND TELNET CLIENTS
Pilot Pro Palm-Top
COMMERCIAL WORKSTATION CONSOLES
Alpha consoles
Sun consoles
Iris consoles
NeWS consoles
NeXT consoles
Sony NEWS workstations
Common Desktop Environment
Non-Unix Consoles
COMMON TERMINAL TYPES
Altos
Hewlett-Packard (hp)
Honeywell-Bull
Lear-Siegler (adm)
Prime
Qume (qvt)
Televideo (tvi)
Visual (vi)
Wyse (wy)
Kermit terminal emulations
NON-ANSI TERMINAL EMULATIONS
Avatar
RBcomm
LCD DISPLAYS
Matrix Orbital
OLDER TERMINAL TYPES
AT&T (att, tty)
Ampex (Dialogue)
Ann Arbor (aa)
Applied Digital Data Systems (adds)
C. Itoh Electronics
Control Data (cdc)
Getronics
Human Designed Systems (Concept)
Contel Business Systems.
Data General (dg)
Datamedia (dm)
Falco

Just something sorta cool for you to check out

{
 local t=`tputm 'clear' 'setaf 75'` l a b f=/tmp/ps IFS=' ';
exec 6<>$f;ps L|tr -s ' ' &>$f;
while read -u6 l;do a=${l/% */} b=${l/* /};
figlet -rtw $((${COLUMNS} /2 )) -f big "$l";
tput sgr0;
command ps wwo pid:6,user:8,vsize:8,comm:20,$a:50 k -$a -A;cont;
done;
};

Terminal Escape Code Zen originally appeared on AskApache.com

If you want to learn best tricks and tips, there's only one way to do it... at least only one way that I know of. Here are some notes I created while learning about the intricacies of php sessions, it's all in the code.

[Session]
; Handler used to store/retrieve data.
session.save_handler = files

Argument passed to save_handler. In the case of files, this is the path where data files are stored. As of PHP 4.0.1, you can define the path as:

session.save_path = "N;/path"

where N is an integer. Instead of storing all the session files in /path, what this will do is use subdirectories N-levels deep, and store the session data in those directories. This is useful if you or your OS have problems with lots of files in one directory, and is a more efficient layout for servers that handle lots of sessions.

;
; NOTE 1: PHP will not create this directory structure automatically.
;         You can use the script in the ext/session dir for that purpose.
; NOTE 2: See the section on garbage collection below if you choose to
;         use subdirectories for session storage
;
; The file storage module creates files using mode 600 by default.
; You can change that by using
;
;     session.save_path = "N;MODE;/path"
;
; where MODE is the octal representation of the mode. Note that this
; does not overwrite the process's umask.
;session.save_path = "/tmp"

session.c

/* {{{ PHP_INI
 */
PHP_INI_BEGIN()
        STD_PHP_INI_BOOLEAN("session.bug_compat_42",    "1",         PHP_INI_ALL, OnUpdateBool,   bug_compat,         php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.bug_compat_warn",  "1",         PHP_INI_ALL, OnUpdateBool,   bug_compat_warn,    php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.save_path",          "",          PHP_INI_ALL, OnUpdateSaveDir,save_path,          php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.name",               "PHPSESSID", PHP_INI_ALL, OnUpdateString, session_name,       php_ps_globals,    ps_globals)
        PHP_INI_ENTRY("session.save_handler",           "files",     PHP_INI_ALL, OnUpdateSaveHandler)
        STD_PHP_INI_BOOLEAN("session.auto_start",       "0",         PHP_INI_ALL, OnUpdateBool,   auto_start,         php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.gc_probability",     "1",         PHP_INI_ALL, OnUpdateLong,    gc_probability,     php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.gc_divisor",         "100",       PHP_INI_ALL, OnUpdateLong,    gc_divisor,        php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.gc_maxlifetime",     "1440",      PHP_INI_ALL, OnUpdateLong,    gc_maxlifetime,     php_ps_globals,    ps_globals)
        PHP_INI_ENTRY("session.serialize_handler",      "php",       PHP_INI_ALL, OnUpdateSerializer)
        STD_PHP_INI_ENTRY("session.cookie_lifetime",    "0",         PHP_INI_ALL, OnUpdateLong,    cookie_lifetime,    php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cookie_path",        "/",         PHP_INI_ALL, OnUpdateString, cookie_path,        php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cookie_domain",      "",          PHP_INI_ALL, OnUpdateString, cookie_domain,      php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.cookie_secure",    "",          PHP_INI_ALL, OnUpdateBool,   cookie_secure,      php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.cookie_httponly",  "",          PHP_INI_ALL, OnUpdateBool,   cookie_httponly,    php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.use_cookies",      "1",         PHP_INI_ALL, OnUpdateBool,   use_cookies,        php_ps_globals,    ps_globals)
        STD_PHP_INI_BOOLEAN("session.use_only_cookies", "0",         PHP_INI_ALL, OnUpdateBool,   use_only_cookies,   php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.referer_check",      "",          PHP_INI_ALL, OnUpdateString, extern_referer_chk, php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.entropy_file",       "",          PHP_INI_ALL, OnUpdateString, entropy_file,       php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.entropy_length",     "0",         PHP_INI_ALL, OnUpdateLong,    entropy_length,     php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cache_limiter",      "nocache",   PHP_INI_ALL, OnUpdateString, cache_limiter,      php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.cache_expire",       "180",       PHP_INI_ALL, OnUpdateLong,    cache_expire,       php_ps_globals,    ps_globals)
        PHP_INI_ENTRY("session.use_trans_sid",          "0",         PHP_INI_ALL, OnUpdateTransSid)
        STD_PHP_INI_ENTRY("session.hash_function",      "0",         PHP_INI_ALL, OnUpdateLong,    hash_func,          php_ps_globals,    ps_globals)
        STD_PHP_INI_ENTRY("session.hash_bits_per_character",      "4",         PHP_INI_ALL, OnUpdateLong,    hash_bits_per_character,          php_ps_globals,    ps_globals)
 
        /* Commented out until future discussion */
        /* PHP_INI_ENTRY("session.encode_sources", "globals,track", PHP_INI_ALL, NULL) */
PHP_INI_END()
/* }}} */

Session Errors

The session id contains illegal characters, valid characters are a-z, A-Z, 0-9 and '-,'
fcntl(%d, F_SETFD, FD_CLOEXEC) failed: %s (%d)
open(%s, O_RDWR) failed: %s (%d)
ps_files_cleanup_dir: opendir(%s) failed: %s (%d)
read failed: %s (%d)
read returned less bytes than requested
write failed: %s (%d)
write wrote less bytes than requested
mm_malloc failed, avail %d, err %s
cannot allocate new data segment
Skipping numeric key %ld.
A session is active. You cannot change the session module's ini settings at this time.
Cannot find save handler %s
Cannot find serialization handler %s
Unknown session.serialize_handler. Failed to encode session object.
Cannot encode non-existent session.
Unknown session.serialize_handler. Failed to decode session object.
Failed to decode session object. Session has been destroyed.
Invalid session hash function
The ini setting hash_bits_per_character is out of range (should be 4, 5, or 6) - using 4 for now
No storage module chosen - failed to initialize session.
Failed to initialize storage module: %s (path: %s)
The session bug compatibility code will not
Your script possibly relies on a session side-effect which existed until PHP 4.2.3. Please be advised that the session extension does not consider global variables as a source of data, unless register_globals is enabled. You can disable this functionality and this warning by setting session.bug_compat_42 or session.bug_compat_warn to off, respectively.
Failed to write session data (%s). Please
Cannot send session cache limiter - headers already sent (output started at %s:%d)
Cannot send session cache limiter - headers already sent
Cannot send session cookie - headers already sent by (output started at %s:%d)
Cannot send session cookie - headers already sent
Cannot find save handler %s
Cannot find unknown save handler
purged %d expired session objects
Trying to destroy uninitialized session
Session object destruction failed
Cannot find named PHP session module (%s)
Argument %d is not a valid callback
Cannot regenerate session id - headers already sent
Session object destruction failed

PS_GC_FUNC(files)
{
        PS_FILES_DATA;
 
        /* we don't perform any cleanup, if dirdepth is larger than 0.
           we return SUCCESS, since all cleanup should be handled by
           an external entity (i.e. find -ctime x | xargs rm) */
 
        if (data->dirdepth == 0) {
                *nrdels = ps_files_cleanup_dir(data->basedir, maxlifetime TSRMLS_CC);
        }
 
        return SUCCESS;
}

mod_files.c

/* If you change the logic here, please also update the error message in
 * ps_files_open() appropriately */
static int ps_files_valid_key(const char *key)
{
        size_t len;
        const char *p;
        char c;
        int ret = 1;
 
        for (p = key; (c = *p); p++) {
                /* valid characters are a..z,A..Z,0..9 */
                if (!((c >= 'a' && c <= 'z')
                                || (c >= 'A' && c <= 'Z')
                                || (c >= '0' && c <= '9')
                                || c == ','
                                || c == '-')) {
                        ret = 0;
                        break;
                }
        }
 
        len = p - key;
 
        if (len == 0) {
                ret = 0;
        }
 
        return ret;
}

static int ps_files_cleanup_dir(const char *dirname, int maxlifetime TSRMLS_DC)
{
        DIR *dir;
        char dentry[sizeof(struct dirent) + MAXPATHLEN];
        struct dirent *entry = (struct dirent *) &dentry;
        struct stat sbuf;
        char buf[MAXPATHLEN];
        time_t now;
        int nrdels = 0;
        size_t dirname_len;
 
        dir = opendir(dirname);
        if (!dir) {
                php_error_docref(NULL TSRMLS_CC, E_NOTICE, "ps_files_cleanup_dir: opendir(%s) failed: %s (%d)", dirname, strerror(errno), errno);
                return (0);
        }
 
        time(&now);
 
        dirname_len = strlen(dirname);
 
        /* Prepare buffer (dirname never changes) */
        memcpy(buf, dirname, dirname_len);
        buf[dirname_len] = PHP_DIR_SEPARATOR;
 
        while (php_readdir_r(dir, (struct dirent *) dentry, &entry) == 0 && entry) {
                /* does the file start with our prefix? */
                if (!strncmp(entry->d_name, FILE_PREFIX, sizeof(FILE_PREFIX) - 1)) {
                        size_t entry_len = strlen(entry->d_name);
 
                        /* does it fit into our buffer? */
                        if (entry_len + dirname_len + 2 < MAXPATHLEN) {
                                /* create the full path.. */
                                memcpy(buf + dirname_len + 1, entry->d_name, entry_len);
 
                                /* NUL terminate it and */
                                buf[dirname_len + entry_len + 1] = '\0';
 
                                /* check whether its last access was more than maxlifet ago */
                                if (VCWD_STAT(buf, &sbuf) == 0 &&
#ifdef NETWARE
                                                (now - sbuf.st_mtime.tv_sec) > maxlifetime) {
#else
                                                (now - sbuf.st_mtime) > maxlifetime) {
#endif
                                        VCWD_UNLINK(buf);
                                        nrdels++;
                                }
                        }
                }
        }
 
        closedir(dir);
 
        return (nrdels);
}

ext/session/mod_files.c

#define PS_FILES_DATA ps_files *data = PS_GET_MOD_DATA()
 
PS_OPEN_FUNC(files)
{
        ps_files *data;
        const char *p, *last;
        const char *argv[3];
        int argc = 0;
        size_t dirdepth = 0;
        int filemode = 0600;
 
        if (*save_path == '\0') {
                /* if save path is an empty string, determine the temporary dir */
                save_path = php_get_temporary_directory();
 
                if (strcmp(save_path, "/tmp")) {
                        if (PG(safe_mode) && (!php_checkuid(save_path, NULL, CHECKUID_CHECK_FILE_AND_DIR))) {
                                return FAILURE;
                        }
                        if (php_check_open_basedir(save_path TSRMLS_CC)) {
                                return FAILURE;
                        }
                }
        }
 
        /* split up input parameter */
        last = save_path;
        p = strchr(save_path, ';');
        while (p) {
                argv[argc++] = last;
                last = ++p;
                p = strchr(p, ';');
                if (argc > 2) break;
        }
        argv[argc++] = last;
 
        if (argc > 1) {
                errno = 0;
                dirdepth = (size_t) strtol(argv[0], NULL, 10);
                if (errno == ERANGE) {
                        php_error(E_WARNING, "The first parameter in session.save_path is invalid");
                        return FAILURE;
                }
        }
 
        if (argc > 2) {
                errno = 0;
                filemode = strtol(argv[1], NULL, 8);
                if (errno == ERANGE || filemode < 0 || filemode > 07777) {
                        php_error(E_WARNING, "The second parameter in session.save_path is invalid");
                        return FAILURE;
                }
        }
        save_path = argv[argc - 1];
 
        data = emalloc(sizeof(*data));
        memset(data, 0, sizeof(*data));
 
        data->fd = -1;
        data->dirdepth = dirdepth;
        data->filemode = filemode;
        data->basedir_len = strlen(save_path);
        data->basedir = estrndup(save_path, data->basedir_len);
 
        PS_SET_MOD_DATA(data);
 
        return SUCCESS;
}

[PHP 5.2.0 session.save_path safe_mode and open_basedir bypass]
 
Author: Maksymilian Arciemowicz (SecurityReason)
Date:
- - Written: 02.10.2006
- - Public: 08.12.2006
SecurityAlert Id: 43
CVE: CVE-2006-6383
SecurityRisk: High
Affected Software: PHP 5.2.0
Advisory URL: http://securityreason.com/achievement_securityalert/43
Vendor: http://www.php.net
 
- --- 0.Description ---
PHP is an HTML-embedded scripting language. Much of its syntax is borrowed from
C, Java and Perl with a couple of unique PHP-specific features thrown in. The
goal of the language is to allow web developers to write dynamically generated
pages quickly.
 
A nice introduction to PHP by Stig Sather Bakken can be found at
http://www.zend.com/zend/art/intro.php on the Zend website. Also, much  of the
PHP Conference Material is freely available.
 
Session support in PHP consists of a way to preserve certain data across
subsequent accesses. This enables you to build more customized applications and
increase the appeal of your web site.
 
A visitor accessing your web site is assigned a unique id, the so-called
session id. This is either stored in a cookie on the user side or is propagated
in the URL.
 
session.save_path defines the argument which is passed to the save handler. If
you choose the default files handler, this is the path where the files are
created. Defaults to /tmp. See also session_save_path().
 
There is an optional N argument to this directive that determines the number of
directory levels your session files will be spread around in. For example,
setting to '5;/tmp' may end up creating a session file and location like
/tmp/4/b/1/e/3/sess_4b1e384ad74619bd212e236e52a5a174If . In order to use N you
must create all of these directories before use. A small shell script exists in
ext/session to do this, it's called mod_files.sh. Also note that if N is used
and greater than 0 then automatic garbage collection will not be performed, see
a copy of php.ini for further information. Also, if you use N, be sure to
surround session.save_path in "quotes" because the separator (;) is also used
for comments in php.ini.
 
- --- 1. session.save_path safe mode and open basedir bypass ---
session.save_path can be set in ini_set(), session_save_path() function. In
session.save_path there must be path where you will save yours tmp file. But
syntax for session.save_path can be:
 
[/PATH]
 
OR
 
[N;/PATH]
 
N - can be a string.
 
EXAMPLES:
 
1. session_save_path("/DIR/WHERE/YOU/HAVE/ACCESS")
2. session_save_path("5;/DIR/WHERE/YOU/HAVE/ACCESS")
 
and
 
3.
session_save_path("/DIR/WHERE/YOU/DONT/HAVE/ACCESS\0;/DIR/WHERE/YOU/HAVE/ACCESS")

CACHE_LIMITER_FUNC(public)
{
        char buf[MAX_STR + 1];
        struct timeval tv;
        time_t now;
 
        gettimeofday(&tv, NULL);
        now = tv.tv_sec + PS(cache_expire) * 60;
#define EXPIRES "Expires: "
        memcpy(buf, EXPIRES, sizeof(EXPIRES) - 1);
        strcpy_gmt(buf + sizeof(EXPIRES) - 1, &now);
        ADD_HEADER(buf);
 
        snprintf(buf, sizeof(buf) , "Cache-Control: public, max-age=%ld", PS(cache_expire) * 60); /* SAFE */
        ADD_HEADER(buf);
 
        last_modified(TSRMLS_C);
}
 
CACHE_LIMITER_FUNC(private_no_expire)
{
        char buf[MAX_STR + 1];
 
        snprintf(buf, sizeof(buf), "Cache-Control: private, max-age=%ld, pre-check=%ld", PS(cache_expire) * 60, PS(cache_expire) * 60); /* SAFE */
        ADD_HEADER(buf);
 
        last_modified(TSRMLS_C);
}
 
CACHE_LIMITER_FUNC(private)
{
        ADD_HEADER("Expires: Thu, 19 Nov 1981 08:52:00 GMT");
        CACHE_LIMITER(private_no_expire)(TSRMLS_C);
}
 
CACHE_LIMITER_FUNC(nocache)
{
        ADD_HEADER("Expires: Thu, 19 Nov 1981 08:52:00 GMT");
        /* For HTTP/1.1 conforming clients and the rest (MSIE 5) */
        ADD_HEADER("Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0");
        /* For HTTP/1.0 conforming clients */
        ADD_HEADER("Pragma: no-cache");
}
 
static php_session_cache_limiter_t php_session_cache_limiters[] = {
        CACHE_LIMITER_ENTRY(public)
        CACHE_LIMITER_ENTRY(private)
        CACHE_LIMITER_ENTRY(private_no_expire)
        CACHE_LIMITER_ENTRY(nocache)
        {0}
};
 
static int php_session_cache_limiter(TSRMLS_D)
{
        php_session_cache_limiter_t *lim;
 
        if (PS(cache_limiter)[0] == '\0') return 0;
 
        if (SG(headers_sent)) {
                char *output_start_filename = php_get_output_start_filename(TSRMLS_C);
                int output_start_lineno = php_get_output_start_lineno(TSRMLS_C);
 
                if (output_start_filename) {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cache limiter - headers already sent (output started at %s:%d)",
                                output_start_filename, output_start_lineno);
                } else {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cache limiter - headers already sent");
                }
                return -2;
        }
 
        for (lim = php_session_cache_limiters; lim->name; lim++) {
                if (!strcasecmp(lim->name, PS(cache_limiter))) {
                        lim->func(TSRMLS_C);
                        return 0;
                }
        }
 
        return -1;
}

static void php_session_send_cookie(TSRMLS_D)
{
        smart_str ncookie = {0};
        char *date_fmt = NULL;
        char *e_session_name, *e_id;
 
        if (SG(headers_sent)) {
                char *output_start_filename = php_get_output_start_filename(TSRMLS_C);
                int output_start_lineno = php_get_output_start_lineno(TSRMLS_C);
 
                if (output_start_filename) {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cookie - headers already sent by (output started at %s:%d)",
                                output_start_filename, output_start_lineno);
                } else {
                        php_error_docref(NULL TSRMLS_CC, E_WARNING, "Cannot send session cookie - headers already sent");
                }
                return;
        }
 
        /* URL encode session_name and id because they might be user supplied */
        e_session_name = php_url_encode(PS(session_name), strlen(PS(session_name)), NULL);
        e_id = php_url_encode(PS(id), strlen(PS(id)), NULL);
 
        smart_str_appends(&ncookie, COOKIE_SET_COOKIE);
        smart_str_appends(&ncookie, e_session_name);
        smart_str_appendc(&ncookie, '=');
        smart_str_appends(&ncookie, e_id);
 
        efree(e_session_name);
        efree(e_id);
 
        if (PS(cookie_lifetime) > 0) {
                struct timeval tv;
                time_t t;
 
                gettimeofday(&tv, NULL);
                t = tv.tv_sec + PS(cookie_lifetime);
 
                if (t > 0) {
                        date_fmt = php_std_date(t TSRMLS_CC);
                        smart_str_appends(&ncookie, COOKIE_EXPIRES);
                        smart_str_appends(&ncookie, date_fmt);
                        efree(date_fmt);
                }
        }
 
        if (PS(cookie_path)[0]) {
                smart_str_appends(&ncookie, COOKIE_PATH);
                smart_str_appends(&ncookie, PS(cookie_path));
        }
 
        if (PS(cookie_domain)[0]) {
                smart_str_appends(&ncookie, COOKIE_DOMAIN);
                smart_str_appends(&ncookie, PS(cookie_domain));
        }
 
        if (PS(cookie_secure)) {
                smart_str_appends(&ncookie, COOKIE_SECURE);
        }
 
        if (PS(cookie_httponly)) {
                smart_str_appends(&ncookie, COOKIE_HTTPONLY);
        }
 
        smart_str_0(&ncookie);
 
        /*      'replace' must be 0 here, else a previous Set-Cookie
                header, probably sent with setcookie() will be replaced! */
        sapi_add_header_ex(ncookie.c, ncookie.len, 0, 0 TSRMLS_CC);
}

PHP Session File Hacks originally appeared on AskApache.com

I was going through some backups from years ago, and ran:

locate .htaccess | xargs -I'{}' cat '{}' >> master-htaccesser.txt

My site is named after reading source code because that is what helps me the most when I'm trying to learn something unusually difficult. Just like functions and aliases, it is very helpful to have cheatsheets for common commands.. not much is better than real-world examples. Unfortunately because this was compiled from hundreds of htaccess files on multiple hosts and platforms, and due to the concatenation, it's not organized.

Normally I would not publish something like this, who knows how much unreleased tricks I forgot about.. but in order to say thanks to all those working for open-source, the FSF, and to all those who don't steal content, and to all the incredible authors who shared with me (I twitter most of what I find, and follow my favs), here ya go..

ATTN: Please let me know if this is total junk or not, this is around 1/500th of my master-htaccesser.txt file.. and I'd be happy to post more if it helps..

Checking for Cookie

Used this to stop mp3-scrapers.. checks for a cookie ending in MP3P=02357

Options -Indexes
RewriteEngine On
RewriteCond %{HTTP_COOKIE} !^.*MP3P=([0-9]+).* [NC]
RewriteRule .* - [F,L]

Setting Environment Var if Proxied

RewriteEngine On
RewriteRule "\.(gif|png|jpg)$" "-" [ENV=proxied_image:1]
RewriteCond "%{ENV:proxied_image}" "!1"
RewriteRule "^" "-" [ENV=proxied_other:1]

nokeepalive for ErrorDocs and Abusers

Options SymLinksIfOwnerMatch IncludesNOEXEC
AddOutputFilter Includes html
SetEnv nokeepalive
Order Allow,Deny
Allow from all
# 1 YEAR
<filesMatch "\.(js|css)$">
Header unset Pragma
FileETag None
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
Header unset Last-Modified
Header unset ETag
 
SetOutputFilter DEFLATE
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</filesMatch>
SecFilterEngine Off
 
# 1 YEAR
<filesMatch "\.(js|css)$">
Header unset Pragma
FileETag None
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
Header unset Last-Modified
Header unset ETag
 
SetOutputFilter DEFLATE
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</filesMatch>

Unreleased Tests for AskApache Password Protection

# +ASKAPACHE PASSPRO 4.6.6
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +APRO SIDS
# +SID 21030002
Order Deny,Allow
Deny from All
Satisfy Any
AuthType Digest
AuthName "Protected By AskApache"
AuthDigestDomain / http://www.askapache.com/
AuthDigestFile /home/.greer/askapache/sites/askapache.com/.htpasswda3
Require valid-user
<filesMatch "\.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$">
Allow from All
</filesMatch>
<filesMatch "(async-upload|admin-ajax)\.php$">
<ifModule mod_security.c>
SecFilterEngine Off
</ifModule>
Allow from All
</filesMatch>
# -SID 21030002
# -APRO SIDS
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE PASSPRO 4.6.6
 
# +ASKAPACHE PASSPRO 4.6.6
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +APRO SIDS
# +SID Test
ErrorDocument 401 /wp-content/askapache/test.gif
ErrorDocument 403 /wp-content/askapache/test.gif
ErrorDocument 404 /wp-content/askapache/test.gif
ErrorDocument 500 /wp-content/askapache/test.gif
ServerSignature On
<ifModule mod_alias.c>
RedirectMatch 305 ^.*modaliastest$ http://www.askapache.com
</ifModule>
<ifModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{QUERY_STRING} modrewritetest [NC]
RewriteRule .* http://www.askapache.com [R=307,L]
</ifModule>
<files modsec_check.gif>
<ifModule mod_security.c>
SetEnv MODSEC_ENABLE On
SecFilterEngine On
SecFilterDefaultAction "nolog,noauditlog,pass"
SecAuditEngine Off
SecFilterInheritance Off
SecFilter modsecuritytest "deny,nolog,noauditlog,status:503"
Deny from All
</ifModule>
</files>
<files basic_auth_test.gif>
AuthType Basic
AuthName "askapache test"
AuthUserFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-basic
Require valid-user
</files>
<files digest_check.gif>
AuthType Digest
AuthName "askapache test"
AuthDigestDomain /wp-content/askapache/ http://www.askapache.com/wp-content/askapache/
AuthUserFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-digest
Require none
</files>
<files authuserfile_test.gif>
AuthType Digest
AuthName "askapache test"
AuthDigestDomain /wp-content/askapache/ http://www.askapache.com/wp-content/askapache/
AuthUserFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-digest
Require valid-user
</files>
<files authdigestfile_test.gif>
AuthType Digest
AuthName "askapache test"
AuthDigestDomain /wp-content/askapache/ http://www.askapache.com/wp-content/askapache/
AuthDigestFile /home/.greer/askapache/sites/askapache.com/htdocs/wp-content/askapache/.htpasswd-digest
Require valid-user
</files>
# -SID Test
# -APRO SIDS
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE PASSPRO 4.6.6

Warming up to the really advanced tests

Options +ExecCGI
Order Deny,Allow
Deny from All
Allow from 208.113.134.190 64.111.114.111 208.113.134.203 208.113.152.201 env=REDIRECT_STATUS
Satisfy Any
Options +FollowSymLinks
AddHandler application/x-httpd-php .php
 
RewriteEngine On
RewriteBase /
RewriteCond %{QUERY_STRING} !^tyy+$ [NC]
RewriteCond %{REMOTE_USER} ^(.+)$
RewriteRule ^(.*)$ $1/-%1 [R=302,L]
Options +ExecCGI +FollowSymLinks
DirectoryIndex index.php
ErrorDocument 404 /cgi-bin/pro/index.php
 
AuthType Digest
AuthName "AskApache Pro"
AuthDigestFile /home/askapache/sites/askapache.com/.htpasswd-pro
AuthDigestDomain /cgi-bin/pro/ http://www.askapache.com/cgi-bin/pro/ https://www.askapache.com/cgi-bin/pro/
Require user askapacheDirectoryIndex p.php
ErrorDocument 403 /cgi-bin/p/p.php
ErrorDocument 401 /cgi-bin/p/p.php
ErrorDocument 500 /cgi-bin/p/p.php
ErrorDocument 503 /cgi-bin/p/p.php
 
RewriteEngine On
RewriteBase /
RewriteRule .* - [E=INFO_API_VERSION:%{API_VERSION}]
RewriteRule .* - [E=INFO_AUTH_TYPE:%{AUTH_TYPE}]
RewriteRule .* - [E=INFO_CONTENT_LENGTH:%{CONTENT_LENGTH}]
RewriteRule .* - [E=INFO_CONTENT_TYPE:%{CONTENT_TYPE}]
RewriteRule .* - [E=INFO_DOCUMENT_ROOT:%{DOCUMENT_ROOT}]
RewriteRule .* - [E=INFO_GATEWAY_INTERFACE:%{GATEWAY_INTERFACE}]
RewriteRule .* - [E=INFO_HTTPS:%{HTTPS}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT:%{HTTP:Accept}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_LANGUAGE:%{HTTP:Accept-Language}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_ENCODING:%{HTTP:Accept-Encoding}]
RewriteRule .* - [E=INFO_HTTP_ACCEPT_CHARSET:%{HTTP:Accept-Charset}]
RewriteRule .* - [E=INFO_HTTP_CACHE_CONTROL:%{HTTP:Cache-Control}]
RewriteRule .* - [E=INFO_HTTP_CONNECTION:%{HTTP:Connection}]
RewriteRule .* - [E=INFO_HTTP_COOKIE:%{HTTP_COOKIE}]
RewriteRule .* - [E=INFO_HTTP_FORWARDED:%{HTTP_FORWARDED}]
RewriteRule .* - [E=INFO_HTTP_HOST:%{HTTP_HOST}]
RewriteRule .* - [E=INFO_HTTP_KEEP_ALIVE:%{HTTP_KEEP_ALIVE}]
RewriteRule .* - [E=INFO_HTTP_PROXY_CONNECTION:%{HTTP_PROXY_CONNECTION}]
RewriteRule .* - [E=INFO_HTTP_REFERER:%{HTTP:Referer}]
RewriteRule .* - [E=INFO_HTTP_USER_AGENT:%{HTTP_USER_AGENT}]
RewriteRule .* - [E=INFO_IS_SUBREQ:%{IS_SUBREQ}]
RewriteRule .* - [E=INFO_ORIG_PATH_INFO:%{ORIG_PATH_INFO}]
RewriteRule .* - [E=INFO_ORIG_PATH_TRANSLATED:%{ORIG_PATH_TRANSLATED}]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_FILENAME:%{ORIG_SCRIPT_FILENAME}]
RewriteRule .* - [E=INFO_ORIG_SCRIPT_NAME:%{ORIG_SCRIPT_NAME}]
RewriteRule .* - [E=INFO_PATH:%{PATH}]
RewriteRule .* - [E=INFO_PATH_INFO:%{PATH_INFO}]
RewriteRule .* - [E=INFO_QUERY_STRING:%{QUERY_STRING}]
RewriteRule .* - [E=INFO_REDIRECT_QUERY_STRING:%{REDIRECT_QUERY_STRING}]
RewriteRule .* - [E=INFO_REDIRECT_REMOTE_USER:%{REDIRECT_REMOTE_USER}]
RewriteRule .* - [E=INFO_REDIRECT_STATUS:%{REDIRECT_STATUS}]
RewriteRule .* - [E=INFO_REDIRECT_URL:%{REDIRECT_URL}]
RewriteRule .* - [E=INFO_REMOTE_ADDR:%{REMOTE_ADDR}]
RewriteRule .* - [E=INFO_REMOTE_HOST:%{REMOTE_HOST}]
RewriteRule .* - [E=INFO_REMOTE_IDENT:%{REMOTE_IDENT}]
RewriteRule .* - [E=INFO_REMOTE_PORT:%{REMOTE_PORT}]
RewriteRule .* - [E=INFO_REMOTE_USER:%{REMOTE_USER}]
RewriteRule .* - [E=INFO_REQUEST_FILENAME:%{REQUEST_FILENAME}]
RewriteRule .* - [E=INFO_REQUEST_METHOD:%{REQUEST_METHOD}]
RewriteRule .* - [E=INFO_REQUEST_URI:%{REQUEST_URI}]
RewriteRule .* - [E=INFO_REDIRECT_REQUEST_URI:%{REDIRECT_REQUEST_URI}]
RewriteRule .* - [E=INFO_SCRIPT_FILENAME:%{SCRIPT_FILENAME}]
RewriteRule .* - [E=INFO_SCRIPT_GROUP:%{SCRIPT_GROUP}]
RewriteRule .* - [E=INFO_SCRIPT_NAME:%{SCRIPT_NAME}]
RewriteRule .* - [E=INFO_SCRIPT_URI:%{SCRIPT_URI}]
RewriteRule .* - [E=INFO_SCRIPT_URL:%{SCRIPT_URL}]
RewriteRule .* - [E=INFO_SCRIPT_USER:%{SCRIPT_USER}]
RewriteRule .* - [E=INFO_SERVER_ADDR:%{SERVER_ADDR}]
RewriteRule .* - [E=INFO_SERVER_ADMIN:%{SERVER_ADMIN}]
RewriteRule .* - [E=INFO_SERVER_NAME:%{SERVER_NAME}]
RewriteRule .* - [E=INFO_SERVER_PORT:%{SERVER_PORT}]
RewriteRule .* - [E=INFO_SERVER_PROTOCOL:%{SERVER_PROTOCOL}]
RewriteRule .* - [E=INFO_SERVER_SIGNATURE:%{SERVER_SIGNATURE}]
RewriteRule .* - [E=INFO_SERVER_SOFTWARE:%{SERVER_SOFTWARE}]
RewriteRule .* - [E=INFO_THE_REQUEST:%{THE_REQUEST}]
RewriteRule .* - [E=INFO_TIME:%{TIME}]
RewriteRule .* - [E=INFO_TIME_DAY:%{TIME_DAY}]
RewriteRule .* - [E=INFO_TIME_HOUR:%{TIME_HOUR}]
RewriteRule .* - [E=INFO_TIME_MIN:%{TIME_MIN}]
RewriteRule .* - [E=INFO_TIME_MON:%{TIME_MON}]
RewriteRule .* - [E=INFO_TIME_SEC:%{TIME_SEC}]
RewriteRule .* - [E=INFO_TIME_WDAY:%{TIME_WDAY}]
RewriteRule .* - [E=INFO_TIME_YEAR:%{TIME_YEAR}]
RewriteRule .* - [E=INFO_TZ:%{TZ}]
RewriteRule .* - [E=INFO_UNIQUE_ID:%{UNIQUE_ID}]
 
RequestHeader set INFO_API_VERSION "%{INFO_API_VERSION}e"
RequestHeader set INFO_AUTH_TYPE "%{INFO_AUTH_TYPE}e"
RequestHeader set INFO_CONTENT_LENGTH "%{INFO_CONTENT_LENGTH}e"
RequestHeader set INFO_CONTENT_TYPE "%{INFO_CONTENT_TYPE}e"
RequestHeader set INFO_DOCUMENT_ROOT "%{INFO_DOCUMENT_ROOT}e"
RequestHeader set INFO_GATEWAY_INTERFACE "%{INFO_GATEWAY_INTERFACE}e"
RequestHeader set INFO_HTTPS "%{INFO_HTTPS}e"
RequestHeader set INFO_HTTP_ACCEPT "%{INFO_HTTP_ACCEPT}e"
RequestHeader set INFO_HTTP_ACCEPT_LANGUAGE "%{INFO_HTTP_ACCEPT_LANGUAGE}e"
RequestHeader set INFO_HTTP_ACCEPT_ENCODING "%{INFO_HTTP_ACCEPT_ENCODING}e"
RequestHeader set INFO_HTTP_ACCEPT_CHARSET "%{INFO_HTTP_ACCEPT_CHARSET}e"
RequestHeader set INFO_HTTP_CACHE_CONTROL "%{INFO_HTTP_CACHE_CONTROL}e"
RequestHeader set INFO_HTTP_CONNECTION "%{INFO_HTTP_CONNECTION}e"
RequestHeader set INFO_HTTP_COOKIE "%{INFO_HTTP_COOKIE}e"
RequestHeader set INFO_HTTP_FORWARDED "%{INFO_HTTP_FORWARDED}e"
RequestHeader set INFO_HTTP_HOST "%{INFO_HTTP_HOST}e"
RequestHeader set INFO_HTTP_KEEP_ALIVE "%{INFO_HTTP_KEEP_ALIVE}e"
RequestHeader set INFO_HTTP_PROXY_CONNECTION "%{INFO_HTTP_PROXY_CONNECTION}e"
RequestHeader set INFO_HTTP_REFERER "%{INFO_HTTP_REFERER}e"
RequestHeader set INFO_HTTP_USER_AGENT "%{INFO_HTTP_USER_AGENT}e"
RequestHeader set INFO_IS_SUBREQ "%{INFO_IS_SUBREQ}e"
RequestHeader set INFO_ORIG_PATH_INFO "%{INFO_ORIG_PATH_INFO}e"
RequestHeader set INFO_ORIG_PATH_TRANSLATED "%{INFO_ORIG_PATH_TRANSLATED}e"
RequestHeader set INFO_ORIG_SCRIPT_FILENAME "%{INFO_ORIG_SCRIPT_FILENAME}e"
RequestHeader set INFO_ORIG_SCRIPT_NAME "%{INFO_ORIG_SCRIPT_NAME}e"
RequestHeader set INFO_PATH "%{INFO_PATH}e"
RequestHeader set INFO_PATH_INFO "%{INFO_PATH_INFO}e"
RequestHeader set INFO_QUERY_STRING "%{INFO_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_QUERY_STRING "%{INFO_REDIRECT_QUERY_STRING}e"
RequestHeader set INFO_REDIRECT_REMOTE_USER "%{INFO_REDIRECT_REMOTE_USER}e"
RequestHeader set INFO_REDIRECT_STATUS "%{INFO_REDIRECT_STATUS}e"
RequestHeader set INFO_REDIRECT_URL "%{INFO_REDIRECT_URL}e"
RequestHeader set INFO_REMOTE_ADDR "%{INFO_REMOTE_ADDR}e"
RequestHeader set INFO_REMOTE_HOST "%{INFO_REMOTE_HOST}e"
RequestHeader set INFO_REMOTE_IDENT "%{INFO_REMOTE_IDENT}e"
RequestHeader set INFO_REMOTE_PORT "%{INFO_REMOTE_PORT}e"
RequestHeader set INFO_REMOTE_USER "%{INFO_REMOTE_USER}e"
RequestHeader set INFO_REQUEST_FILENAME "%{INFO_REQUEST_FILENAME}e"
RequestHeader set INFO_REQUEST_METHOD "%{INFO_REQUEST_METHOD}e"
RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"
RequestHeader set INFO_REQUEST_URI "%{INFO_REQUEST_URI}e"
RequestHeader set INFO_SCRIPT_FILENAME "%{INFO_SCRIPT_FILENAME}e"
RequestHeader set INFO_SCRIPT_GROUP "%{INFO_SCRIPT_GROUP}e"
RequestHeader set INFO_SCRIPT_NAME "%{INFO_SCRIPT_NAME}e"
RequestHeader set INFO_SCRIPT_URI "%{INFO_SCRIPT_URI}e"
RequestHeader set INFO_SCRIPT_URL "%{INFO_SCRIPT_URL}e"
RequestHeader set INFO_SCRIPT_USER "%{INFO_SCRIPT_USER}e"
RequestHeader set INFO_SERVER_ADDR "%{INFO_SERVER_ADDR}e"
RequestHeader set INFO_SERVER_ADMIN "%{INFO_SERVER_ADMIN}e"
RequestHeader set INFO_SERVER_NAME "%{INFO_SERVER_NAME}e"
RequestHeader set INFO_SERVER_PORT "%{INFO_SERVER_PORT}e"
RequestHeader set INFO_SERVER_PROTOCOL "%{INFO_SERVER_PROTOCOL}e"
RequestHeader set INFO_SERVER_SIGNATURE "%{INFO_SERVER_SIGNATURE}e"
RequestHeader set INFO_SERVER_SOFTWARE "%{INFO_SERVER_SOFTWARE}e"
RequestHeader set INFO_THE_REQUEST "%{INFO_THE_REQUEST}e"
RequestHeader set INFO_TIME "%{INFO_TIME}e"
RequestHeader set INFO_TIME_DAY "%{INFO_TIME_DAY}e"
RequestHeader set INFO_TIME_HOUR "%{INFO_TIME_HOUR}e"
RequestHeader set INFO_TIME_MIN "%{INFO_TIME_MIN}e"
RequestHeader set INFO_TIME_MON "%{INFO_TIME_MON}e"
RequestHeader set INFO_TIME_SEC "%{INFO_TIME_SEC}e"
RequestHeader set INFO_TIME_WDAY "%{INFO_TIME_WDAY}e"
RequestHeader set INFO_TIME_YEAR "%{INFO_TIME_YEAR}e"
RequestHeader set INFO_TZ "%{INFO_TZ}e"
RequestHeader set INFO_UNIQUE_ID "%{INFO_UNIQUE_ID}e"
 
Options +FollowSymLinks +ExecCGI
DirectoryIndex /cgi-bin/rewrite-test/index.php
 
Header echo ^.*
 
AuthType Digest
AuthName "AskApache Pro"
AuthDigestFile /home/askapache/sites/askapache.com/.htpasswd-pro
AuthDigestDomain / http://www.askapache.com/cgi-bin/rewrite-test/ https://www.askapache.com/cgi-bin/rewrite-test/
Require user askapache
 
SetEnv MODSEC_ENABLE=On
 
SetEnvIfNoCase ^WWW-Auth "(.+)" HTTP_WWW_AUTHORIZATION=$1
SetEnvIfNoCase ^If "(.+)" HTTP_IF_MODIFIED_SINCE=$1
SetEnvIfNoCase ^If-None-Match$ "(.+)" HTTP_IF_NONE_MATCH=$1
SetEnvIfNoCase ^Cache-Control$ "(.+)" HTTP_CACHE_CONTROL=$1
SetEnvIfNoCase ^Connection$ "(.+)" HTTP_CONNECTION=$1
SetEnvIfNoCase ^Keep-Alive$ "(.+)" HTTP_KEEP_ALIVE=$1
SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
SetEnvIfNoCase ^Authorization$ ".+username=\"(.+)\".+" HTTP_REMOTE_USER=$1
SetEnvIfNoCase ^Content-Type$ "(.+)" HTTP_CONTENT_TYPE=$1
SetEnvIfNoCase ^Content-Length$ "(.+)" HTTP_CONTENT_LENGTH=$1
SetEnvIfNoCase Server_Addr "(.+)" SERVER_ADDR=$1
SetEnvIfNoCase Request_Method "(.+)" REQUEST_METHOD=$1
SetEnvIfNoCase Request_Protocol "(.+)" REQUEST_PROTOCOL=$1
SetEnvIfNoCase Request_URI "(.+)" REQUEST_URI=$1
 
ErrorDocument 100 /cgi-bin/rewrite-test/index.php?g=100
ErrorDocument 101 /cgi-bin/rewrite-test/index.php?g=101
ErrorDocument 102 /cgi-bin/rewrite-test/index.php?g=102
ErrorDocument 200 /cgi-bin/rewrite-test/index.php?g=200
ErrorDocument 201 /cgi-bin/rewrite-test/index.php?g=201
ErrorDocument 202 /cgi-bin/rewrite-test/index.php?g=202
ErrorDocument 203 /cgi-bin/rewrite-test/index.php?g=203
ErrorDocument 204 /cgi-bin/rewrite-test/index.php?g=204
ErrorDocument 205 /cgi-bin/rewrite-test/index.php?g=205
ErrorDocument 206 /cgi-bin/rewrite-test/index.php?g=206
ErrorDocument 207 /cgi-bin/rewrite-test/index.php?g=207
ErrorDocument 300 /cgi-bin/rewrite-test/index.php?g=300
ErrorDocument 301 /cgi-bin/rewrite-test/index.php?g=301
ErrorDocument 302 /cgi-bin/rewrite-test/index.php?g=302
ErrorDocument 303 /cgi-bin/rewrite-test/index.php?g=303
ErrorDocument 304 /cgi-bin/rewrite-test/index.php?g=304
ErrorDocument 305 /cgi-bin/rewrite-test/index.php?g=305
ErrorDocument 306 /cgi-bin/rewrite-test/index.php?g=306
ErrorDocument 307 /cgi-bin/rewrite-test/index.php?g=307
ErrorDocument 400 /cgi-bin/rewrite-test/index.php?g=400
ErrorDocument 401 /cgi-bin/rewrite-test/index.php?g=401
ErrorDocument 402 /cgi-bin/rewrite-test/index.php?g=402
ErrorDocument 403 /cgi-bin/rewrite-test/index.php?g=403
ErrorDocument 404 /cgi-bin/rewrite-test/index.php?g=404
ErrorDocument 405 /cgi-bin/rewrite-test/index.php?g=405
ErrorDocument 406 /cgi-bin/rewrite-test/index.php?g=406
ErrorDocument 407 /cgi-bin/rewrite-test/index.php?g=407
ErrorDocument 408 /cgi-bin/rewrite-test/index.php?g=408
ErrorDocument 409 /cgi-bin/rewrite-test/index.php?g=409
ErrorDocument 410 /cgi-bin/rewrite-test/index.php?g=410
ErrorDocument 411 /cgi-bin/rewrite-test/index.php?g=411
ErrorDocument 412 /cgi-bin/rewrite-test/index.php?g=412
ErrorDocument 413 /cgi-bin/rewrite-test/index.php?g=413
ErrorDocument 414 /cgi-bin/rewrite-test/index.php?g=414
ErrorDocument 415 /cgi-bin/rewrite-test/index.php?g=415
ErrorDocument 416 /cgi-bin/rewrite-test/index.php?g=416
ErrorDocument 417 /cgi-bin/rewrite-test/index.php?g=417
ErrorDocument 418 /cgi-bin/rewrite-test/index.php?g=418
ErrorDocument 419 /cgi-bin/rewrite-test/index.php?g=419
ErrorDocument 420 /cgi-bin/rewrite-test/index.php?g=420
ErrorDocument 421 /cgi-bin/rewrite-test/index.php?g=421
ErrorDocument 422 /cgi-bin/rewrite-test/index.php?g=422
ErrorDocument 423 /cgi-bin/rewrite-test/index.php?g=423
ErrorDocument 424 /cgi-bin/rewrite-test/index.php?g=424
ErrorDocument 425 /cgi-bin/rewrite-test/index.php?g=425
ErrorDocument 426 /cgi-bin/rewrite-test/index.php?g=426
ErrorDocument 500 /cgi-bin/rewrite-test/index.php?g=500
ErrorDocument 501 /cgi-bin/rewrite-test/index.php?g=501
ErrorDocument 502 /cgi-bin/rewrite-test/index.php?g=502
ErrorDocument 503 /cgi-bin/rewrite-test/index.php?g=503
ErrorDocument 504 /cgi-bin/rewrite-test/index.php?g=504
ErrorDocument 505 /cgi-bin/rewrite-test/index.php?g=505
ErrorDocument 506 /cgi-bin/rewrite-test/index.php?g=506
ErrorDocument 507 /cgi-bin/rewrite-test/index.php?g=507
ErrorDocument 508 /cgi-bin/rewrite-test/index.php?g=508
ErrorDocument 509 /cgi-bin/rewrite-test/index.php?g=509
ErrorDocument 510 /cgi-bin/rewrite-test/index.php?g=510
 
RewriteEngine On
RewriteBase /
 
RewriteRule .* - [E=IN_AUTH_TYPE:%{AUTH_TYPE}]
RewriteRule .* - [E=IN_CONTENT_LENGTH:%{CONTENT_LENGTH}]
RewriteRule .* - [E=IN_CONTENT_TYPE:%{CONTENT_TYPE}]
RewriteRule .* - [E=IN_DATE_GMT:%{DATE_GMT}]
RewriteRule .* - [E=IN_DATE_LOCAL:%{DATE_LOCAL}]
RewriteRule .* - [E=IN_DOCUMENT_NAME:%{DOCUMENT_NAME}]
RewriteRule .* - [E=IN_DOCUMENT_PATH_INFO:%{DOCUMENT_PATH_INFO}]
RewriteRule .* - [E=IN_DOCUMENT_ROOT:%{DOCUMENT_ROOT}]
RewriteRule .* - [E=IN_DOCUMENT_URI:%{DOCUMENT_URI}]
RewriteRule .* - [E=IN_GATEWAY_INTERFACE:%{GATEWAY_INTERFACE}]
RewriteRule .* - [E=IN_LAST_MODIFIED:%{LAST_MODIFIED}]
RewriteRule .* - [E=IN_PATH_INFO:%{PATH_INFO}]
RewriteRule .* - [E=IN_PATH_TRANSLATED:%{PATH_TRANSLATED}]
RewriteRule .* - [E=IN_QUERY_STRING:%{QUERY_STRING}]
RewriteRule .* - [E=IN_QUERY_STRING_UNESCAPED:%{QUERY_STRING_UNESCAPED}]
RewriteRule .* - [E=IN_REMOTE_ADDR:%{REMOTE_ADDR}]
RewriteRule .* - [E=IN_REMOTE_HOST:%{REMOTE_HOST}]
RewriteRule .* - [E=IN_REMOTE_IDENT:%{REMOTE_IDENT}]
RewriteRule .* - [E=IN_REMOTE_PORT:%{REMOTE_PORT}]
RewriteRule .* - [E=IN_REMOTE_USER:%{REMOTE_USER}]
RewriteRule .* - [E=IN_REDIRECT_HANDLER:%{REDIRECT_HANDLER}]
RewriteRule .* - [E=IN_REDIRECT_QUERY_STRING:%{REDIRECT_QUERY_STRING}]
RewriteRule .* - [E=IN_REDIRECT_REMOTE_USER:%{REDIRECT_REMOTE_USER}]
RewriteRule .* - [E=IN_REDIRECT_STATUS:%{REDIRECT_STATUS}]
RewriteRule .* - [E=IN_REDIRECT_URL:%{REDIRECT_URL}]
RewriteRule .* - [E=IN_REQUEST_METHOD:%{REQUEST_METHOD}]
RewriteRule .* - [E=IN_REQUEST_URI:%{REQUEST_URI}]
RewriteRule .* - [E=IN_SCRIPT_FILENAME:%{SCRIPT_FILENAME}]
RewriteRule .* - [E=IN_SCRIPT_NAME:%{SCRIPT_NAME}]
RewriteRule .* - [E=IN_SERVER_ADMIN:%{SERVER_ADMIN}]
RewriteRule .* - [E=IN_SERVER_NAME:%{SERVER_NAME}]
RewriteRule .* - [E=IN_SERVER_ADDR:%{SERVER_ADDR}]
RewriteRule .* - [E=IN_SERVER_PORT:%{SERVER_PORT}]
RewriteRule .* - [E=IN_SERVER_PROTOCOL:%{SERVER_PROTOCOL}]
RewriteRule .* - [E=IN_SERVER_SIGNATURE:%{SERVER_SIGNATURE}]
RewriteRule .* - [E=IN_SERVER_SOFTWARE:%{SERVER_SOFTWARE}]
RewriteRule .* - [E=IN_USER_NAME:%{USER_NAME}]
RewriteRule .* - [E=IN_TZ:%{TZ}]
RewriteRule .* - [E=IN_API_VERSION:%{API_VERSION}]
RewriteRule .* - [E=IN_HTTPS:%{HTTPS}]
RewriteRule .* - [E=IN_HTTP_ACCEPT:%{HTTP_ACCEPT}]
RewriteRule .* - [E=IN_HTTP_ACCEPT_CHARSET:%{HTTP_ACCEPT_CHARSET}]
RewriteRule .* - [E=IN_HTTP_ACCEPT_ENCODING:%{HTTP_ACCEPT_ENCODING}]
RewriteRule .* - [E=IN_HTTP_ACCEPT_LANGUAGE:%{HTTP_ACCEPT_LANGUAGE}]
RewriteRule .* - [E=IN_HTTP_CACHE_CONTROL:%{HTTP_CACHE_CONTROL}]
RewriteRule .* - [E=IN_HTTP_CONNECTION:%{HTTP_CONNECTION}]
RewriteRule .* - [E=IN_HTTP_COOKIE:%{HTTP_COOKIE}]
RewriteRule .* - [E=IN_HTTP_FORWARDED:%{HTTP_FORWARDED}]
RewriteRule .* - [E=IN_HTTP_HOST:%{HTTP_HOST}]
RewriteRule .* - [E=IN_HTTP_KEEP_ALIVE:%{HTTP_KEEP_ALIVE}]
RewriteRule .* - [E=IN_HTTP_PROXY_CONNECTION:%{HTTP_PROXY_CONNECTION}]
RewriteRule .* - [E=IN_HTTP_REFERER:%{HTTP_REFERER}]
RewriteRule .* - [E=IN_HTTP_USER_AGENT:%{HTTP_USER_AGENT}]
RewriteRule .* - [E=IN_IS_SUBREQ:%{IS_SUBREQ}]
RewriteRule .* - [E=IN_ORIG_PATH_INFO:%{ORIG_PATH_INFO}]
RewriteRule .* - [E=IN_ORIG_PATH_TRANSLATED:%{ORIG_PATH_TRANSLATED}]
RewriteRule .* - [E=IN_ORIG_SCRIPT_FILENAME:%{ORIG_SCRIPT_FILENAME}]
RewriteRule .* - [E=IN_ORIG_SCRIPT_NAME:%{ORIG_SCRIPT_NAME}]
RewriteRule .* - [E=IN_PATH:%{PATH}]
RewriteRule .* - [E=IN_PHP_SELF:%{PHP_SELF}]
RewriteRule .* - [E=IN_REQUEST_FILENAME:%{REQUEST_FILENAME}]
RewriteRule .* - [E=IN_REQUEST_TIME:%{REQUEST_TIME}]
RewriteRule .* - [E=IN_SCRIPT_GROUP:%{SCRIPT_GROUP}]
RewriteRule .* - [E=IN_SCRIPT_USER:%{SCRIPT_USER}]
RewriteRule .* - [E=IN_THE_REQUEST:%{THE_REQUEST}]
RewriteRule .* - [E=IN_TIME:%{TIME}]
RewriteRule .* - [E=IN_TIME_DAY:%{TIME_DAY}]
RewriteRule .* - [E=IN_TIME_HOUR:%{TIME_HOUR}]
RewriteRule .* - [E=IN_TIME_MIN:%{TIME_MIN}]
RewriteRule .* - [E=IN_TIME_MON:%{TIME_MON}]
RewriteRule .* - [E=IN_TIME_SEC:%{TIME_SEC}]
RewriteRule .* - [E=IN_TIME_WDAY:%{TIME_WDAY}]
RewriteRule .* - [E=IN_TIME_YEAR:%{TIME_YEAR}]
RewriteRule .* - [E=IN_PATH:%{PATH}]
RewriteRule .* - [E=IN_SCRIPT_URI:%{SCRIPT_URI}]
RewriteRule .* - [E=IN_SCRIPT_URL:%{SCRIPT_URL}]
RewriteRule .* - [E=IN_UNIQUE_ID:%{UNIQUE_ID}]
 
RewriteRule .* - [E=ENV_PATH:%{ENV:PATH}]
RewriteRule .* - [E=ENV_SCRIPT_URI:%{ENV:SCRIPT_URI}]
RewriteRule .* - [E=ENV_SCRIPT_URL:%{ENV:SCRIPT_URL}]
RewriteRule .* - [E=ENV_UNIQUE_ID:%{ENV:UNIQUE_ID}]
 
RequestHeader set AUTH_TYPE "%{IN_AUTH_TYPE}e"
RequestHeader set CONTENT_LENGTH "%{IN_CONTENT_LENGTH}e"
RequestHeader set CONTENT_TYPE "%{IN_CONTENT_TYPE}e"
RequestHeader set DATE_GMT "%{IN_DATE_GMT}e"
RequestHeader set DATE_LOCAL "%{IN_DATE_LOCAL}e"
RequestHeader set DOCUMENT_NAME "%{IN_DOCUMENT_NAME}e"
RequestHeader set DOCUMENT_PATH_INFO "%{IN_DOCUMENT_PATH_INFO}e"
RequestHeader set DOCUMENT_ROOT "%{IN_DOCUMENT_ROOT}e"
RequestHeader set DOCUMENT_URI "%{IN_DOCUMENT_URI}e"
RequestHeader set GATEWAY_INTERFACE "%{IN_GATEWAY_INTERFACE}e"
RequestHeader set LAST_MODIFIED "%{IN_LAST_MODIFIED}e"
RequestHeader set PATH_INFO "%{IN_PATH_INFO}e"
RequestHeader set PATH_TRANSLATED "%{IN_PATH_TRANSLATED}e"
RequestHeader set QUERY_STRING "%{IN_QUERY_STRING}e"
RequestHeader set QUERY_STRING_UNESCAPED "%{IN_QUERY_STRING_UNESCAPED}e"
RequestHeader set REMOTE_ADDR "%{IN_REMOTE_ADDR}e"
RequestHeader set REMOTE_HOST "%{IN_REMOTE_HOST}e"
RequestHeader set REMOTE_IDENT "%{IN_REMOTE_IDENT}e"
RequestHeader set REMOTE_PORT "%{IN_REMOTE_PORT}e"
RequestHeader set REMOTE_USER "%{IN_REMOTE_USER}e"
RequestHeader set REDIRECT_HANDLER "%{IN_REDIRECT_HANDLER}e"
RequestHeader set REDIRECT_QUERY_STRING "%{IN_REDIRECT_QUERY_STRING}e"
RequestHeader set REDIRECT_REMOTE_USER "%{IN_REDIRECT_REMOTE_USER}e"
RequestHeader set REDIRECT_STATUS "%{IN_REDIRECT_STATUS}e"
RequestHeader set REDIRECT_URL "%{IN_REDIRECT_URL}e"
RequestHeader set REQUEST_METHOD "%{IN_REQUEST_METHOD}e"
RequestHeader set REQUEST_URI "%{IN_REQUEST_URI}e"
RequestHeader set SCRIPT_FILENAME "%{IN_SCRIPT_FILENAME}e"
RequestHeader set SCRIPT_NAME "%{IN_SCRIPT_NAME}e"
RequestHeader set SCRIPT_URI "%{IN_SCRIPT_URI}e"
RequestHeader set SCRIPT_URL "%{IN_SCRIPT_URL}e"
RequestHeader set SERVER_ADMIN "%{IN_SERVER_ADMIN}e"
RequestHeader set SERVER_NAME "%{IN_SERVER_NAME}e"
RequestHeader set SERVER_ADDR "%{IN_SERVER_ADDR}e"
RequestHeader set SERVER_PORT "%{IN_SERVER_PORT}e"
RequestHeader set SERVER_PROTOCOL "%{IN_SERVER_PROTOCOL}e"
RequestHeader set SERVER_SIGNATURE "%{IN_SERVER_SIGNATURE}e"
RequestHeader set SERVER_SOFTWARE "%{IN_SERVER_SOFTWARE}e"
RequestHeader set UNIQUE_ID "%{IN_UNIQUE_ID}e"
RequestHeader set USER_NAME "%{IN_USER_NAME}e"
RequestHeader set TZ "%{IN_TZ}e"
RequestHeader set API_VERSION "%{IN_API_VERSION}e"
RequestHeader set HTTPS "%{IN_HTTPS}e"
RequestHeader set HTTP_ACCEPT "%{IN_HTTP_ACCEPT}e"
RequestHeader set HTTP_ACCEPT_CHARSET "%{IN_HTTP_ACCEPT_CHARSET}e"
RequestHeader set HTTP_ACCEPT_ENCODING "%{IN_HTTP_ACCEPT_ENCODING}e"
RequestHeader set HTTP_ACCEPT_LANGUAGE "%{IN_HTTP_ACCEPT_LANGUAGE}e"
RequestHeader set HTTP_CACHE_CONTROL "%{IN_HTTP_CACHE_CONTROL}e"
RequestHeader set HTTP_CONNECTION "%{IN_HTTP_CONNECTION}e"
RequestHeader set HTTP_COOKIE "%{IN_HTTP_COOKIE}e"
RequestHeader set HTTP_FORWARDED "%{IN_HTTP_FORWARDED}e"
RequestHeader set HTTP_HOST "%{IN_HTTP_HOST}e"
RequestHeader set HTTP_KEEP_ALIVE "%{IN_HTTP_KEEP_ALIVE}e"
RequestHeader set HTTP_PROXY_CONNECTION "%{IN_HTTP_PROXY_CONNECTION}e"
RequestHeader set HTTP_REFERER "%{IN_HTTP_REFERER}e"
RequestHeader set HTTP_USER_AGENT "%{IN_HTTP_USER_AGENT}e"
RequestHeader set IS_SUBREQ "%{IN_IS_SUBREQ}e"
RequestHeader set ORIG_PATH_INFO "%{IN_ORIG_PATH_INFO}e"
RequestHeader set ORIG_PATH_TRANSLATED "%{IN_ORIG_PATH_TRANSLATED}e"
RequestHeader set ORIG_SCRIPT_FILENAME "%{IN_ORIG_SCRIPT_FILENAME}e"
RequestHeader set ORIG_SCRIPT_NAME "%{IN_ORIG_SCRIPT_NAME}e"
RequestHeader set PATH "%{IN_PATH}e"
RequestHeader set PHP_SELF "%{IN_PHP_SELF}e"
RequestHeader set REQUEST_FILENAME "%{IN_REQUEST_FILENAME}e"
RequestHeader set REQUEST_TIME "%{IN_REQUEST_TIME}e"
RequestHeader set SCRIPT_GROUP "%{IN_SCRIPT_GROUP}e"
RequestHeader set SCRIPT_USER "%{IN_SCRIPT_USER}e"
RequestHeader set THE_REQUEST "%{IN_THE_REQUEST}e"
RequestHeader set TIME "%{IN_TIME}e"
RequestHeader set TIME_DAY "%{IN_TIME_DAY}e"
RequestHeader set TIME_HOUR "%{IN_TIME_HOUR}e"
RequestHeader set TIME_MIN "%{IN_TIME_MIN}e"
RequestHeader set TIME_MON "%{IN_TIME_MON}e"
RequestHeader set TIME_SEC "%{IN_TIME_SEC}e"
RequestHeader set TIME_WDAY "%{IN_TIME_WDAY}e"
RequestHeader set TIME_YEAR "%{IN_TIME_YEAR}e"
 
SetEnvIfNoCase ^WWW-Auth "(.+)" HTTP_WWW_AUTHORIZATION=$1
SetEnvIfNoCase ^If "(.+)" HTTP_IF_MODIFIED_SINCE=$1
SetEnvIfNoCase ^If-None-Match$ "(.+)" HTTP_IF_NONE_MATCH=$1
SetEnvIfNoCase ^Cache-Control$ "(.+)" HTTP_CACHE_CONTROL=$1
SetEnvIfNoCase ^Connection$ "(.+)" HTTP_CONNECTION=$1
SetEnvIfNoCase ^Keep-Alive$ "(.+)" HTTP_KEEP_ALIVE=$1
SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
SetEnvIfNoCase ^Content-Type$ "(.+)" HTTP_CONTENT_TYPE=$1
SetEnvIfNoCase ^Content-Length$ "(.+)" HTTP_CONTENT_LENGTH=$1
SetEnvIfNoCase ^Authorization$ ".+username=\"([a-zA-Z0-9]+)\".+" REMOTE_USER=$1
SetEnvIfNoCase Server_Addr "(.+)" SERVER_ADDR=$1
SetEnvIfNoCase Request_Method "(.+)" REQUEST_METHOD=$1
SetEnvIfNoCase Request_Protocol "(.+)" REQUEST_PROTOCOL=$1
SetEnvIfNoCase Request_URI "(.+)" REQUEST_URI=$1
 
RequestHeader set IF_MODIFIED_SINCE "%{HTTP_IF_MODIFIED_SINCE}e"
RequestHeader set IF_NONE_MATCH "%{HTTP_IF_NONE_MATCH}e"
RequestHeader set CACHE_CONTROL "%{HTTP_CACHE_CONTROL}e"
RequestHeader set CONNECTION "%{HTTP_CONNECTION}e"
RequestHeader set KEEP_ALIVE "%{HTTP_KEEP_ALIVE}e"
RequestHeader set AUTHORIZATION "%{HTTP_AUTHORIZATION}e"
RequestHeader set REMOTE_USER "%{REMOTE_USER}e"
RequestHeader set CONTENT_TYPE "%{HTTP_CONTENT_TYPE}e"
RequestHeader set CONTENT_LENGTH "%{HTTP_CONTENT_LENGTH}e"
RequestHeader set SERVER_ADDR "%{SERVER_ADDR}e"
RequestHeader set REQUEST_METHOD "%{REQUEST_METHOD}e"
RequestHeader set REQUEST_PROTOCOL "%{REQUEST_PROTOCOL}e"
RequestHeader set REQUEST_URI "%{REQUEST_URI}e"
 
RequestHeader set UNIQUE_ID "%{ENV_UNIQUE_ID}e"
RequestHeader set SCRIPT_URL "%{ENV_SCRIPT_URL}e"
RequestHeader set SCRIPT_URI "%{ENV_SCRIPT_URI}e"
RequestHeader set PATH "%{ENV_PATH}e"
 
Options +ExecCGI +FollowSymLinks
 
Order Deny,Allow
Deny from All
Allow from 208.113.134.190  64.111.114.111 208.113.134.203 208.113.152.201 env=REDIRECT_STATUS
Satisfy Any
 
SecFilterEngine Off

More Mod_Security (1)

#
# Order Deny,Allow
# First, all Allow directives are evaluated; at least one must match, or the request is rejected.
# Next, all Deny directives are evaluated. If any matches, the request is rejected.
# Last, any requests which do not match an Allow or a Deny directive are denied by default.
#
Order Allow,Deny
Allow from all
Deny from 217.219.
 
#Order Deny,Allow
# First, all Deny directives are evaluated; if any match, the request is denied unless it also matches an Allow directive.
# Any requests which do not match any Allow or Deny directives are permitted.
 
#SetEnvIf content-type (multipart/form-data)(.*) NEW_CONTENT_TYPE=application/x-www-form-urlencoded$2 OLD_CONTENT_TYPE=$1$2
#RequestHeader set content-type %{NEW_CONTENT_TYPE}e env=NEW_CONTENT_TYPE
SetEnvIfNoCase Content-Type "^multipart/form-data" !MODSEC_NOPOSTBUFFERING
SetEnvIfNoCase Content-Type "^application/x-www-form-urlencoded" !MODSEC_NOPOSTBUFFERING
SetEnv suppress-error-charset
SetEnvIfNoCase Content-Type "^multipart/form-data" !MODSEC_NOPOSTBUFFERING
 
SetEnvIf Request_URI "^/(cgi-bin/search\.php|cgi-bin/java\.cgi|wp-admin/.*)" MODSEC_ENABLE=Off
SetEnvIf Request_URI "^/(online-tools/js-compress.*)" "MODSEC_NOPOSTBUFFERING=Do not buffer file uploads"
SetEnvIfNoCase Remote_Addr ^208\.113\.134\.190$ MODSEC_ENABLE=Off
SetEnvIfNoCase Remote_Addr ^64\.111\.114\.111$ MODSEC_ENABLE=Off
 
### Section 1: Global Environment
#
# The directives in this section affect the overall operation of Apache,
# such as the number of concurrent requests it can handle or where it
# can find its configuration files.
#
#
# TZ: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv TZ America/Indianapolis
 
#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv SERVER_ADMIN webmaster@askapache.com
 
#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature Off
 
#
# Possible values for the Options directive are "None", "All",
# or any combination of:
#   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
#
# The Options directive is both complicated and important.  Please see
# http://httpd.apache.org/docs/2.0/mod/core.html#options
# for more information.
#
Options -Indexes -Includes -ExecCGI -MultiViews
 
#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
DirectoryIndex index.php
 
#
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#
Action php5-cgi /bin/php.cgi
 
#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
AddHandler php5-cgi .php .inc
 
#
# Commonly used filename extensions to character sets. You probably
# want to avoid clashes with the language extensions, unless you
# are good at carefully testing your setup after each change.
# See http://www.iana.org/assignments/character-sets for the
# official list of charset names and their respective RFCs.
#
AddDefaultCharset UTF-8
 
#
# AddType allows you to add to or override the MIME configuration
# file mime.types for specific file types.
#
#
AddType 'application/rdf+xml; charset=UTF-8' .rdf
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml.gz
AddType 'text/html; charset=UTF-8' .html
AddType 'text/html; charset=UTF-8' .html.gz
AddType application/octet-stream .rar .chm .bz2 .tgz .msi .pdf .exe
AddType application/vnd.ms-excel .csv
AddType application/x-httpd-php-source .phps
AddType application/x-pilot .prc .pdb
AddType application/x-shockwave-flash .swf
AddType application/xrds+xml .xrdf
AddType text/plain .ini .sh .bsh .bash .awk .nawk .gawk .csh .var .c .in .h .asc .md5 .sha .sha1
AddType video/x-flv .flv
 
#
# AddEncoding allows you to have certain browsers uncompress
# information on the fly. Note: Not all browsers support this.
# Despite the name similarity, the following Add* directives have nothing
# to do with the FancyIndexing customization directives above.
#
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
 
#
# DefaultType: the default MIME type the server will use for a document
# if it cannot otherwise determine one, such as from filename extensions.
# If your server contains mostly text or HTML documents, "text/plain" is
# a good value.  If most of your content is binary, such as applications
# or images, you may want to use "application/octet-stream" instead to
# keep browsers from trying to display binary files as though they are
# text.
#
DefaultType text/html

Error Documents

#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
#100 Continue
#101 Switching Protocols
#102 Processing
#200 OK
#201 Created
#202 Accepted
#203 Non-Authoritative Information
#204 No Content
#205 Reset Content
#206 Partial Content
#207 Multi-Status
#300 Multiple Choices
#301 Moved Permanently
#302 Found
#303 See Other
#304 Not Modified
#305 Use Proxy
#306 unused
#307 Temporary Redirect
#400 Bad Request
#401 Authorization Required
#402 Payment Required
#403 Forbidden
#404 Not Found
#405 Method Not Allowed
#406 Not Acceptable
#407 Proxy Authentication Required
#408 Request Time-out
#409 Conflict
#410 Gone
#411 Length Required
#412 Precondition Failed
#413 Request Entity Too Large
#414 Request-URI Too Large
#415 Unsupported Media Type
#416 Requested Range Not Satisfiable
#417 Expectation Failed
#418 unused
#419 unused
#420 unused
#421 unused
#422 Unprocessable Entity
#423 Locked
#424 Failed Dependency
#425 No code
#426 Upgrade Required
#500 Internal Server Error
#501 Method Not Implemented
#502 Bad Gateway
#503 Service Temporarily Unavailable
#504 Gateway Time-out
#505 HTTP Version Not Supported
#506 Variant Also Negotiates
#507 Insufficient Storage
#508 unused
#509 unused
#510 Not Extended
 
#ErrorDocument 100 /e/100_CONTINUE.html
#ErrorDocument 101 /e/101_SWITCHING_PROTOCOLS.html
#ErrorDocument 102 /e/102_PROCESSING.html
 
#ErrorDocument 200 /e/200_OK.html
#ErrorDocument 201 /e/201_CREATED.html
#ErrorDocument 202 /e/202_ACCEPTED.html
#ErrorDocument 203 /e/203_NON_AUTHORITATIVE.html
#ErrorDocument 204 /e/204_NO_CONTENT.html
#ErrorDocument 205 /e/205_RESET_CONTENT.html
#ErrorDocument 206 /e/206_PARTIAL_CONTENT.html
#ErrorDocument 207 /e/207_MULTI_STATUS.html
 
#ErrorDocument 300 /e/300_MULTIPLE_CHOICES.html
#ErrorDocument 301 /e/301_MOVED_PERMANENTLY.html
#ErrorDocument 302 /e/302_MOVED_TEMPORARILY.html
#ErrorDocument 303 /e/303_SEE_OTHER.html
#ErrorDocument 304 /e/304_NOT_MODIFIED.html
#ErrorDocument 305 /e/305_USE_PROXY.html
#ErrorDocument 307 /e/307_TEMPORARY_REDIRECT.html
 
ErrorDocument 400 /e/400_BAD_REQUEST.html
ErrorDocument 401 /e/401_UNAUTHORIZED.html
ErrorDocument 402 /e/402_PAYMENT_REQUIRED.html
ErrorDocument 403 /e/403_FORBIDDEN.html
#ErrorDocument 404 /e/404_NOT_FOUND.html
ErrorDocument 404 /index.php?error=404
ErrorDocument 405 /e/405_METHOD_NOT_ALLOWED.html
ErrorDocument 406 /e/406_NOT_ACCEPTABLE.html
ErrorDocument 407 /e/407_PROXY_AUTHENTICATION_REQUIRED.html
ErrorDocument 408 /e/408_REQUEST_TIME_OUT.html
ErrorDocument 409 /e/409_CONFLICT.html
ErrorDocument 410 /e/410_GONE.html
ErrorDocument 411 /e/411_LENGTH_REQUIRED.html
ErrorDocument 412 /e/412_PRECONDITION_FAILED.html
ErrorDocument 413 /e/413_REQUEST_ENTITY_TOO_LARGE.html
ErrorDocument 414 /e/414_REQUEST_URI_TOO_LARGE.html
ErrorDocument 415 /e/415_UNSUPPORTED_MEDIA_TYPE.html
ErrorDocument 416 /e/416_RANGE_NOT_SATISFIABLE.html
ErrorDocument 417 /e/417_EXPECTATION_FAILED.html
ErrorDocument 422 /e/422_UNPROCESSABLE_ENTITY.html
ErrorDocument 423 /e/423_LOCKED.html
ErrorDocument 424 /e/424_FAILED_DEPENDENCY.html
ErrorDocument 426 /e/426_UPGRADE_REQUIRED.html
ErrorDocument 500 /e/500_INTERNAL_SERVER_ERROR.html
ErrorDocument 501 /e/501_NOT_IMPLEMENTED.html
ErrorDocument 502 /e/502_BAD_GATEWAY.html
ErrorDocument 503 /e/503_SERVICE_UNAVAILABLE.html
ErrorDocument 504 /e/504_GATEWAY_TIME_OUT.html
ErrorDocument 505 /e/505_VERSION_NOT_SUPPORTED.html
ErrorDocument 506 /e/506_VARIANT_ALSO_VARIES.html
ErrorDocument 507 /e/507_INSUFFICIENT_STORAGE.html
ErrorDocument 510 /e/510_NOT_EXTENDED.html

Caching

#
#  HEADERS and CACHING
#
Header unset Pragma
FileETag None
Header unset ETag
 
# 1 YEAR
<filesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
Header unset P3P
Header unset Pragma
FileETag None
Header unset ETag
Header set Cache-Control "public,max-age=29030400"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified
</filesMatch>
 
# 2 HOURS
<filesMatch "\.(html|htm|xml|txt|xsl|rdf|rss)$">
<ifModule mod_expires.c>
ExpiresActive On
ExpiresDefault A3600
</ifModule>
</filesMatch>

Redirect Hack

#Redirect 400 /e/400
#Redirect 401 /e/401
#Redirect 402 /e/402
#Redirect 403 /e/403
Redirect 404 /index.php?error=404
#Redirect 405 /e/405
#Redirect 406 /e/406
#Redirect 407 /e/407
#Redirect 408 /e/408
#Redirect 409 /e/409
#Redirect 410 /e/410
#Redirect 411 /e/411
#Redirect 412 /e/412
#Redirect 413 /e/413
#Redirect 414 /e/414
#Redirect 415 /e/415
#Redirect 416 /e/416
#Redirect 417 /e/417
#Redirect 418 /e/418
#Redirect 419 /e/419
#Redirect 420 /e/420
#Redirect 421 /e/421
#Redirect 422 /e/422
#Redirect 423 /e/423
#Redirect 424 /e/424
#Redirect 425 /e/425
#Redirect 426 /e/426
#Redirect 500 /e/500
#Redirect 501 /e/501
#Redirect 502 /e/502
#Redirect 503 /e/503
#Redirect 504 /e/504
#Redirect 505 /e/505
#Redirect 506 /e/506
#Redirect 507 /e/507
#Redirect 508 /e/508
#Redirect 509 /e/509
#Redirect 510 /e/510

301 PERMANENT REDIRECTS

#
# Redirect allows you to tell clients about documents which used to exist in
# your server's namespace, but do not anymore. This allows you to tell the
# clients where to look for the relocated document.
#
Redirect 301 /12-lessons-for-those-afraid-of-css.html http://www.askapache.com/css/12-lessons-for-those-afraid-of-css.html
Redirect 301 /2006/htaccess/htaccesselite-ultimate-htaccess-article.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /2007/phpbb/sending-post-form-data-with-php-curl.html http://www.askapache.com/htaccess/sending-post-form-data-with-php-curl.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/php-and-ajax-shell-console.html http://www.askapache.com/tools/php-and-ajax-shell-console.html
Redirect 301 /27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html http://www.askapache.com/htaccess/27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html
Redirect 301 /404-google-wordpress-plugin.html http://www.askapache.com/seo/404-google-wordpress-plugin.html
Redirect 301 /503-service-temporarily-unavailable.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /Overview-about.rdf http://www.askapache.com/askapache-home.rdf
Redirect 301 /abbr-acronym.html http://www.askapache.com/xhtml/abbr-acronym.html
Redirect 301 /adsense-robots.html http://www.askapache.com/google/adsense-robots.html
Redirect 301 /alexa-toolbar-firefox.html http://www.askapache.com/tools/alexa-toolbar-firefox.html
Redirect 301 /allowing-access-from-1-static-ip-and-deny-the-rest.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /anti-virus-spyware-rootkit.html http://www.askapache.com/security/anti-virus-spyware-rootkit.html
Redirect 301 /apache-ssl-in-htaccess-examples.html http://www.askapache.com/htaccess/apache-ssl-in-htaccess-examples.html
Redirect 301 /awk-tutorial.html http://www.askapache.com/awk/awk-tutorial.html
Redirect 301 /best-adsense-optimization.html http://www.askapache.com/wordpress/best-adsense-optimization.html
Redirect 301 /commonly-used-htaccess-code-examples.html http://www.askapache.com/htaccess/commonly-used-htaccess-code-examples.html
Redirect 301 /css-background-image-sprite.html http://www.askapache.com/css/css-background-image-sprite.html
Redirect 301 /css-browser-screenshots.html http://www.askapache.com/tools/css-browser-screenshots.html
Redirect 301 /css-class-example.html http://www.askapache.com/css/css-class-example.html
Redirect 301 /curl-multi-downloads.html http://www.askapache.com/php/curl-multi-downloads.html
Redirect 301 /custom-boot-menu-in-windows-xp.html http://www.askapache.com/windows/custom-boot-menu-in-windows-xp.html
Redirect 301 /donate http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /donate/ http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /htaccess.txt http://z.askapache.com/p/htaccess.txt
Redirect 301 /htaccess/404-errorpages.html http://www.askapache.com/seo/google-ajax-search-seo-tips.html
Redirect 301 /htaccess/feedsmith http://www.askapache.com/htaccess/feedsmith-htaccess.html
Redirect 301 /htaccess/http-status-codes.html http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html
Redirect 301 /htaccess/instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /htaccess/speed-up-the-apache-web-server-with-configuration-hacks.html http://www.askapache.com/htaccess/apache-web-server-speed-configuration-hacks.html
Redirect 301 /instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /security/bypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/bypassing-vlanbypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/rigging-the-dreamhost-site-of-the-month-contest.html http://www.askapache.com/dreamhost/rigging-the-dreamhost-site-of-the-month-contest.html
Redirect 301 /seo/tailrankcom-robot.html http://www.askapache.com/seo/tailrank-robot.html
Redirect 301 /webmaster/caching-tutorial-for-webmasters.html http://www.askapache.com/htaccess/caching-tutorial-for-webmasters.html
Redirect 301 /webmaster/lft-traceroute-tool.html http://www.askapache.com/tools/lft-traceroute-tool.html

301 PERMANENT REDIRECTMATCH

#
#  PERMANENT REDIRECTMATCH
#
#RedirectMatch 301 ^/&(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/(.+)\.html/$ http://www.askapache.com/$1.html
RedirectMatch 301 ^/&amp(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/.*feed\.gif$ http://z.askapache.com/feed.gif
RedirectMatch 301 ^/([^/]+)//$ http://www.askapache.com/$1/
RedirectMatch 301 ^/(.+)/htaccesselite-ultimate-htaccess-article.html(.*) http://www.askapache.com/htaccess/apache-htaccess.html
RedirectMatch 301 ^/(.+)\.html/([a-z][a-z])/$ http://www.askapache.com/$1.html
RedirectMatch 301 ^/([\(]+)(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/([^9]*)9O1X.3y(.*)/(.*)$ http://www.askapache.com/$2
RedirectMatch 301 ^/.3y(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/200([0-9])/([0-9])(.*)$ http://www.askapache.com/top-100/
RedirectMatch 301 ^/200([0-9])/([^01])(.*)$ http://www.askapache.com/$2$3
RedirectMatch 301 ^/about/glossary(.*)$ http://www.askapache.com/glossary$1
RedirectMatch 301 ^/apache-speed(.*)$ http://www.askapache.com/htaccess/apache-speed$1
RedirectMatch 301 ^/category/(.+)$ http://www.askapache.com/$1
RedirectMatch 301 ^/docs/(.*)$ http://askapache.info/$1
RedirectMatch 301 ^/htaccess/feedsmith-htaccess(.*) http://www.askapache.com/htaccess/redirecting-wordpress-feeds-to-feedburner.html
RedirectMatch 301 ^/robots-txt(.*)$ http://www.askapache.com/robots.txt
RedirectMatch 301 ^/hosting/?$ http://www.askapache.com/hosting/
RedirectMatch 301 ^/.+favicon.ico$ http://www.askapache.com/favicon.ico
#RedirectMatch 301 ^/wp-content/uploads/(.*)$ http://z.askapache.com/uploads/$1
#RedirectMatch 301 ^/z/(.+)$ http://z.askapache.com/$1
#RedirectMatch 301 ^/(z|t|i|j|c|p)/(.*)$ http://z.askapache.com/$1/$2
 
#
#  TEMPORARY REDIRECTMATCH
#
RedirectMatch 307 ^/getflash/?$ http://www.adobe.com/shockwave/download/download.cgi?P1_Prod_Version=ShockwaveFlash
RedirectMatch 307 ^/dream/?$ http://www.askapache.com/dreamhost/
RedirectMatch 307 ^/(cse|apachecse|apachecsetest|apachesearch)/?$ http://www.google.com/coop/cse?cx=002660089121042511758%3Akk7rwc2gx0i
RedirectMatch 307 ^/search/lr-lang(.*)$ http://feeds.askapache.com/apache/htaccess
 
#
#  PERMANENT GONE
#
RedirectMatch 410 ^/funny(.*)

My Favorite modsec_v1 stuff

# Pass: Allows request to continue, further filters could still halt request.
# Allow: Allows matching requests through, will not be tested against other filters.
# Deny: Stops the request outright, returns a HTTP 500 error code by default.
# Status: Used to specify an alternate HTTP error code.
# Redirect: Matching requests are redirected to the provided URL.
# Exec: Allows execution of a local system binary or script.
# Log: Logs request only.
# Nolog: Does not log request.
# Chain: Allows you to create list of filters for more granulated security. All filters must be cleared before action is taken with the final filter.
 
SecFilterEngine On
SecFilterCheckURLEncoding On
SecFilterCheckUnicodeEncoding Off
SecFilterScanPOST On
 
#SecUploadKeepFiles On
#SecUploadDir /home/askapache/sites/askapache.com/logs/sec-upload
#SecUploadApproveScript /home/askapache/sites/askapache.com/htdocs/cgi-bin/check.sh
 
#SecRuleEngine On
#SecAuditEngine On
#SecAuditEngine Off
SecAuditEngine RelevantOnly
SecAuditLog /home/askapache/sites/askapache.com/logs/modsec_audit.log
SecFilterDebugLog /home/askapache/sites/askapache.com/logs/modsec_debug.log
SecFilterDebugLevel 1
#SecAuditLogRelevantStatus "^(?:1|2(?!00)|5|4(?!04))"
SecAuditLogRelevantStatus "^(1|2(?!00)|4([0-9](0|1|2|5|6|7|8|9))|5)"
SecFilterForceByteRange 1 255
 
#SecFilterInheritance Off
# 0 EMERGENCY - system is unusable
# 1 ALERT - action must be taken immediately
# 2 CRITICAL - critical conditions
# 3 ERROR - error conditions
# 4 WARNING - warning conditions
# 5 NOTICE - normal but significant conditions
# 6 INFO - informational
# 7 DEBUG - debug-level messages
 
SecFilterDefaultAction "deny,severity:6,status:403"
 
#SecFilterSelective REQUEST_METHOD "POST" "pass,auditlog,severity:6"
SecFilterSelective REQUEST_URI "^/(xmlrpc|wp-comments-post)\.php" "pass,log,auditlog,severity:6"
#SecFilterSelective REMOTE_ADDR ^$ "pass,log,auditlog,severity:6"
#SecFilterSelective REMOTE_ADDR ^203\.221\.91\.20$ "pass,log,auditlog,severity:6"
 
#SecFilterSelective REQUEST_URI "^/htaccess.*" "pass,log,auditlog"
#SecFilterSelective REQUEST_URI "^/feed.*" "pass,log,auditlog"
#SecFilterSelective REQUEST_URI "^/.*trackback.*" "pass,log,auditlog"
#SecFilterSelective REQUEST_URI "^/valid.*" "pass,log,auditlog"
 
#Enforce proper HTTP requests
SecFilterSelective SERVER_PROTOCOL "!^HTTP/(0\.9|1\.0|1\.1)$" "id:340000,rev:1,severity:6,msg:'Bad HTTP Protocol'"
 
# Only accept request encodings we know how to handle
SecFilterSelective REQUEST_METHOD "!^(GET|HEAD|POST)$" "chain,id:340001,rev:1,severity:6,msg:'Restricted HTTP function,status:405'"
SecFilterSelective HTTP_Content-Type "!(^$|^application/x-www-form-urlencoded$|^multipart/form-data)"
 
# Require Content-Length to be provided with every POST request
SecFilterSelective REQUEST_METHOD "^POST$" "chain,id:340003,rev:1,severity:6,msg:'Content Length not provided with POST',status:411"
SecFilterSelective HTTP_Content-Length "^$"
 
# Don't accept transfer encodings we know we don't handle
# (and you don't need it anyway)
SecFilterSelective HTTP_Transfer-Encoding "!^$" "id:340004,rev:1,severity:6,msg:'Dis-allowed Transfer Encoding'"
 
#Generic rule for allowed characters, adjust for your site before activating
##SecFilterSelective REQUEST_URI "!^[a-zA-Z0-9\.\+\_\/\-\?\=\&\%\#]+$" "chain,id:390002,rev:1,severity:4,msg:'Restricted HTTP character set'"
##SecFilterSelective REQUEST_URI "!^/(openid|wp-admin|wp-includes|wp-content|wp-login.php)"
 
#HTTP response splitting generic sigs
#SecFilter "Content-Length\:.*Content-Type\:.*Content-Type\:" "id:340005,rev:1,severity:4,msg:'HTTP response splitting'"
 
#HTTP response splitting generic sigs
#SecFilter "Content-Length\:" "chain,id:340006,rev:1,severity:4,msg:'HTTP response splitting'"
#SecFilter "Content-Type\:"
 
#catch smuggling attacks
#SecFilter "^(GET|POST).*Host:.*^(GET|POST)"  "id:300012,rev:1,severity:4,msg:'catch smuggling attacks'"
 
#XSS insertion into Content-Type
#SecFilterSelective THE_REQUEST "Content-Type\:.*(<[[:space:]]*(script|about|applet|activex|chrome)*>.*(script|about|applet|activex|chrome)[[:space:]]*>|onmouseover=|javascript\:)" "id:300002,rev:1,severity:4,msg:'XSS attack in Content-type header'"
 
#Code injection via content length
#SecFilterSelective HTTP_Content-Length|HTTP_USER_AGENT "\;(system|passthru|exec)\(" "id:330003,rev:1,severity:4,msg:'Code Injection in Content-Length header'"
 
#Don't accept chunked encodings modsecurity can not look at these, so this is a hole that can bypass your rules, the rule before this one should cover this, but hey paranoia is cheap
#SecFilterSelective HTTP_Transfer-Encoding "chunked" "id:300003,rev:1,severity:4,msg:'Chunked Transfer Encoding denied'"
 
##generic recursion signatures
#SecFilterSelective REQUEST_URI "\.\./\.\./" "id:300004,rev:2,severity:4,msg:'Generic Path Recursion1 denied'"
#SecFilterSelective THE_REQUEST "\.\|\./\.\|\./\.\|" "id:300005,rev:1,severity:4,msg:'Generic Path Recursion2 denied'"
#SecFilterSelective THE_REQUEST "\.\.\./" "id:300006,rev:1,severity:4,msg:'Bogus Path denied'"
 
#Generic PHP exploit signatures
#SecFilterSelective REQUEST_URI "<\?php (chr|fwrite|fopen|system|echr|passthru|popen|proc_open|shell_exec|exec|proc_nice|proc_terminate|proc_get_status|proc_close|pfsockopen|leak|apache_child_terminate|posix_kill|posix_mkfifo|posix_setpgid|posix_setsid|posix_setuid|phpinfo)\(.*\)\;" "id:330002,rev:1,severity:4,msg:'Generic PHP exploit pattern denied'"
 
#Prevent SQL injection in cookies
##SecFilterSelective COOKIE_VALUES "((select|grant|delete|insert|drop|alter|replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]+[[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]|UNION SELECT.*\'.*\'.*,[0-9].*INTO.*FROM)" "id:300011,rev:1,severity:4,msg:'Generic SQL injection in cookie'"
 
#Prevent SQL injection in UA
##SecFilterSelective HTTP_USER_AGENT "((select|grant|delete|insert|drop|alter|replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]+[[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]|UNION SELECT.*\'.*\'.*,[0-9].*INTO.*FROM)" "id:300012,rev:1,severity:4,msg:'Generic SQL injection in User Agent header'"
 
# Generic filter to prevent SQL injection attacks
# Understand that all SQL filters are very limited and are very difficult to prevent false postives and negatives.
# Please report false positives/negatives to mike@gotroot.com
#SecFilter "((select|grant|delete|insert|drop|alter|replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]+[[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\*| |\,]|UNION SELECT.*\'.*\'.*,[0-9].*INTO.*FROM)" "id:300013,rev:1,severity:4,msg:'Generic SQL injection protection'"
 
#generic XSS PHP attack types
##SecFilterSelective REQUEST_URI "\.php\?" "chain,id:300010,rev:1,severity:4,msg:'Generic PHP XSS exploit pattern denied'"
#SecFilter "(javascript\:/(.*new\x20ActiveXObject.*Sh\.regwrite|.*window\.opener\.document\.body.\innerHTML=window\.opener\.document\.body\.innerHTML\.replace)|onmouseover=\'javascript)"
 
#Generic XSS filter
#please report false positives
##SecFilterSelective REQUEST_URI "!/mt\.cgi" "chain,msg:'XSS2'"
#SecFilter "<[[:space:]]*(script|about|applet|activex|chrome)*>.*(script|about|applet|activex|chrome)[[:space:]]*>"
 
#XSS in referrer and UA headers
##SecFilterSelective HTTP_REFERER|HTTP_USER_AGENT "<[[:space:]]*(script|about|applet|activex|chrome)*>.*(script|about|applet|activex|chrome)[[:space:]]*>" "msg:'XSS3'"
 
#HTTP header PHP code injection attacks
##SecFilterSelective HTTP_CLIENT_IP|HTTP_USER_AGENT|HTTP_Referer "(<\?php|<[[:space:]]?\?[[:space:]]?php|<\? php)" "msg:'PHP1'"
 
#Generic PHP remote file injection
##SecFilterSelective REQUEST_URI "!(/do_command)" "chain,msg:'PHP2'"
##SecFilterSelective REQUEST_URI "\.php\?.*=(https?|ftp)\:/.*(cmd|command)="
 
#script, perl, etc. code in HTTP_Referer string
##SecFilterSelective HTTP_Referer "\#\!.*/" "msg:'perl script1'"
 
#remote file inclusion generic attack signature
#SecFilterSelective THE_REQUEST  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)\?" "chain,msg:'remote 1'"
#SecFilter "((name|pm_path|pagina|path|include_location|root|page|open)=(http|https|ftp)|(cmd|command|inc)=)"
 
#remote file inclusion generic attack signature
#SecFilterSelective THE_REQUEST  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)\?\&(cmd|command|inc|name)=" "msg:'remote2'"
 
#remote file inclusion generic attack signature
#SecFilterSelective ARGS  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)" "chain,msg:'file inclusion1'"
#SecFilter "\?\&(cmd|inc|name)="
 
#remote file inclusion generic attack signature
#SecFilterSelective ARGS  "\.(dat|gif|jpg|png|bmp|txt|vir|dot)\?\&(cmd|inc|name)=" "msg:'file inclusion2'"
 
#remote file inclusion generic attack signature
#SecFilterSelective REQUEST_URI  "\.php\?.*=(http|https|ftp)\:/.*\?&cmd=" "msg:'file inclusion3'"
 
#Bogus file extensions generic signature
#SecFilterSelective THE_REQUEST  "[A-Za-z0-9]\.(gif|jpg|png|bmp)\.txt" "msg:'file extension'"
 
#PHP remote path attach generic signature
#SecFilterSelective REQUEST_URI  "\.ph(p(3|4)?).*path=(http|https|ftp)\:/" "msg:'remote path1'"
#SecFilterSelective REQUEST_URI  "\.php.*path=(http|https|ftp)\:/" "msg:'remote path2'"
 
#generic php attack sigs
#SecFilterSelective REQUEST_URI "(&(cmd|command)=(id|uname)\x20|cmd\?(cmd|command)=|(spy|cmd|cmd_out|sh)\.(gif|jpg|png|bmp|txt)\?&(cmd|command)=|\.php\?&(cmd|command)=)" "msg:'php attack1'"
 
# WEB-MISC apache directory disclosure attempt
#SecFilterSelective THE_REQUEST "////" "msg:'apache directory disclosure'"
 
#PHP defenses
##SecFilterSelective ARG_PHPSESSID "!^$" "msg:'something in phpsessid'"
##SecFilterSelective COOKIE_PHPSESSID "!^$" "msg:'something in cookie phpsessid'"
 
#PHP defenses
##SecFilterSelective COOKIE_ASKAPACHEID "!^[0-9a-z]*$" "msg:'bad value for cookie'"
 
#cross site scripting attempt TYPE + JAVASCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/javascript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + JAVASCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]application\/x-javascript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + JSCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/jscript" "msg:'cross-site1'"
 
# cross site scripting attempt STYLE + VBSCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/vbscript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + VBSCRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]application\/x-vbscript" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + ECMACRIPT
##SecFilterSelective THE_REQUEST "TYPE\s*=\s*[\'\"]text\/ecmascript" "msg:'cross-site1'"
 
# cross site scripting attempt STYLE + EXPRESSION
##SecFilterSelective THE_REQUEST "STYLE[\s]*=[\s]*[^>]expression[\s]*\(" "msg:'cross-site1'"
 
#cross site scripting attempt STYLE + EXPRESSION
##SecFilterSelective THE_REQUEST "[\s]*expression[\s]*\([^}]}[\s]*<\/STYLE>" "msg:'cross-site1'"
 
# cross site scripting attempt using XML
##SecFilterSelective THE_REQUEST "<!\[CDATA\[<\]\]>SCRIPT" "msg:'cross-site1'"
 
#cross site scripting attempt executing hidden Javascript
##SecFilterSelective THE_REQUEST "eval[\s]*\([\s]*[^\.]\.innerHTML[\s]*\)" "msg:'cross-site1'"
 
#cross site scripting attempt executing hidden Javascript
##SecFilterSelective THE_REQUEST "window\.execScript[\s]*\(" "msg:'cross-site1'"
 
#cross site scripting attempt to execute Javascript code
###SecFilterSelective THE_REQUEST "/(((URL|SRC|HREF|LOWSRC)[\s]*=)|(url[\s]*[\(]))[\s]*[\'\"]*javascript[\:]" "msg:'cross-site1'"
 
#cross site scripting HTML Image tag set to javascript attempt
#SecFilterSelective THE_REQUEST "img src=javascript" "msg:'cross-site1'"
 
#Fake image file shell attacvk
#SecFilterSelective HTTP_Content-Type "image/.*" "msg:'image shell 1'"
#SecFilterSelective POST_PAYLOAD "chr\(" "msg:'image shell2'"
 
#bogus graphics file
#SecFilterSelective HTTP_Content-Disposition "\.php"  "chain,msg:'bogus graphics'"
#SecFilterSelective HTTP_Content-Type "(image/gif|image/jpg|image/png|image/bmp)"
 
# Allow only letters, digits, underscore, and square brackets (for arrays)
# in variable names#
#SecFilterSelective ARGS_NAMES "!^[][a-zA-Z0-9_]+$"
 
#---------------------------------------------
# reject keywords that appear in POST or GET
#=============================================
SecFilterSignatureAction "nolog,auditlog,deny,severity:6,status:403"
 
SecFilterSelective REQUEST_URI  "^/.*/wp-comments-post\.php" "id:50200,msg:'WORDPRESS SPAM bad wp-comments-post location'"
 
<files wp-comments-post.php>
# fail for empty comment fields
SecFilterSelective "ARG_comment_post_ID|ARG_submit" "^$" "id:50300,msg:'WORDPRESS SPAM MISSING comment_post_ID'"
SecFilterSelective "ARG_comment_post_ID" "!^[0-9]{1,6}$" "id:50301,msg:'WORDPRESS SPAM BAD comment_post_ID'"
SecFilterSelective "HTTP_Cookie" "^$" "id:50302,msg:'WORDPRESS SPAM No cookie'"
 
##SecFilterSelective "comment_post_DI" "^$" "id:50310,msg:'WORDPRESS SPAM MISSING comment_post_DI'"
##SecFilterSelective "comment_post_DI" "!^[0-9]{1,2}$" "id:50311,msg:'WORDPRESS SPAM MISSING comment_post_DI'"
</files>
<pre>
 
<h2>MODSEC Continued.. Custom Anti-Spam (WordPress) I made</h2>
<pre>
SecFilterSignatureAction "nolog,noauditlog,deny,severity:6,redirect:http://www.askapache.com/feed/"
SecFilterSelective ARGS|THE_REQUEST "00bp\.com|360\.yahoo|987mb\.com|Ambien|American airline" "id:50010,msg:'SPAM 10'
SecFilterSelective ARGS|THE_REQUEST "Ativan|Caresoprodol|Darvocet|Ephedra|Ephedrine" "id:50011,msg:'SPAM 11'
SecFilterSelective ARGS|THE_REQUEST "Gambling|Lexapro|Tramadol|Venlafaxine" "id:50012,msg:'SPAM 12'
SecFilterSelective ARGS|THE_REQUEST "\[URL=|abgood|acura|acyclovir|adderall" "id:50013,msg:'SPAM 13'
SecFilterSelective ARGS|THE_REQUEST "adipex|alcohol|alprazolam|amateur|amrit" "id:50014,msg:'SPAM 14'
SecFilterSelective ARGS|THE_REQUEST "anal sex|analfinder|angelina jolie|asshole|axspace\.com" "id:50015,msg:'SPAM 15'
SecFilterSelective ARGS|THE_REQUEST "baccarat|bankrupt|bikini|biotic|black jack|blowjob" "id:50016,msg:'SPAM 16'
SecFilterSelective ARGS|THE_REQUEST "blackjack|blog\.360|brutality|buddhism|butalbital" "id:50017,msg:'SPAM 17'
SecFilterSelective ARGS|THE_REQUEST "cadillac|canalis|card credit|card stud|carisoprodol" "id:50018,msg:'SPAM 18'
SecFilterSelective ARGS|THE_REQUEST "carmen|cash advance|cash credit|casino|catch\.com" "id:50019,msg:'SPAM 19'
SecFilterSelective ARGS|THE_REQUEST "celebrex|celexa|cellulite|cheap|cheerleader" "id:50020,msg:'SPAM 20'
SecFilterSelective ARGS|THE_REQUEST "chevrolet|child abuse|cialis|cigarette|cipro" "id:50021,msg:'SPAM 21'
SecFilterSelective ARGS|THE_REQUEST "citroen|claritin|cleavage|clomid|codeine" "id:50022,msg:'SPAM 22'
SecFilterSelective ARGS|THE_REQUEST "consulting23|craps online|credit card|credit debt|crestor" "id:50023,msg:'SPAM 23'
SecFilterSelective ARGS|THE_REQUEST "dealership|debt free|desnudas|diazepam|dick" "id:50024,msg:'SPAM 24'
SecFilterSelective ARGS|THE_REQUEST "dildo|drugstore|earrings|endometrioma|endowment" "id:50025,msg:'SPAM 25'
SecFilterSelective ARGS|THE_REQUEST "erotic|estrogen|fioricet|francaise|freehost\.com" "id:50026,msg:'SPAM 26'
SecFilterSelective ARGS|THE_REQUEST "freehostia|freemb\.com|fuck|geocities\.com|hacking myspace" "id:50027,msg:'SPAM 27'
SecFilterSelective ARGS|THE_REQUEST "holdem|honda|hotels|hydrocodone|hypnotic" "id:50028,msg:'SPAM 28'
SecFilterSelective ARGS|THE_REQUEST "hyundai|implants|incest|instant approval|insurance" "id:50029,msg:'SPAM 29'
SecFilterSelective ARGS|THE_REQUEST "interracial|jaguar|jenny movie|johanson|kasino" "id:50030,msg:'SPAM 30'
SecFilterSelective ARGS|THE_REQUEST "lesbian|levitra|lipitor|loan|lolita" "id:50031,msg:'SPAM 31'
SecFilterSelective ARGS|THE_REQUEST "lorazepam|lorcet|lyrics|madamic|majorette" "id:50032,msg:'SPAM 32'
SecFilterSelective ARGS|THE_REQUEST "malaria|mastercar|masturbate|masturbation|maturewomen" "id:50033,msg:'SPAM 33'
SecFilterSelective ARGS|THE_REQUEST "mazda|medication|medicine|megsfree5\.com|mercedes" "id:50034,msg:'SPAM 34'
SecFilterSelective ARGS|THE_REQUEST "meridia|metformin|mitsubishi|mortgage|myspace profile" "id:50035,msg:'SPAM 35'
SecFilterSelective ARGS|THE_REQUEST "naked|neocool|nexium|nimire\.com|nissan" "id:50036,msg:'SPAM 36'
SecFilterSelective ARGS|THE_REQUEST "nokia|nude|nudism|nymph|open toe" "id:50037,msg:'SPAM 37'
SecFilterSelective ARGS|THE_REQUEST "oprodol|orgasm|oxycodone|oxycontin|packages" "id:50038,msg:'SPAM 38'
SecFilterSelective ARGS|THE_REQUEST "painrelief|pantyhose|paxil|payday|penis" "id:50039,msg:'SPAM 39'
SecFilterSelective ARGS|THE_REQUEST "percocet|pharmacy|phentermine|phetermine|phpbb_root" "id:50040,msg:'SPAM 40'
SecFilterSelective ARGS|THE_REQUEST "pictaboo|pictorial|pills|pissing|play craps" "id:50041,msg:'SPAM 41'
SecFilterSelective ARGS|THE_REQUEST "playgirl|pocker web|poker|pontiac|poquer" "id:50042,msg:'SPAM 42'
SecFilterSelective ARGS|THE_REQUEST "porn|pounder|prescription|preteen|prevacid" "id:50043,msg:'SPAM 43'
SecFilterSelective ARGS|THE_REQUEST "price1|prilosec|propecia|proza|prozac" "id:50044,msg:'SPAM 44'
SecFilterSelective ARGS|THE_REQUEST "puddled|pussy|refinance|rentals|replica" "id:50045,msg:'SPAM 45'
SecFilterSelective ARGS|THE_REQUEST "ringtones|roulette|screensaver|seduced|sexual" "id:50046,msg:'SPAM 46'
SecFilterSelective ARGS|THE_REQUEST "sexy|shemale|shiloh|singulair|site-host" "id:50047,msg:'SPAM 47'
SecFilterSelective ARGS|THE_REQUEST "slot machine|slot maschine|slots machine|solpip\.com|soma" "id:50048,msg:'SPAM 48'
SecFilterSelective ARGS|THE_REQUEST "sperm|starlets|supplier|suzuki|tadalafil" "id:50049,msg:'SPAM 49'
SecFilterSelective ARGS|THE_REQUEST "toyota|tylenol|ultram|valium|viagra" "id:50050,msg:'SPAM 50'
SecFilterSelective ARGS|THE_REQUEST "vigora|vioxx|wallpaper|warez|webcam" "id:50051,msg:'SPAM 51'
SecFilterSelective ARGS|THE_REQUEST "webpages\.com|wellbutrin|whitesluts|wholesale|whore" "id:50052,msg:'SPAM 52'
SecFilterSelective ARGS|THE_REQUEST "windshield|xanax|xenical|y lohan|yourgirls" "id:50053,msg:'SPAM 53'
SecFilterSelective ARGS|THE_REQUEST "youtube\.com|zantac|sex offenders|hotgay|Zoloft|celtic women" "id:50054,msg:'SPAM 54'
SecFilterSelective ARGS|THE_REQUEST "dollhouse|freehot|kardashian|oralsex" "id:50055,msg:'SPAM 55'
SecFilterSelective ARGS|THE_REQUEST "freeimghost" "id:50056,msg:'SPAM 56'

Unreleased AskApache Lightning code - caching plugin for WordPress

RewriteEngine On
RewriteBase /
 
RewriteCond %{REQUEST_URI} ^/(stats/|missing\.html|failed_auth\.html|test/).* [NC,OR]
RewriteCond %{ENV:REDIRECT_STATUS} =200
RewriteRule .* - [L]
 
RewriteCond %{HTTP_HOST} !=www.askapache.com
RewriteCond %{HTTP_HOST} !=z.askapache.com
RewriteRule .? http://www.askapache.com%{REQUEST_URI}%{QUERY_STRING} [R=301,L]
 
#RewriteCond %{HTTPS} =on
#RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /(wp-login.php|wp-admin)(.*)\ HTTP/ [NC]
#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(.*)\ HTTP/ [NC]
#RewriteRule .* http://www.askapache.com/$1 [R=301,L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(.+)%20(.+)\ HTTP/ [NC]
RewriteRule .* http://www.askapache.com/%1-%2 [R=301,L]
 
RewriteCond %{THE_REQUEST} ^GET\ /valid-html/.*\ HTTP/ [NC]
RewriteCond %{HTTP_REFERER} ^https?://www.askapache.com(.+).*$ [NC]
RewriteRule .* http://validator.w3.org/check?uri=http://www.askapache.com%1;ss=1;outline=1;debug [R=307,L,NE]
 
RewriteCond %{THE_REQUEST} ^GET\ /valid-css/.*\ HTTP/ [NC]
RewriteRule .* http://jigsaw.w3.org/css-validator/validator?uri=z.askapache.com/z/c/apache-10.css [R=301,L]
 
RewriteCond %{THE_REQUEST} ^GET\ /search/.*\ HTTP/ [NC]
RewriteCond %{QUERY_STRING} ^s=(.+)$ [NC]
RewriteRule .* http://www.askapache.com/search/%1? [R=302,L]
 
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}/X%{REQUEST_URI}index.html -f
RewriteRule ^(.*)$ /X%{REQUEST_URI}index.html [L]
 
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}/X%{REQUEST_URI} -f
RewriteRule ^(.*)$ /X%{REQUEST_URI} [L]

Passing Parameters to CGI through HTTP Headers

This deserves a note.. this trick lets you bypass so many hosting environment restrictions it's not even funny... like you can run bash as your webserver instead of apache! (kinda...)

RewriteCond %{REQUEST_URI} ^/cgi-bin/(hash|java)\.cgi$ [NC]
RewriteCond %{HTTP:P3P} ^(([^:]+):(.+))$ [NC]
RewriteRule .* - [E=HTTP_JS:%{HTTP:P3P},E=HTTP_US:%2,E=HTTP_PA:%3]

Redirect to FeedBurner

Yes.. I was the first to do this (parse the real rewrites internally in wordpress for no bypassing possibilities)...

#RewriteCond %{REQUEST_URI} ^/feed/.*?$ [NC,OR]
#RewriteCond %{HTTP_REFERER} ^https?://www.askapache.com/wp-admin(.+).*$ [NC]
RewriteCond %{HTTP_USER_AGENT} FeedBurner [NC]
RewriteRule .* - [S=1]
 
RewriteCond %{REQUEST_URI} ^/(feed|rdf|rss|rss2|atom).* [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp-(feed|rdf|rss|rss2|atom).* [NC,OR]
RewriteCond %{REQUEST_FILENAME} ^.*/(wp-atom|wp-feed|wp-rdf|wp-rss|wp-rss2)\.php$ [NC,OR]
RewriteCond %{REQUEST_FILENAME} ^.*/wp-includes/feed[^\.]*\.php$ [NC,OR]
RewriteCond %{QUERY_STRING} .*(feed|rdf|rss|rss2|atom).* [NC]
RewriteRule .* http://feeds.askapache.com/apache/htaccess? [R=302,L]

More Unreleased Caching Tests

# +ASKAPACHE CRAZYCACHE 2.3
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +ACACHE RULES
# +RULE
RewriteCond %{THE_REQUEST} ^(GET|HEAD)\ //?(.+)\.rdf\ HTTP/ [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-cache/%2.html/index.rdf -f
RewriteRule ^(.+)\.rdf$ /wp-cache/%2.html/index.rdf [L,S=3]
 
RewriteCond %{QUERY_STRING} !^.*cacheit.* [NC]
#RewriteCond %{HTTP_USER_AGENT} W3C_Validator [OR,NC]
#RewriteCond %{REMOTE_ADDR} = [OR]
 
RewriteCond %{REQUEST_METHOD} =POST [OR]
RewriteCond %{THE_REQUEST} ^$ [OR]
RewriteCond %{QUERY_STRING} nocache [OR]
RewriteCond %{REMOTE_ADDR} = [OR]
RewriteCond %{HTTP_COOKIE} ^.*wordpress_logged_in_.*$ [NC]
RewriteRule .+ - [S=2]
 
RewriteCond %{DOCUMENT_ROOT}/wp-cache%{REQUEST_URI} !-d
RewriteRule .+ - [S=1]
 
RewriteCond %{DOCUMENT_ROOT}/wp-cache%{REQUEST_URI}/index.html -f
RewriteRule .+ /wp-cache%{REQUEST_URI}/index.html [L]
# -RULE
# -ACACHE RULES
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE CRAZYCACHE 2.3
 
# +ASKAPACHE CRAZYCACHE 2.3
#######################################################
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
# +ACACHE RULES
# +RULE
FileETag None
AddOutputFilterByType DEFLATE text/html
 
AddType 'text/html; charset=UTF-8' .html
 
Header set P3P "policyref=\"http://www.askapache.com/w3c/p3p.xml\""
Header set X-Pingback "http://www.askapache.com/xmlrpc.php"
Header set Content-Language "en-US"
Header set Vary "Accept-Encoding,Accept"
 
<ifModule mod_expires.c>
ExpiresActive On
ExpiresDefault M7200
</ifModule>
# -RULE
# -ACACHE RULES
# - - - - - - - - - - - - - - - - - - - - - - - - - - -
#               __                          __
#   ____ ______/ /______ _____  ____ ______/ /_  ___
#  / __ `/ ___/ //_/ __ `/ __ \/ __ `/ ___/ __ \/ _ \
# / /_/ (__  ) ,< / /_/ / /_/ / /_/ / /__/ / / /  __/
# \__,_/____/_/|_|\__,_/ .___/\__,_/\___/_/ /_/\___/
#                     /_/
#######################################################
# -ASKAPACHE CRAZYCACHE 2.3
 
Options +IndexesOptions +FollowSymLinks
AddHandler application/x-httpd-php .php
 
RewriteEngine On
RewriteBase /
RewriteCond %{QUERY_STRING} !^tyy+$ [NC]
RewriteCond %{REMOTE_USER} ^(.+)$
RewriteRule ^(.*)$ $1/-%1 [R=302,L]
Options +FollowSymLinks
DirectoryIndex index.php
ErrorDocument 404 /cgi-bin/pro/index.php
 
AuthName "Protection"
AuthUserFile /home/askapache/sites/askapache.com/.htpasswda1
AuthGroupFile /dev/null
AuthType Basic
Require valid-user
Satisfy Any
 
<ifModule mod_security.c>
SecFilterEngine Off
</ifModule>
Options +ExecCGI -Indexes -Includes +FollowSymLinks
Order Deny,Allow
Deny from All
Allow from 208.113.134.190 64.111.114.111  env=REDIRECT_STATUS
 
#SetEnvIf Remote_Addr ^$ MODSEC_ENABLE=Off
#SetEnvIf Server_Addr ^$ MODSEC_ENABLE=Off
#SetEnvIf Remote_Addr ^({SERVER_ADDR}e)$ GOOD=$1
#SetEnvIf Server_Addr GOOD R

Default HTACCESS

#
# Directives controlling the display of server-generated directory listings.
#
# Required modules: mod_autoindex, mod_alias
#
# To see the listing of a directory, the Options directive for the
# directory must include "Indexes", and the directory must not contain
# a file matching those listed in the DirectoryIndex directive.
#
Options +Indexes +MultiViews -ExecCGI +FollowSymLinks
DirectoryIndex index.html
 
#
#
# IndexOptions: Controls the appearance of server-generated directory
# listings.
#
IndexOptions FancyIndexing IconHeight=22 IconWidth=20 IgnoreClient NameWidth=* DescriptionWidth=* ScanHTMLTitles SuppressLastModified XHTML FoldersFirst SuppressHTMLPreamble
 
#
# AddIcon* directives tell the server which icon to show for different
# files or filename extensions.  These are only displayed for
# FancyIndexed directories.
#
 
AddIconByEncoding (CMP,http://z.askapache.com/i/s/compressed.gif) x-compress x-gzip
 
AddIconByType (IMG,http://z.askapache.com/i/s/image.gif) image/*
AddIconByType (SND,http://z.askapache.com/i/s/sound2.gif) audio/*
AddIconByType (TXT,http://z.askapache.com/i/s/text.gif) text/*
AddIconByType (VID,http://z.askapache.com/i/s/movie.gif) video/*
 
AddIcon http://z.askapache.com/i/s/hand.right.gif README
AddIcon http://z.askapache.com/i/s/folder.gif ^^DIRECTORY^^
AddIcon http://z.askapache.com/i/s/dir.png ^^DIRECTORY^^
AddIcon http://z.askapache.com/i/s/blank.gif ^^BLANKICON^^
AddDescription "100 Continue" 100* 100/index.html
AddDescription "101 Switching Protocols" 101* 101/index.html
AddDescription "102 Processing" 102* 102/index.html
AddDescription "200 OK" 200* 200/index.html
AddDescription "201 Created" 201* 201/index.html
AddDescription "202 Accepted" 202* 202/index.html
AddDescription "203 Non-Authoritative Information" 203* 203/index.html
AddDescription "204 No Content" 204* 204/index.html
AddDescription "205 Reset Content" 205* 205/index.html
AddDescription "206 Partial Content" 206* 206/index.html
AddDescription "207 Multi-Status" 207* 207/index.html
AddDescription "300 Multiple Choices" 300* 300/index.html
AddDescription "301 Moved Permanently" 301* 301/index.html
AddDescription "302 Found" 302* 302/index.html
AddDescription "303 See Other" 303* 303/index.html
AddDescription "304 Not Modified" 304* 304/index.html
AddDescription "305 Use Proxy" 305* 305/index.html
AddDescription "306 unused" 306* 306/index.html
AddDescription "307 Temporary Redirect" 307* 307/index.html
AddDescription "400 Bad Request" 400* 400/index.html
AddDescription "401 Authorization Required" 401* 401/index.html
AddDescription "402 Payment Required" 402* 402/index.html
AddDescription "403 Forbidden" 403* 403/index.html
AddDescription "404 Not Found" 404* 404/index.html
AddDescription "405 Method Not Allowed" 405* 405/index.html
AddDescription "406 Not Acceptable" 406* 406/index.html
AddDescription "407 Proxy Authentication Required" 407* 407/index.html
AddDescription "408 Request Time-out" 408* 408/index.html
AddDescription "409 Conflict" 409* 409/index.html
AddDescription "410 Gone" 410* 410/index.html
AddDescription "411 Length Required" 411* 411/index.html
AddDescription "412 Precondition Failed" 412* 412/index.html
AddDescription "413 Request Entity Too Large" 413* 413/index.html
AddDescription "414 Request-URI Too Large" 414* 414/index.html
AddDescription "415 Unsupported Media Type" 415* 415/index.html
AddDescription "416 Requested Range Not Satisfiable" 416* 416/index.html
AddDescription "417 Expectation Failed" 417* 417/index.html
AddDescription "418 unused" 418* 418/index.html
AddDescription "419 unused" 419* 419/index.html
AddDescription "420 unused" 420* 420/index.html
AddDescription "421 unused" 421* 421/index.html
AddDescription "422 Unprocessable Entity" 422* 422/index.html
AddDescription "423 Locked" 423* 423/index.html
AddDescription "424 Failed Dependency" 424* 424/index.html
AddDescription "425 No code" 425* 425/index.html
AddDescription "426 Upgrade Required" 426* 426/index.html
AddDescription "500 Internal Server Error" 500* 500/index.html
AddDescription "501 Method Not Implemented" 501* 501/index.html
AddDescription "502 Bad Gateway" 502* 502/index.html
AddDescription "503 Service Temporarily Unavailable" 503* 503/index.html
AddDescription "504 Gateway Time-out" 504* 504/index.html
AddDescription "505 HTTP Version Not Supported" 505* 505/index.html
AddDescription "506 Variant Also Negotiates" 506* 506/index.html
AddDescription "507 Insufficient Storage" 507* 507/index.html
AddDescription "508 unused" 508* 508/index.html
AddDescription "509 unused" 509* 509/index.html
AddDescription "510 Not Extended" 510* 510/index.html
AddDescription ".htaccess ErrorDocuments" *
 
#
# HeaderName is the name of a file which should be prepended to
# directory indexes.
ReadmeName /X/error/README.html
HeaderName /X/error/HEADER.html
 
# DefaultIcon is which icon to show for where none is explicitly set.
DefaultIcon http://z.askapache.com/i/s/generic.gif

Ok I'm done commenting..

#
# IndexIgnore is a set of filenames which directory indexing should ignore
# and not include in the listing.  Shell-style wildcarding is permitted.
#
IndexIgnore .??*  *_notes *~
RewriteEngine On
RewriteCond %{HTTPS} !=on [NC]
RewriteRule .* https://www.askapache.com%{REQUEST_URI}
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},E=REMOTE_USER:%{HTTP:Authorization}]
 
#RewriteCond %{QUERY_STRING} openid.mode=authorize
#RewriteCond %{REQUEST_URI} ^/openid.*$ [NC]
#RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},E=REMOTE_USER:%{HTTP:Authorization}]
Options +FollowSymLinks
DirectoryIndex index.html
 
#SetEnvIfNoCase ^Cookie$ "(.*)" HTTP_MY_COOKIE=$1
#SetEnvIfNoCase Remote_Addr "(.*)" HTTP_MY_REMOTE_ADDR=$1
ExpiresActive Off
FileETag None
Header unset Connection
Header set Connection "close"
Header unset Last-Modified
Header unset ETag
Header unset Accept-Ranges
Header unset Vary
Header unset Content-Type
Header unset X-Pingback
Header unset P3P
#Header add RouterBits "%D %t"
#Header add Location "http://www.askapache.com/feed/"
#Header add Found "http://askapache.com/feed/"
#Header add Content-Location "http://www.askapache.com/feed/"
#Header add Refresh "http://www.askapache.com/feed/"
#Header set Hi "%{HTTP_MY_REMOTE_ADDR}e"
### Section 1: Global Environment
#
# The directives in this section affect the overall operation of Apache,
# such as the number of concurrent requests it can handle or where it
# can find its configuration files.
#
#
# TZ: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv TZ America/Indianapolis
 
#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
SetEnv SERVER_ADMIN webmaster@askapache.com
 
#
# Possible values for the Options directive are "None", "All",
# or any combination of:
#   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
#
# The Options directive is both complicated and important.  Please see
# http://httpd.apache.org/docs/2.0/mod/core.html#options
# for more information.
#
Options  +FollowSymLinks -ExecCGI -Indexes -Includes -MultiViews
 
#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
DirectoryIndex index.html index.php /index.php
 
#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<filesMatch "^\.ht">
    Order allow,deny
    Deny from all
  Satisfy All
</filesMatch>
 
#
# DefaultType: the default MIME type the server will use for a document
# if it cannot otherwise determine one, such as from filename extensions.
# If your server contains mostly text or HTML documents, "text/plain" is
# a good value.  If most of your content is binary, such as applications
# or images, you may want to use "application/octet-stream" instead to
# keep browsers from trying to display binary files as though they are
# text.
#
DefaultType text/html
 
#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature Off
 
##############################################
#           HEADERS and CACHING              #
##############################################
Header unset Pragma
Header unset Last-Modified
FileETag None
 
Redirect 400 /e/400
Redirect 401 /e/401
Redirect 402 /e/402
Redirect 403 /e/403
Redirect 404 /e/404
Redirect 405 /e/405
Redirect 406 /e/406
Redirect 407 /e/407
Redirect 408 /e/408
Redirect 409 /e/409
Redirect 410 /e/410
Redirect 411 /e/411
Redirect 412 /e/412
Redirect 413 /e/413
Redirect 414 /e/414
Redirect 415 /e/415
Redirect 416 /e/416
Redirect 417 /e/417
Redirect 418 /e/418
Redirect 419 /e/419
Redirect 420 /e/420
Redirect 421 /e/421
Redirect 422 /e/422
Redirect 423 /e/423
Redirect 424 /e/424
Redirect 425 /e/425
Redirect 426 /e/426
Redirect 500 /e/500
Redirect 501 /e/501
Redirect 502 /e/502
Redirect 503 /e/503
Redirect 504 /e/504
Redirect 505 /e/505
Redirect 506 /e/506
Redirect 507 /e/507
Redirect 508 /e/508
Redirect 509 /e/509
Redirect 510 /e/510
 
#
# Redirect allows you to tell clients about documents which used to exist in
# your server's namespace, but do not anymore. This allows you to tell the
# clients where to look for the relocated document.
##############################################
#          PERMANENT REDIRECTS               #
##############################################
Redirect 301 /12-lessons-for-those-afraid-of-css.html http://www.askapache.com/css/12-lessons-for-those-afraid-of-css.html
Redirect 301 /2006/htaccess/htaccesselite-ultimate-htaccess-article.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /2007/phpbb/sending-post-form-data-with-php-curl.html http://www.askapache.com/htaccess/sending-post-form-data-with-php-curl.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/custom-phpini-with-fastcgi-on-dreamhost.html http://www.askapache.com/dreamhost/custom-phpini-with-fastcgi-on-dreamhost.html
Redirect 301 /2007/webmaster/php-and-ajax-shell-console.html http://www.askapache.com/tools/php-and-ajax-shell-console.html
Redirect 301 /27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html http://www.askapache.com/htaccess/27-request-methods-for-use-with-apache-and-rewritecond-and-htaccess.html
Redirect 301 /404-google-wordpress-plugin.html http://www.askapache.com/seo/404-google-wordpress-plugin.html
Redirect 301 /503-service-temporarily-unavailable.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /Overview-about.rdf http://www.askapache.com/askapache-home.rdf
Redirect 301 /abbr-acronym.html http://www.askapache.com/xhtml/abbr-acronym.html
Redirect 301 /adsense-robots.html http://www.askapache.com/google/adsense-robots.html
Redirect 301 /alexa-toolbar-firefox.html http://www.askapache.com/tools/alexa-toolbar-firefox.html
Redirect 301 /allowing-access-from-1-static-ip-and-deny-the-rest.html http://www.askapache.com/htaccess/apache-htaccess.html
Redirect 301 /anti-virus-spyware-rootkit.html http://www.askapache.com/security/anti-virus-spyware-rootkit.html
Redirect 301 /apache-ssl-in-htaccess-examples.html http://www.askapache.com/htaccess/apache-ssl-in-htaccess-examples.html
Redirect 301 /awk-tutorial.html http://www.askapache.com/awk/awk-tutorial.html
Redirect 301 /best-adsense-optimization.html http://www.askapache.com/wordpress/best-adsense-optimization.html
Redirect 301 /commonly-used-htaccess-code-examples.html http://www.askapache.com/htaccess/commonly-used-htaccess-code-examples.html
Redirect 301 /css-background-image-sprite.html http://www.askapache.com/css/css-background-image-sprite.html
Redirect 301 /css-browser-screenshots.html http://www.askapache.com/tools/css-browser-screenshots.html
Redirect 301 /css-class-example.html http://www.askapache.com/css/css-class-example.html
Redirect 301 /curl-multi-downloads.html http://www.askapache.com/php/curl-multi-downloads.html
Redirect 301 /custom-boot-menu-in-windows-xp.html http://www.askapache.com/windows/custom-boot-menu-in-windows-xp.html
Redirect 301 /donate http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /donate/ http://www.dreamhost.com/donate.cgi?id=8261
Redirect 301 /htaccess.txt http://z.askapache.com/p/htaccess.txt
Redirect 301 /htaccess/404-errorpages.html http://www.askapache.com/seo/google-ajax-search-seo-tips.html
Redirect 301 /htaccess/feedsmith http://www.askapache.com/htaccess/feedsmith-htaccess.html
Redirect 301 /htaccess/http-status-codes.html http://www.askapache.com/htaccess/apache-status-code-headers-errordocument.html
Redirect 301 /htaccess/instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /htaccess/speed-up-the-apache-web-server-with-configuration-hacks.html http://www.askapache.com/htaccess/apache-web-server-speed-configuration-hacks.html
Redirect 301 /instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html
Redirect 301 /security/bypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/bypassing-vlanbypassing-vlan.html http://www.askapache.com/security/hacking-vlan-switched-networks.html
Redirect 301 /security/rigging-the-dreamhost-site-of-the-month-contest.html http://www.askapache.com/dreamhost/rigging-the-dreamhost-site-of-the-month-contest.html
Redirect 301 /seo/tailrankcom-robot.html http://www.askapache.com/seo/tailrank-robot.html
Redirect 301 /webmaster/caching-tutorial-for-webmasters.html http://www.askapache.com/htaccess/caching-tutorial-for-webmasters.html
Redirect 301 /webmaster/lft-traceroute-tool.html http://www.askapache.com/tools/lft-traceroute-tool.html
 
##############################################
#          PERMANENT REDIRECTMATCH           #
##############################################
#RedirectMatch 301 ^/&(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/&amp(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/([^/]+)//$ http://www.askapache.com/$1/
RedirectMatch 301 ^/(.+)/htaccesselite-ultimate-htaccess-article.html(.*) http://www.askapache.com/htaccess/apache-htaccess.html
RedirectMatch 301 ^/(.+)\.html/([a-z][a-z])/$ http://www.askapache.com/$1.html
RedirectMatch 301 ^/([\(]+)(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/([^9]*)9O1X.3y(.*)/(.*)$ http://www.askapache.com/$2
RedirectMatch 301 ^/.3y(.*)$ http://www.askapache.com/
RedirectMatch 301 ^/200([0-9])/([0-9])(.*)$ http://www.askapache.com/top-100/
RedirectMatch 301 ^/200([0-9])/([^01])(.*)$ http://www.askapache.com/$2$3
RedirectMatch 301 ^/about/glossary(.*)$ http://www.askapache.com/glossary$1
RedirectMatch 301 ^/apache-speed(.*)$ http://www.askapache.com/htaccess/apache-speed$1
RedirectMatch 301 ^/category/(.+)$ http://www.askapache.com/$1
RedirectMatch 301 ^/docs/(.*)$ http://askapache.info/$1
RedirectMatch 301 ^/htaccess/feedsmith-htaccess(.*) http://www.askapache.com/htaccess/redirecting-wordpress-feeds-to-feedburner.html
RedirectMatch 301 ^/robots-txt(.*)$ http://www.askapache.com/robots.txt
#RedirectMatch 301 ^/wp-content/uploads/(.*)$ http://z.askapache.com/uploads/$1
RedirectMatch 301 ^/z/(.+)$ http://z.askapache.com/$1
RedirectMatch 301 ^/(z|t|i|j|c|p)/(.*)$ http://z.askapache.com/$1/$2
RedirectMatch 301 ^/hosting/?$ http://www.askapache.com/hosting/
 
##############################################
#          TEMPORARY REDIRECTMATCH           #
##############################################
RedirectMatch 307 ^/getflash/?$ http://www.adobe.com/shockwave/download/download.cgi?P1_Prod_Version=ShockwaveFlash
RedirectMatch 307 ^/dream/?$ http://www.askapache.com/dreamhost/
RedirectMatch 307 ^/(cse|apachecse|apachecsetest|apachesearch)/?$ http://www.google.com/coop/cse?cx=002660089121042511758%3Akk7rwc2gx0i
 
#
# Commonly used filename extensions to character sets. You probably
# want to avoid clashes with the language extensions, unless you
# are good at carefully testing your setup after each change.
# See http://www.iana.org/assignments/character-sets for the
# official list of charset names and their respective RFCs.
#
AddDefaultCharset UTF-8
 
#
# AddType allows you to add to or override the MIME configuration
# file mime.types for specific file types.
#
#
AddType 'application/rdf+xml; charset=UTF-8' .rdf
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml.gz
AddType 'text/html; charset=UTF-8' .html
AddType 'text/html; charset=UTF-8' .html.gz
AddType application/octet-stream .rar .chm .bz2 .tgz .msi .pdf .exe
AddType application/vnd.ms-excel .csv
AddType application/x-httpd-php-source .phps
AddType application/x-pilot .prc .pdb
AddType application/x-shockwave-flash .swf
AddType application/xrds+xml .xrdf
AddType text/plain .ini .sh .bsh .bash .awk .nawk .gawk .csh .var .c .in .h .asc .md5 .sha .sha1
AddType video/x-flv .flv
 
#
# AddEncoding allows you to have certain browsers uncompress
# information on the fly. Note: Not all browsers support this.
# Despite the name similarity, the following Add* directives have nothing
# to do with the FancyIndexing customization directives above.
#
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
 
#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
AddHandler php-cgi .php
 
#
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#
Action php-cgi /cgi-bin/php.cgi
 
#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
#100 Continue
#101 Switching Protocols
#102 Processing
#200 OK
#201 Created
#202 Accepted
#203 Non-Authoritative Information
#204 No Content
#205 Reset Content
#206 Partial Content
#207 Multi-Status
#300 Multiple Choices
#301 Moved Permanently
#302 Found
#303 See Other
#304 Not Modified
#305 Use Proxy
#306 unused
#307 Temporary Redirect
#400 Bad Request
#401 Authorization Required
#402 Payment Required
#403 Forbidden
#404 Not Found
#405 Method Not Allowed
#406 Not Acceptable
#407 Proxy Authentication Required
#408 Request Time-out
#409 Conflict
#410 Gone
#411 Length Required
#412 Precondition Failed
#413 Request Entity Too Large
#414 Request-URI Too Large
#415 Unsupported Media Type
#416 Requested Range Not Satisfiable
#417 Expectation Failed
#418 unused
#419 unused
#420 unused
#421 unused
#422 Unprocessable Entity
#423 Locked
#424 Failed Dependency
#425 No code
#426 Upgrade Required
#500 Internal Server Error
#501 Method Not Implemented
#502 Bad Gateway
#503 Service Temporarily Unavailable
#504 Gateway Time-out
#505 HTTP Version Not Supported
#506 Variant Also Negotiates
#507 Insufficient Storage
#508 unused
#509 unused
#510 Not Extended
 
#ErrorDocument 100 /X/err/1/HTTP_CONTINUE.html
#ErrorDocument 101 /X/err/1/HTTP_SWITCHING_PROTOCOLS.html
#ErrorDocument 102 /X/err/1/HTTP_PROCESSING.html
 
#ErrorDocument 200 /X/err/2/HTTP_OK.html
#ErrorDocument 201 /X/err/2/HTTP_CREATED.html
#ErrorDocument 202 /X/err/2/HTTP_ACCEPTED.html
#ErrorDocument 203 /X/err/2/HTTP_NON_AUTHORITATIVE.html
#ErrorDocument 204 /X/err/2/HTTP_NO_CONTENT.html
#ErrorDocument 205 /X/err/2/HTTP_RESET_CONTENT.html
#ErrorDocument 206 /X/err/2/HTTP_PARTIAL_CONTENT.html
#ErrorDocument 207 /X/err/2/HTTP_MULTI_STATUS.html
 
#ErrorDocument 300 /X/err/HTTP_MULTIPLE_CHOICES.html
#ErrorDocument 301 /X/err/HTTP_MOVED_PERMANENTLY.html
#ErrorDocument 302 /X/err/HTTP_MOVED_TEMPORARILY.html
#ErrorDocument 303 /X/err/HTTP_SEE_OTHER.html
#ErrorDocument 304 /X/err/HTTP_NOT_MODIFIED.html
#ErrorDocument 305 /X/err/HTTP_USE_PROXY.html
#ErrorDocument 307 /X/err/HTTP_TEMPORARY_REDIRECT.html
#ErrorDocument 404 /X/err/HTTP_NOT_FOUND.html
 
ErrorDocument 400 /X/err/4/HTTP_BAD_REQUEST.html
ErrorDocument 401 /X/err/4/HTTP_UNAUTHORIZED.html
ErrorDocument 402 /X/err/4/HTTP_PAYMENT_REQUIRED.html
ErrorDocument 403 /X/err/4/HTTP_FORBIDDEN.html
ErrorDocument 405 /X/err/4/HTTP_METHOD_NOT_ALLOWED.html
ErrorDocument 406 /X/err/4/HTTP_NOT_ACCEPTABLE.html
ErrorDocument 407 /X/err/4/HTTP_PROXY_AUTHENTICATION_REQUIRED.html
ErrorDocument 408 /X/err/4/HTTP_REQUEST_TIME_OUT.html
ErrorDocument 409 /X/err/4/HTTP_CONFLICT.html
ErrorDocument 410 /X/err/4/HTTP_GONE.html
ErrorDocument 411 /X/err/4/HTTP_LENGTH_REQUIRED.html
ErrorDocument 412 /X/err/4/HTTP_PRECONDITION_FAILED.html
ErrorDocument 413 /X/err/4/HTTP_REQUEST_ENTITY_TOO_LARGE.html
ErrorDocument 414 /X/err/4/HTTP_REQUEST_URI_TOO_LARGE.html
ErrorDocument 415 /X/err/4/TTP_UNSUPPORTED_MEDIA_TYPE.html
ErrorDocument 416 /X/err/4/HTTP_RANGE_NOT_SATISFIABLE.html
ErrorDocument 417 /X/err/4/HTTP_EXPECTATION_FAILED.html
ErrorDocument 422 /X/err/4/HTTP_UNPROCESSABLE_ENTITY.html
ErrorDocument 423 /X/err/4/HTTP_LOCKED.html
ErrorDocument 424 /X/err/4/HTTP_FAILED_DEPENDENCY.html
ErrorDocument 426 /X/err/4/HTTP_UPGRADE_REQUIRED.html
 
ErrorDocument 500 /X/err/5/HTTP_INTERNAL_SERVER_ERROR.html
ErrorDocument 501 /X/err/5/HTTP_NOT_IMPLEMENTED.html
ErrorDocument 502 /X/err/5/HTTP_BAD_GATEWAY.html
ErrorDocument 503 /X/err/5/HTTP_SERVICE_UNAVAILABLE.html
ErrorDocument 504 /X/err/5/HTTP_GATEWAY_TIME_OUT.html
ErrorDocument 505 /X/err/5/HTTP_VERSION_NOT_SUPPORTED.html
ErrorDocument 506 /X/err/5/HTTP_VARIANT_ALSO_VARIES.html
ErrorDocument 507 /X/err/5/HTTP_INSUFFICIENT_STORAGE.html
ErrorDocument 510 /X/err/5/HTTP_NOT_EXTENDED.html
ErrorDocument 404 /index.php?error=404
 
# 1 YEAR
<filesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
Header unset P3P
Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
</filesMatch>
 
# 2 HOURS
<filesMatch "\.(html|htm|xml|txt|xsl)$">
<ifModule mod_expires.c>
ExpiresActive On
ExpiresDefault A3600
</ifModule>
</filesMatch>
 
RewriteEngine On
RewriteBase /
 
RewriteCond %{ENV:REDIRECT_STATUS} =200
RewriteRule .* - [L]
 
RewriteCond %{REQUEST_METHOD} !^(GET|HEAD) [OR]
#RewriteCond %{QUERY_STRING} !^$ [OR]
RewriteCond %{HTTP_COOKIE} ^.*(comment_author_|wordpress|wp-postpass_).*$ [NC]
RewriteRule ^(.*)$ - [S=6]
 
RewriteCond %{HTTP_USER_AGENT} .*W3C_Validator.* [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml.gz -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml.gz [L,S=5]
 
RewriteCond %{HTTP:Accept} application/xhtml\+xml [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml.gz -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml.gz [L,S=4]
 
RewriteCond %{HTTP:Accept-Encoding} gzip [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.html.gz -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.html.gz [L,S=3]
 
RewriteCond %{HTTP_USER_AGENT} .*W3C_Validator.* [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml [L,S=2]
 
RewriteCond %{HTTP:Accept} application/xhtml\+xml [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.xhtml -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.xhtml [L,S=1]
 
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/aa/$1/index.html -f
RewriteRule ^(.*)$ /wp-content/cache/aa/$1/index.html [L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(.+)\.rdf\ HTTP/ [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/rdf/%1.html/index.rdf -f
RewriteRule ^(.*)$ /wp-content/rdf/%1.html/index.rdf [L,S=1]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(error|w3c|openid)(/?.*)\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /X/%1%2 [L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(y_key_bf61afd498f7623a\.html|google3bb7b7e1032ad0d4\.html|yadis\.xrdf|askapache-home\.rdf|os-description\.xml|labels\.rdf|gnu-fdl\.txt|wlmmanifest\.xml|robots\.txt)\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .* /X/%1 [L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /X(.*)\ HTTP/ [NC]
RewriteRule .* - [F]
 
RewriteRule ^osq?(.*)$ /wp-content/plugins/wp-opensearch.php?$1 [QSA,L]
 
#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-login(.+)\ HTTP/ [NC]
#RewriteRule .+ https://www.askapache.com/wp-login%1 [R,L]
 
RewriteCond %{REQUEST_URI} ^/cgi-bin/(hash|java)\.cgi$ [NC]
RewriteCond %{HTTP:P3P} ^(([^:]+):(.+))$ [NC]
RewriteRule .* - [E=HTTP_JS:%{HTTP:P3P},E=HTTP_US:%2,E=HTTP_PA:%3]
 
RewriteCond %{HTTP_ACCEPT} application/xrds\+xml
RewriteCond %{HTTP_ACCEPT} !application/xrds\+xml\s*;\s*q\s*=\s*0(\.0{1,3})?\s*(,|$)
RewriteRule ^$ http://www.askapache.com/yadis.xrdf [R,L]
 
RewriteCond %{HTTP_USER_AGENT} !^(FeedBurner|FeedValidator|talkr.com).* [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(feed|rdf|rss|rss2|atom)/?.*\ HTTP/ [NC]
#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(feed|rdf|rss|rss2|atom|wp-atom|wp-feed|wp-rdf|wp-rss|wp-rss2).*\ HTTP/ [NC]
RewriteRule .* http://feeds.askapache.com/apache/htaccess? [R=302,L]
 
#RewriteCond %{REMOTE_ADDR} ^$
#RewriteCond %{REQUEST_FILENAME} !-f
#RewriteCond %{REQUEST_FILENAME} !-d
#RewriteRule ^(.+)\.phps$ /cgi-bin/phps.php?file=$1.php [L,NC]

Actual Htaccess Files from My Server originally appeared on AskApache.com

Looking for some advanced uses for the shell? Here is some of my best. Works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.

Want to get started? Type the following in your shell to download the script and run it. You should go over the code first and then you can just save it as your .bash_profile - whenever you want to get updated to the latest version just type aaup, a cool update function that auto dos2unix's and runs via an exec command.

curl -O http://static.askapache.com/askapache-bash-profile.txt && source askapache-bash-profile.txt
Or if you trust me and my hosting provider (and you aren't on production)
curl -o ~/.bash_profile http://static.askapache.com/askapache-bash-profile.txt && exec bash -l

For those of you power users and server admins that use Bash, ksh, csh, vanilla sh, etc.., or if you are just passionate about shell scripting, because it allows you to get advanced tasks done fast and efficiently, not to mention automated automatically. I give you my .bash_profile file. You should edit it to fit your needs, (especially the exported vars like PATH, LDFLAGS, if you don't understand something just comment it out) but it's pretty universal because I work on alot of other people's servers not to mention many different distros and platforms, and when I get hired to do some server work through a shell, I bring this script along for the ride.

askapache-bash-profile.txt

You can download the latest version: http://static.askapache.com/askapache-bash-profile.txt

The functions and variables below are the way bash sees them, using declare -f, and alias, to make it easier for you to read and understand them. The actual file at http://static.askapache.com/askapache-bash-profile.txt will always be the most updated version, as I use it personally. And it has the whole file the way I wrote it, meaning many extra notes and much simpler to follow. Enjoy!

Gist of the Script..

I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:

#!/bin/bash
 
source ~/.bash_profile &>/dev/nulll
 
pm "PM is a function to output nice messages with color"
yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS"
yn "Show Calendar" && aa_calendar
yn "Show Fortune" && aa_fortune

alias chmod='command chmod -c'
alias cpr='command cp -rpv'
alias df='command df -kTh'
alias df1='command df -iTa'
alias diff='diff -up'
alias dsiz='du -sk * | sort -n --'
alias du='command du -kh'
alias du1='echo *|tr " " "\n" |xargs -iFF command du -hs FF|sort'
alias env='command env | sort'
alias h='history'
alias inice='ionice -c3 -n7 nice'
alias j='jobs -l'
alias la='command ls -Al --color=auto'
alias lc='command ls -lAcr --color=auto'
alias less='vless'
alias lessc='ccze -A |`type -P less` -R'
alias lk='command ls -lASr --color=auto'
alias llh='ll -h'
alias lll='stat -c %a\ %N\ %G\ %U ${PWD}/*|sort'
alias lr='command ls -lAR --color=auto'
alias lt='command ls -lAtr --color=auto'
alias lu='command ls -lAur --color=auto'
alias lx='command ls -lAXB --color=auto'
alias mann='command man -H'
alias n='/usr/bin/nano3'
alias p='command ps -HAcl -F S -A f|uniq -w3'
alias path='echo -e ${PATH//:/\\n}'
alias php='php -d report_memleaks=1 -d report_zend_debug=1 -d log_errors=0 -d ignore_repeated_errors=0 -d ignore_repeated_source=0 -d error_reporting=30719 -d display_startup_errors=1 -d display_errors=1'
alias pp='command ps -HAcl -F S -A f'
alias ps1='command ps -lFA'
alias ps2='command ps -H'
alias resetw='echo $'\''\33[H\33[2J'\'''
alias subash='sudo sh -c '\''export HOME=/root; cd /root; exec bash -l'\'''
alias top='top -c'
alias tree='command tree -Csuflapi'
alias updatedb='( ( updatedb 2>/dev/null ) & )'
alias vim='command vim --noplugin'
alias who='command who -ar -pld'
alias wtf='watch -n 1 w -hs'

Custom bash_profile for Advanced Shell Users originally appeared on AskApache.com

This bash shell script searches in a directory you specify for any files containing a string and replaces that string with another string. It's nice because it asks you if you would like to be prompted for each operation, or to run without user-input. I use this shell script to change static IP addresses or domains, file locations, database info, etc.

Note: I'm going to be updating this soon..

For now here is a one-liner for you that works very fast and is also low-resource-intensive!

nice -n19 sh -c 'S=askapache && R=htaccess; find . -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e s%${S}%${R}% FF'

ScreenShot

Shell Script Code

You can also download it: FIND AND REPLACE SCRIPT Version 1.4

#!/bin/bash
# Find and Replace Version 2, 2009-04-19
# Version 1.4
# GNU Free Documentation License 1.2
# 2009-04-19 - AskApache (www.askapache.com)
umask 0022
 
# set -o xtrace && set -o verbose #uncomment for verbose debugging
 
# nicify to avoid getting killed
renice 19 -p $$ &>/dev/null
 
### SHELL OPTIONS
set +o noclobber  # allowed to clobber files
set +o noglob     # globbing on
set -e            # abort on first error
shopt -s extglob  # more globbing
 
PATH=/bin:/sbin:/usr/bin:/usr/local/bin:/usr/libexec:/usr/sbin
VER="1.4"
COLORON=yes
 
SEARCH=${1:-QQQQQQQQ};
REPLACE=${2:-AskApache};
SDIR=${3:-/tmp};
 
# script_title
#==-==-==-==-==-==-==-==-==-==-==#
function script_title(){
  clear;echo -e "${C[1]} __________________________________________________________________________ "
  echo -e "| ${C[2]}             ___       __    ___                 __             ${C[1]}         |"
  echo -e "| ${C[2]}            / _ | ___ / /__ / _ | ___  ___ _____/ /  ___        ${C[1]}         |"
  echo -e "| ${C[2]}           / __ |(_-</  '_// __ |/ _ \/ _ \`/ __/ _ \/ -_)       ${C[1]}         |"
  echo -e "| ${C[3]}          /_/ |_/___/_/\_\/_/ |_/ .__/\_,_/\__/_//_/\__/        ${C[1]}         |"
  echo -e "| ${C[3]}                               /_/                              ${C[1]}         |"
  echo -e "|                                                                          |"
  echo -e "|                 ${C[4]} FIND AND REPLACE SCRIPT Version $VER ${C[1]}                    |"
  echo -e "${C[1]} __________________________________________________________________________ ${C[0]} \n\n"
}
 
# pm
#==-==-==-==-==-==-==-==-==-==-==#
function pm()
{ case ${2:-d} in d) echo -en "\n\n${C[2]}>>> ${C[4]}$1 ${C[0]}\n\n"; ;; i*) echo -e "\n${C[6]}=> ${C[4]}${1} ${C[0]}\n"; ;; esac; return 0; }
 
# ok_continue
#==-==-==-==-==-==-==-==-==-==-==#
function ok_continue()
{ local ans; echo -en "${C[4]} \n [ Press any key to continue ] ${C[0]} \n"; read -n 1 ans; return 0; }
 
# yes_no
#==-==-==-==-==-==-==-==-==-==-==#
function yes_no()
{ local a YN=65; echo -en "${C[2]}>>> ${C[4]}${1:-Q} [y/n] ${C[0]}"; read -n 1 a; case $a in [yY]) YN=0; ;; esac; return $YN; }
 
# p_done
#==-==-==-==-==-==-==-==-==-==-==#
function p_done()
{ echo -e "\n${C[8]} DONE ${C[0]} \n"; return 0; }
 
# get_settings
#==-==-==-==-==-==-==-==-==-==-==#
function get_settings(){
  local a cha
  script_title
  while true; do
  for a in "SDIR" "SEARCH" "REPLACE"; do
   cha=g; echo -en "\n ${C[4]}(Enter for Default: ${!a} )${C[0]}\n ${a}${C[6]}=> ${C[0]} ";
   read -e cha; echo; [[ ${#cha} -gt 2 ]] && eval "$a"=$cha;
  done
  yes_no "ARE THESE SETTINGS CORRECT" && echo -e "\n\n" && break
  done
}
 
# search_and_replace
#==-==-==-==-==-==-==-==-==-==-==#
function search_and_replace(){
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do
  thefile=${FOUNDFILES[$i]}
 
  if [[ $1 -eq 0 ]]; then
   echo -e "\n\n\n___________________________________________________________________\n"
   echo -e "${C[4]}FILE:  ${C[3]}${thefile} ${C[2]}($(command du -hs ${thefile}|awk '{ print $1}'))"
   echo -e "${C[4]}Type:  ${C[6]}$(command file -b ${thefile})\n${C[7]}Matching Lines:${C[0]}"
   command grep --color=always $SEARCH $thefile;echo
 
   while true; do
    yes_no "Replace occurances of ${SEARCH} with ${REPLACE}?" && echo -e "\nREPLACING...\n" && sed -i -e "s%${SEARCH}%${REPLACE}%g" $thefile && p_done && break
    echo -e "\nSKIPPING..\n" && p_done && break
   done
 
   else
    sed -i -e "s%${SEARCH}%${REPLACE}%g" $thefile
   fi
 
  done
  return 0;
}
 
############################################################################################################
### MAIN CODE
############################################################################################################
 
if [[ $SEARCH == "QQQQQQQQ" ]]; then
  E='\033[';
  [[ -z "$TERM" ]]&&[[ -z "$1" ]]||declare -a C=('${E}0m' '${E}1\;30m' '${E}0\;32m' '${E}1\;32m' '${E}1\;37m' '${E}1\;36m' '${E}1\;35m' '${E}1\;37m' '${E}30\;42m' '${E}1\;36m' )
  script_title            # DISPLAY SCRIPT TITLE
 
  get_settings            # GET SCRIPT SETTINGS
  cd $SDIR
  declare -a FOUNDFILES=( `command grep -R -l $SEARCH . 2>/dev/null` );
  declare -a FOUNDMATCHES=(  );
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do F=${FOUNDFILES[$i]}; FOUNDMATCHES[$i]=$(command grep -c . $F); done
  pm "FOUND ${SEARCH} IN ${#FOUNDFILES[@]} FILES" i
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do echo -e "${C[4]}${FOUNDFILES[$i]} ${C[6]}=>${C[4]} ${FOUNDMATCHES[$i]} matches"; done
  echo -e "\n"
 
 while true; do
  yes_no "Replace all occurances of ${SEARCH} with ${REPLACE} without prompting?" && search_and_replace 1 && p_done && break
  search_and_replace 0 && p_done && break
 done
 
else
 cd $SDIR
 command grep -H -R -l $SEARCH . 2>/dev/null | xargs -iFFF sed -i -e "s%${SEARCH}%${REPLACE}%g" FFF
fi
 
exit 0

Simplified for crontab

*Mike*
To run from a crontab you wouldn't want a full-fledged script like this, I would just create a small bash script like so:

#!/bin/sh
PATH=/usr/bin:$PATH:/usr/local/bin
command grep -H -R -l $1 $3 2>/dev/null | xargs -iFFF sed -i -e "s%${1}%${$2}%g" FFF

This is like 100x faster, to call from crontab every 35 minutes and search the directory /home/user/tmp for 'search' and replace it with 'replace' every 35 minutes.

*/35 * * * * /bin/sh /home/user/search-replace.sh 'search' 'replace' '/home/user/tmp' 1>/dev/null

Search And Replace shell script helpful for Upgrades originally appeared on AskApache.com

While researching a unix/linux tool awk I came upon one of the most thorough and helpful tutorials I've ever seen devoted to a particular topic. It's old-school just the way I like it. I contacted the author, Bruce Barnett because I just HAD to have this article for my readers, who are predominantly running solaris/unix/bsd/linux and he kindly gave permission. So, without further ado, I present: Awk - A Tutorial and Introduction - by Bruce Barnett

Copyright 2001,2004 Bruce Barnett and General Electric Company All rights reserved. You are allowed to print copies of this tutorial for your personal use, and link to this page, but you are not allowed to make electronic copies, or redistribute this tutorial in any form without permission. Last update: Tue Mar 9 11:07:08 EST 2004

Awk is an extremely versatile programming language for working on files. We'll teach you just enough to understand the examples in this book, plus a smidgen.

Table of Contents

• Why learn AWK?
• Basic Structure
• Executing an AWK script
• Which shell to use with AWK?
• Dynamic Variables
• The Essential Syntax of AWK
• Arithmetic Expressions
• Unary arithmetic operators
• The Autoincrement and Autodecrement Operators
• Assignment Operators
• Conditional expressions
• Regular Expressions
• And/Or/Not
• Commands
• AWK Built-in Variables
• FS - The Input Field Separator Variable
• OFS - The Output Field Separator Variable
• NF - The Number of Fields Variable
• NR - The Number of Records Variable
• RS - The Record Separator Variable
• ORS - The Record Separator Variable
• FILENAME - The Current Filename Variable"
• Associative Arrays
• Multi-dimensional Arrays
• Example of using AWK's Associative Arrays
• Output of the script
• Picture Perfect PRINTF Output
• PRINTF - formatting output
• Escape Sequences
• Format Specifiers
• Width - specifying minimum field size
• Left Justification
• The Field Precision Value
• Explicit File output
• AWK Numerical Functions
• Trigonometric Functions
• Exponents, logs and square roots
• Truncating Integers
• "Random Numbers
• The Lotto script
• String Functions
• The Length function
• The Index Function
• The Substr function
• GAWK's Tolower and Toupper function
• The Split function
• NAWK's string functions
• The Match function
• The System function
• The Getline function
• The systime function
• The Strftime function
• User Defined Functions
• AWK patterns
• Formatting AWK programs
• Environment Variables
• ARGC - Number or arguments (NAWK/GAWK)
• ARGV - Array of arguments (NAWK/GAWK)
• ARGIND - Argument Index (GAWK only)
• FNR (NAWK/GAWK)
• OFMT (NAWK/GAWK)
• RSTART, RLENGTH and match (NAWK/GAWK)
• SUBSEP - Multi-dimensional array separator (NAWK/GAWK)
• ENVIRON - environment variables (GAWK only)
• IGNORECASE (GAWK only)
• CONVFMT - conversion format (GAWK only)
• ERRNO - system errors (GAWK only)
• FIELDWIDTHS - fixed width fields (GAWK only)
• AWK, NAWK, GAWK, or PERL

Why learn AWK?

In the past I have covered grep and sed. This section discusses AWK, another cornerstone of UNIX shell programming. There are three variations of AWK:

AWK - the original from AT&T
NAWK - A newer, improved version from AT&T
GAWK - The Free Software foundation's version

Originally, I didn't plan to discuss NAWK, but several UNIX vendors have replaced AWK with NAWK, and there are several incompatibilities between the two. It would be cruel of me to not warn you about the differences. So I will highlight those when I come to them. It is important to know than all of AWK's features are in NAWK and GAWK. Most, if not all, of NAWK's features are in GAWK. NAWK ships as part of Solaris. GAWK does not. However, many sites on the Internet have the sources freely available. If you user Linux, you have GAWK.

Why is AWK so important? It is an excellent filter and report writer. Many UNIX utilities generates rows and columns of information. AWK is an excellent tool for processing these rows and columns, and is easier to use AWK than most conventional programming languages. It can be considered to be a pseudo-C interpretor, as it understands the same arithmatic operators as C. AWK also has string manipulation functions, so it can search for particular strings and modify the output. AWK also has associative arrays, which are incredible useful, and is a feature most computing languages lack. Associative arrays can make a complex problem a trivial exercise.

I won't exhaustively cover AWK. That is, I will cover the essential parts, and avoid the many variants of AWK. It might be too confusing to discuss three different versions of AWK. I won't cover the GNU version of AWK called "gawk." Similarly, I will not discuss the new AT&T AWK called "nawk." The new AWK comes on the Sun system, and you may find it superior to the old AWK in many ways. In particular, it has better diagnostics, and won't print out the infamous "bailing out near line ..." message the original AWK is prone to do. Instead, "nawk" prints out the line it didn't understand, and highlights the bad parts with arrows. If you find yourself needing a feature that is very difficult or impossible to do in AWK, I suggest you either use NAWK, or convert your AWK script into PERL using the "a2p" conversion program which comes with PERL. PERL is a marvelous language, and I use it all the time, but I do not plan to cover PERL in these tutorials. Having made my intention clear, I can continue with a clear conscience.

Many UNIX utilities have strange names. AWK is one of those utilities. It is not an abbreviation for awkward. In fact, it is an elegant and simple language. The work "AWK" is derived from the initials of the language's three developers: A. Aho, B. W. Kernighan and P. Weinberger.

Basic Structure

The essential organization of an AWK program follows the form:

pattern { action }

The pattern specifies when the action is performed. Like most UNIX utilities, AWK is line oriented. That is, the pattern specifies a test that is performed with each line read as input. If the condition is true, then the action is taken. The default pattern is something that matches every line. This is the blank or null pattern. Two other important patterns are specified by the keywords "BEGIN" and "END." As you might expect, these two words specify actions to be taken before any lines are read, and after the last line is read. The AWK program below:

BEGIN   { print "START" }
{ print }
END     { print "STOP" }

adds one line before and one line after the input file. This isn't very useful, but with a simple change, we can make this into a typical AWK program:

BEGIN   { print "File  Owner"," }
 { print $8, "  ", $3}
END     { print " - DONE -" }

The characters "\t" Indicates a tab character so the output lines up on even boundries. The "$8" and "$3" have a meaning similar to a shell script. Instead of the eighth and third argument, they mean the eighth and third field of the input line. You can think of a field as a column, and the action you specify operates on each line or row read in.

There are two differences between AWK and a shell processing the characters within double quotes. AWK understands special characters follow the "" character. The UNIX shells do not. Also, unlike the shell (and PERL) AWK does not evaluate variables within strings. The second line could not be written like this:

{print "$8  $3" }

That example would print "$8 $3." Inside the quotes, the dollar sign is not a special character. Outside, it corresponds to a field. What do I mean by the third and eight field? Consider the "/usr/bin/ls -l" command, which has eight columns of information. The System V version, "/usr/5bin/ls -l," has 9 columns. The third column is the owner, and the eighth column in the name of the file. This AWK program can be used to process the output of the "ls -l" command, printing out the filename, then the owner, for each file. I'll show you how.

Executing an AWK script

Assuming the script is called "FileOwner," the invocation would be

ls -l | FileOwner

This might generate the following if there were only two files in the current directory:

File  Owner
 a.file  barnett
 another.file    barnett
 - DONE -

There are two problems with this script. Both problems are easy to fix, but I'll hold off on this until I cover the basics.

The script itself can be written in many ways. The C shell version would look like this:

#!/bin/csh -f
awk '
BEGIN   { print "File\tOwner" } \
{ print $8, "\t", $3}  \
END     { print " - DONE -" } \
'

View file: awk_example1.csh

As you can see in the above script, each line of the AWK script must have a backslash if it is not the last line of the script. This is necessary as the C shell doesn't, by default, allow strings to span more than one line. The Bourne shell allows quoted strings to span several lines:

#!/bin/sh
 awk '
 BEGIN   { print "File\tOwner" }
 { print $8, "\t", $3}
 END     { print " - DONE -" }

View file: awk_example1.sh The third form is to store the commands in a file, and execute

awk -f filename

Since AWK is also an interpretor, you can make the file executable by add one line in the beginning of the file:

#!/bin/awk -f
 BEGIN   { print "File\tOwner" }
 { print $8, "\t", $3}
 END     { print " - DONE -" }

View file: awk_example1.awk

Change the permission with the chmod command, and the script becomes a new command. Notice the "-f" option of AWK above, which is also used in the third format. It specifies the file containing the instructions. As you can see, AWK considers lines that start with a "#" to be a comment. Well, anything from the "#" to the end of the line is a comment. However, I always comment my AWK scripts with the "#" at the start of the line, for reasons I'll discuss later.

Which format should you use? I prefer the last format when possible. It's shorter and simpler. It's also easier to debug problems. If you need to use a shell, and want to avoid using too many files, you can combine them as we did in the first and second example.

Which shell to use with AWK?

The format of AWK is not free-form. You cannot put new line breaks just anywhere. They must go in particular locations. To be precise, in the original AWK you can insert a new line character after the curly braces, and at the end of a command, but not elsewhere. If you wanted to break a long line into two lines at any other place, you must use a backslash:

#!/bin/awk -f
 BEGIN   { print "File\tOwner" }
 { print $8, "\t",
 $3}
 END     { print " - DONE -" }

View file: awk_example2.awk

The Bourne shell version would be

#!/bin/sh
 awk '
 BEGIN  { print "File\tOwner" }
 { print $8, "\t",
 $3}
 END    { print "done"}
 '

View file: awk_example2.sh

while the C shell would be

#!/bin/csh -f
 awk '
 BEGIN  { print "File\tOwner" }\
 { print $8, "\t", \\
 $3}\
 END    { print "done"}\
 '

View file: awk_example2.csh

As you can see, this demonstrates how awkward the C shell is when enclosing an AWK script. Not only are back slashes needed for every line, some lines need two. (Note - this is true when using old awk (e.g. on Solaris) because the print statement had to be on one line. Newer AWK's allow the print statement to be on multiple lines.) Many people will warn you about the C shell. Some of the problems are subtle, and you may never see them. Try to include an AWK or sed script within a C shell script, and the back slashes will drive you crazy. This is what convinced me to learn the Bourne shell years ago, when I was starting out. I strongly recommend you use the Bourne shell for any AWK or sed script. If you don't use the Bourne shell, then you should learn it. As a minimum, learn how to set variables, which by some strange coincidence is the subject of the next section.

Dynamic Variables

Since you can make a script an AWK executable by mentioning "#!/bin/awk -f" on the first line, including an AWK script inside a shell script isn't needed unless you want to either eliminate the need for an extra file, or if you want to pass a variable to the insides of an AWK script. Since this is a common problem, now is as good a time to explain the technique. I'll do this by showing a simple AWK program that will only print one column. The number of the column will be specified by the first argument. The first version of the program, which we will call "Column," looks like this:

#!/bin/sh
 column=$1
 awk '{print $column}

View file: Column1.sh

A suggested use is:

ls -l | Column 3

This would print the third column from the ls command, which would be the owner of the file. You can change this into a utility that counts how many files are owned by each user by adding

ls -l | Column 3 | uniq -c | sort -nr

Only one problem: the script doesn't work. The value of the "column" variable is not seen by AWK. Change "awk" to "echo" to check. You need to turn off the quoting when the variable is seen. This can be done by ending the quoting, and restarting it after the variable:

#!/bin/sh
 column=$1
 awk '{print $'$column'}

View file: Column2.sh

This is a very important concept, and throws experienced programmers a curve ball. In many computer languages, a string has a start quote, and end quote, and the contents in between. If you want to include a special character inside the quote, you must prevent the character from having the typical meaning. In C this is down by putting a backslash before the character. In other languages, there is a special combination of characters to to this. In the C and Bourne shell, the quote is just a switch. It turns the interpretation mode on or off. There is really no such concept as "start of string" and "end of string." The quotes toggle a switch inside the interpretor. The quote character is not passed on to the application. This is why there are two pairs of quotes above. Notice there are two dollar signs. The first one is quoted, and is seen by AWK. The second one is not quoted, so the shell evaluates the variable, and replaces "$column" by the value. If you don't understand, either change "awk" to "echo," or change the first line to read "#!/bin/sh -x."

Some improvements are needed, however. The Bourne shell has a mechanism to provide a value for a variable if the value isn't set, or is set and the value is an empty string. This is done by using the format:

${variable:-defaultvalue}

This is shown below, where the default column will be one:

#!/bin/sh
 column=${1:-1}
 awk '{print $'$column'}

View file: Column3.sh

We can save a line by combining these two steps:

#!/bin/sh
 awk '{print $'${1:-1}'}

View file: Column4.sh

It is hard to read, but it is compact. There is one other method that can be used. If you execute an AWK command and include on the command line

variable=value

this variable will be set when the AWK script starts. An example of this use would be:

#!/bin/sh
 awk '{print $c}' c=${1:-1}

View file: Column5.sh

This last variation does not have the problems with quoting the previous example had. You should master the earlier example, however, because you can use it with any script or command. The second method is special to AWK.

The Essential Syntax of AWK

Earlier I discussed ways to start an AWK script. This section will discuss the various grammatical elements of AWK.

Arithmetic Expressions

There are several arithmetic operators, similar to C. These are the binary operators, which operate on two variables:

+--------------------------------------------+
|    AWK Table 1        |
|       Binary Operators        |
|Operator  Type       Meaning      |
+--------------------------------------------+
|+    Arithmetic   Addition      |
|-    Arithmetic   Subtraction    |
|*    Arithmetic   Multiplication |
|/    Arithmetic   Division      |
|%    Arithmetic   Modulo      |
|<space>  String       Concatenation      |
+--------------------------------------------+

Using variables with the value of "7" and "3," AWK returns the following results for each operator when using the print command:

+---------------------+
|Expression   Result   |
+---------------------+
  |7+3     10   |
   |7-3     4   |
  |7*3     21   |
|7/3     2.33333 |
   |7%3     1   |
  |7 3     73   |
+---------------------+

There are a few points to make. The modulus operator finds the remainder after an integer divide. The print command output a floating point number on the divide, but an integer for the rest. The string concatenate operator is confusing, since it isn't even visible. Place a space between two variables and the strings are concatenated together. This also shows that numbers are converted automatically into strings when needed. Unlike C, AWK doesn't have "types" of variables. There is one type only, and it can be a string or number. The conversion rules are simple. A number can easily be converted into a string. When a string is converted into a number, AWK will do so. The string "123" will be converted into the number 123. However, the string "123X" will be converted into the number 0. (NAWK will behave differently, and converts the string into integer 123, which is found in the beginning of the string).

Unary arithmetic operators

The "+" and "-" operators can be used before variables and numbers. If X equals 4, then the statement:

print -x;

will print

"-4."

The Autoincrement and Autodecrement Operators

AWK also supports the "++" and "--" operators of C. Both increment or decrement the variables by one. The operator can only be used with a single variable, and can be before or after the variable. The prefix form modifies the value, and then uses the result, while the postfix form gets the results of the variable, and afterwards modifies the variable. As an example, if X has the value of 3, then the AWK statement

print x++, " ", ++x;

would print the numbers 3 and 5. These operators are also assignment operators, and can be used by themselves on a line:

x++;
--y;

Assignment Operators

Variables can be assigned new values with the assignment operators. You know about "++" and "--." The other assignment statement is simply:

variable = arithmetic_expression

Certain operators have precedence over others; parenthesis can be used to control grouping. The statement

x=1+2*3 4;

is the same as

x = (1 + (2 * 3))  "4";

Both print out "74."

Notice spaces can be added for readability. AWK, like C, has special assignment operators, which combine a calculation with an assignment. Instead of saying

x=x+2;

you can more concisely say:

x+=2;

The complete list follows:

+-----------------------------------------+
|    AWK Table 2       |
|      Assignment Operators     |
|Operator   Meaning         |
|+=       Add result to variable     |
|-=       Subtract result from variable |
|*=       Multiply variable by result   |
|/=       Divide variable by result     |
|%=       Apply modulo to variable     |
+-----------------------------------------+

Conditional expressions

The second type of expression in AWK is the conditional expression. This is used for certain tests, like the if or while. Boolean conditions evaluate to true or false. In AWK, there is a definite difference between a boolean condition, and an arithmetic expression. You cannot convert a boolean condition to an integer or string. You can, however, use an arithmetic expression as a conditional expression. A value of 0 is false, while anything else is true. Undefined variables has the value of 0. Unlike AWK, NAWK lets you use booleans as integers.

Arithmetic values can also be converted into boolean conditions by using relational operators:

+---------------------------------------+
|    AWK Table 3      |
|      Relational Operators    |
|Operator   Meaning        |
+---------------------------------------+
|==        Is equal        |
|!=        Is not equal to      |
|>        Is greater than      |
|>=      Is greater than or equal to |
|<        Is less than      |
|<=      Is less than or equal to    |
 
+---------------------------------------+

These operators are the same as the C operators. They can be used to compare numbers or strings. With respect to strings, lower case letters are greater than upper case letters.

Regular Expressions

Two operators are used to compare strings to regular expressions:

+-----------------------------+
|  AWK Table 4       |
|Regular Expression Operators |
|Operator     Meaning       |
+-----------------------------+
|~       Matches       |
|!~       Doesn't match   |
+-----------------------------+

The order in this case is particular. The regular expression must be enclosed by slashes, and comes after the operator. AWK supports extended regular expressions, so the following are examples of valid tests:

word !~ /START/
lawrence_welk ~ /(one|two|three)/

And/Or/Not

There are two boolean operators that can be used with conditional expressions. That is, you can combine two conditional expressions with the "or" or "and" operators: "&&" and "||." There is also the unary not operator: "!."

Commands

There are only a few commands in AWK. The list and syntax follows:

if ( conditional ) statement [ else statement ]
 while ( conditional ) statement
 for ( expression ; conditional ; expression ) statement
 for ( variable in array ) statement
 break
 continue
 { [ statement ] ...}
 variable=expression
 print [ expression-list ] [ > expression ]
 printf format [ , expression-list ] [ > expression ]
 next
 exit

At this point, you can use AWK as a language for simple calculations; If you wanted to calculate something, and not read any lines for input, you could use the BEGIN keyword discussed earlier, combined with a exit command:

#!/bin/awk -f
BEGIN {
 
# Print the squares from 1 to 10 the first way
 
i=1;
while (i <= 10) {
printf "The square of ", i, " is ", i*i;
i = i+1;
}
 
# do it again, using more concise code
 
for (i=1; i <= 10; i++) {
printf "The square of ", i, " is ", i*i;
}
 
# now end
exit;
}

View file: awk_print_squares.awk The following asks for a number, and then squares it:

#!/bin/awk -f
BEGIN {
print "type a number";
}
{
print "The square of ", $1, " is ", $1*$1;
print "type another number";
}
END {
print "Done"
}

View file: awk_ask_for_square.awk

The above isn't a good filter, because it asks for input each time. If you pipe the output of another program into it, you would generate a lot of meaningless prompts.

Here is a filter that you should find useful. It counts lines, totals up the numbers in the first column, and calculates the average. Pipe "wc -c *" into it, and it will count files, and tell you the average number of words per file, as well as the total words and the number of files.

#!/bin/awk -f
BEGIN {
# How many lines
lines=0;
total=0;
}
{
# this code is executed once for each line
# increase the number of files
lines++;
# increase the total size, which is field #1
total+=$1;
}
END {
# end, now output the total
print lines " lines read";
print "total is ", total;
if (lines > 0 ) {
print "average is ", total/lines;
} else {
print "average is 0";
}
}

View file: average.awk

You can pipe the output of "ls -s" into this filter to count the number of files, the total size, and the average size. There is a slight problem with this script, as it includes the output of "ls" that reports the total. This causes the number of files to be off by one. Changing

lines++;

to

if ($1 != "total" ) lines++;

will fix this problem. Note the code which prevents a divide by zero. This is common in well-written scripts. I also initialize the variables to zero. This is not necessary, but it is a good habit.

AWK Built-in Variables

I have mentioned two kinds of variables: positional and user defined. A user defined variable is one you create. A positional variable is not a special variable, but a function triggered by the dollar sign. Therefore

 print $1;

and

 X=1;
 print $X;

do the same thing: print the first field on the line. There are two more points about positional variables that are very useful. The variable "$0" refers to the entire line that AWK reads in. That is, if you had eight fields in a line,

 print $0;

is similar to

 print $1, $2, $3, $4, $5, $6, $7, $8

This will change the spacing between the fields; otherwise, they behave the same. You can modify positional variables. The following commands

 $2="";
 print;

deletes the second field. If you had four fields, and wanted to print out the second and fourth field, there are two ways. This is the first:

#!/bin/awk -f
 {
 $1="";
 $3="";
 print;
 }

and the second

#!/bin/awk -f { print $2, $4; }

These perform similarly, but not identically. The number of spaces between the values vary. There are two reasons for this. The actual number of fields does not change. Setting a positional variable to an empty string does not delete the variable. It's still there, but the contents has been deleted. The other reason is the way AWK outputs the entire line. There is a field separator that specifies what character to put between the fields on output. The first example outputs four fields, while the second outputs two. In-between each field is a space. This is easier to explain if the characters between fields could be modified to be made more visible. Well, it can. AWK provides special variables for just that purpose.

FS - The Input Field Separator Variable

AWK can be used to parse many system administration files. However, many of these files do not have whitespace as a separator. as an example, the password file uses colons. You can easily change the field separator character to be a colon using the "-F" command line option. The following command will print out accounts that don't have passwords:

awk -F: '{if ($2 == "") print $1 ": no password!"}'

There is a way to do this without the command line option. The variable "FS" can be set like any variable, and has the same function as the "-F" command line option. The following is a script that has the same function as the one above.

#!/bin/awk -f
BEGIN {
FS=":";
}
{
if ( $2 == "" ) {
print $1 ": no password!";
}
}

View file: awk_nopasswd.awk

The second form can be used to create a UNIX utility, which I will name "chkpasswd," and executed like this:

chkpasswd

The command "chkpasswd -F:" cannot be used, because AWK will never see this argument. All interpreter scripts accept one and only one argument, which is immediately after the "#!/bin/awk" string. In this case, the single argument is "-f." Another difference between the command line option and the internal variable is the ability to set the input field separator to be more than one character. If you specify

FS=":  ";

then AWK will split a line into fields wherever it sees those two characters, in that exact order. You cannot do this on the command line.

There is a third advantage the internal variable has over the command line option: you can change the field separator character as many times as you want while reading a file. Well, at most once for each line. You can even change it depending on the line you read. Suppose you had the following file which contains the numbers 1 through 7 in three different formats. Lines 4 through 6 have colon separated fields, while the others separated by spaces.

ONE 1 I
 TWO 2 II
 #START
 THREE:3:III
 FOUR:4:IV
 FIVE:5:V
 #STOP
 SIX 6 VI
 SEVEN 7 VII

The AWK program can easily switch between these formats:

#!/bin/awk -f
{
if ($1 == "#START") {
FS=":";
} else if ($1 == "#STOP") {
FS=" ";
} else {
#print the Roman number in column 3
print $3
}
}

View file: awk_example3.awk

Note the field separator variable retains its value until it is explicitly changed. You don't have to reset it for each line. Sounds simple, right? However, I have a trick question for you. What happens if you change the field separator while reading a line? That is, suppose you had the following line

One Two:Three:4 Five

and you executed the following script:

#!/bin/awk -f
 {
 print $2
 FS=":"
 print $2
 }

What would be printed? "Three" or "Two:Three:4?" Well, the script would print out "Two:Three:4" twice. However, if you deleted the first print statement, it would print out "Three" once! I thought this was very strange at first, but after pulling out some hair, kicking the deck, and yelling at muself and everyone who had anything to do with the development of UNIX, it is intuitively obvious. You just have to be thinking like a professional programmer to realize it is intuitive. I shall explain, and prevent you from causing yourself physical harm.

If you change the field separator before you read the line, the change affects what you read. If you change it after you read the line, it will not redefine the variables. You wouldn't want a variable to change on you as a side-effect of another action. A programming language with hidden side effects is broken, and should not be trusted. AWK allows you to redefine the field separator either before or after you read the line, and does the right thing each time. Once you read the variable, the variable will not change unless you change it. Bravo!

To illustrate this further, here is another version of the previous code that changes the field separator dynamically. In this case, AWK does it by examining field "$0," which is the entire line. When the line contains a colon, the field separator is a colon, otherwise, it is a space:

#!/bin/awk -f
{
if ( $0 ~ /:/ ) {
FS=":";
} else {
FS=" ";
}
#print the third field, whatever format
print $3
}

View file: awk_example4.awk

This example eliminates the need to have the special "#START" and "#STOP" lines in the input.

OFS - The Output Field Separator Variable

There is an important difference between

print $2 $3

and

print $2, $3

The first example prints out one field, and the second prints out two fields. In the first case, the two positional parameters are concatenated together and output without a space. In the second case, AWK prints two fields, and places the output field separator between them. Normally this is a space, but you can change this by modifying the variable "OFS."

If you wanted to copy the password file, but delete the encrypted password, you could use AWK:

#!/bin/awk -f
 BEGIN {
 FS=":";
 OFS=":";
 }
 {
 $2="";
 print }

View file: delete_passwd.awk

Give this script the password file, and it will delete the password, but leave everything else the same. You can make the output field separator any number of characters. You are not limited to a single character.

NF - The Number of Fields Variable

It is useful to know how many fields are on a line. You may want to have your script change its operation based on the number of fields. As an example, the command "ls -l" may generate eight or nine fields, depending on which version you are executing. The System V version, "/usr/bin/ls -l" generates nine fields, which is equivalent to the Berkeley "/usr/ucb/ls -lg" command. If you wanted to print the owner and filename then the following AWK script would work with either version of "ls:"

#!/bin/awk -f
 # parse the output of "ls -l"
 # print owner and filename
 # remember - Berkeley ls -l has 8 fields, System V has 9
 {
 if (NF == 8) {
 print $3, $8;
 } else if (NF == 9) {
 print $3, $9;
 }
 }

View file: owner_group.awk

Don't forget the variable can be prepended with a "$." This allows you to print the last field of any column

#!/bin/awk -f
 { print $NF; }

View file: print_last_field.awk

One warning about AWK. There is a limit of 99 fields in a single line. PERL does not have any such limitations.

NR - The Number of Records Variable

Another useful variable is "NR." This tells you the number of records, or the line number. You can use AWK to only examine certain lines. This example prints lines after the first 100 lines, and puts a line number before each line after 100:

#!/bin/awk -f
 { if (NR >= 100) {
 print NR, $0; }

View file: awk_example5.awk

RS - The Record Separator Variable

Normally, AWK reads one line at a time, and breaks up the line into fields. You can set the "RS" variable to change AWK's definition of a "line." If you set it to an empty string, then AWK will read the entire file into memory. You can combine this with changing the "FS" variable. This example treats each line as a field, and prints out the second and third line:

#!/bin/awk -f
BEGIN {
# change the record separator from newline to nothing
RS=""
# change the field separator from whitespace to newline
FS="n"
}
{
# print the second and third line of the file
print $2, $3;
}
.)l
#!/bin/awk -f
# reports which file is being read
BEGIN {
f="";
}
{  if (f != FILENAME) {
print "reading", FILENAME;
f=FILENAME;
}
print;
}

View file: awk_example6.awk

The two lines are printed with a space between. Also this will only work if the input file is less than 100 lines, therefore this technique is limited. You can use it to break words up, one word per line, using this:

#!/bin/awk -f
 BEGIN {
 RS=" ";
 }
 {
 print ; }

View file: oneword_per_line.awk

but this only works if all of the words are separated by a space. If there is a tab or punctuation inside, it would not.

ORS - The Record Separator Variable

The default output record separator is a newline, like the input. This can be set to be a newline and carriage return, if you need to generate a text file for a non-UNIX system.

#!/bin/awk -f
 # this filter adds a carriage return to all lines
 # before the newline character
 BEGIN {
 ORS="rn"
 }
 { print }

View file: add_cr.awk

FILENAME - The Current Filename Variable

The last variable known to regular AWK is "FILENAME," which tells you the name of the file being read.

#!/bin/awk -f
 BEGIN {
 # change the record separator from newline to nothing
 RS=""
 # change the field separator from whitespace to newline
 FS="n"
 }
 {
 # print the second and third line of the file
 print $2, $3;
 }
 .)l
 #!/bin/awk -f
 # reports which file is being read
 BEGIN {
 f="";
 }
 {  if (f != FILENAME) {
 print "reading", FILENAME;
 f=FILENAME;
 }
 print; }

View file: awk_example6.awk

This can be used if several files need to be parsed by AWK. Normally you use standard input to provide AWK with information. You can also specify the filenames on the command line. If the above script was called "testfilter," and if you executed it with

testfilter file1 file2 file3

It would print out the filename before each change. An alternate way to specify this on the command line is

testfilter file1 - file3 <file2
 

In this case, the second file will be called "-," which is the conventional name for standard input. I have used this when I want to put some information before and after a filter operation. The prefix and postfix files special data before and after the real data. By checking the filename, you can parse the information differently. This is also useful to report syntax errors in particular files:

#!/bin/awk -f
{
if (NF == 6) {
# do the right thing
} else {
if (FILENAME == "-" ) {
print "SYNTAX ERROR, Wrong number of fields,",
"in STDIN, line #:", NR,  "line: ", $0;
} else {
print "SYNTAX ERROR, Wrong number of fields,",
"Filename: ", FILENAME, "line # ", NR,"line: ", $0;
}
}
}

View file: awk_example7.awk

Associative Arrays

I have used dozens of different programming languages over the last 20 years, and AWK is the first language I found that has associative arrays. This term may be meaningless to you, but believe me, these arrays are invaluable, and simplify programming enormously. Let me describe a problem, and show you how associative arrays can be used for reduce coding time, giving you more time to explore another stupid problem you don't want to deal with in the first place.

Let's suppose you have a directory overflowing with files, and you want to find out how many files are owned by each user, and perhaps how much disk space each user owns. You really want someone to blame; it's hard to tell who owns what file. A filter that processes the output of ls would work:

ls -l | filter

But this doesn't tell you how much space each user is using. It also doesn't work for a large directory tree. This requires find and xargs:

find . -type f -print | xargs ls -l | filter

The third column of "ls" is the username. The filter has to count how many times it sees each user. The typical program would have an array of usernames and another array that counts how many times each username has been seen. The index to both arrays are the same; you use one array to find the index, and the second to keep track of the count. I'll show you one way to do it in AWK--the wrong way:

#!/bin/awk -f
# bad example of AWK programming
# this counts how many files each user owns.
BEGIN {
number_of_users=0;
}
{
# must make sure you only examine lines with 8 or more fields
if (NF>7) {
user=0;
# look for the user in our list of users
for (i=1; i<=number_of_users; i++) {
#    is the user known?
if (username[i] == $3) {
# found it - remember where the user is
    user=i;
    }
  }
if (user == 0) {
# found a new user
username[++number_of_users]=$3;
user=number_of_users;
  }
# increase number of counts
count[user]++;
}
}
END {
for (i=1; i<=number_of_users; i++) {
print count[i], username[i]
}
}

View file: awk_example8.awk

I don't want you to read this script. I told you it's the wrong way to do it. If you were a C programmer, and didn't know AWK, you would probably use a technique like the one above. Here is the same program, except this example that uses AWK's associative arrays. The important point is to notice the difference in size between these two versions:

#!/bin/awk -f
 {
 username[$3]++;
 }
 END {
 for (i in username) {
 print username[i], i;
 } }

View file: count_users0.awk

This is shorter, simpler, and much easier to understand--Once you understand exactly what an associative array is. The concept is simple. Instead of using a number to find an entry in an array, use anything you want. An associative array in an array whose index is a string. All arrays in AWK are associative. In this case, the index into the array is the third field of the "ls" command, which is the username. If the user is "bin," the main loop increments the count per user by effectively executing

 username["bin"]++;

UNIX guru's may gleefully report that the 8 line AWK script can be replaced by:

awk '{print $3}' | sort | uniq -c | sort -nr

True, However, this can't count the total disk space for each user. We need to add some more intelligence to the AWK script, and need the right foundation to proceed. There is also a slight bug in the AWK program. If you wanted a "quick and dirty" solution, the above would be fine. If you wanted to make it more robust, you have to handle unusual conditions. If you gave this program an empty file for input, you would get the error:

awk: username is not an array

Also, if you piped the output of "ls -l" to it, the line that specified the total would increment a non-existing user. There are two techniques used to eliminate this error. The first one only counts valid input:

#!/bin/awk -f
{
if (NF>7) {
username[$3]++;
}
}
END {
for (i in username) {
print username[i], i;
}
}

View file: count_users1.awk This fixes the problem of counting the line with the total. However, it still generates an error when an empty file is read as input. To fix this problem, a common technique is to make sure the array always exists, and has a special marker value which specifies that the entry is invalid. Then when reporting the results, ignore the invalid entry.

#!/bin/awk -f
BEGIN {
username[""]=0;
}
{
username[$3]++;
}
END {
for (i in username) {
if (i != "") {
print username[i], i;
  }
}
}

View file: count_users2.awk

This happens to fix the other problem. Apply this technique and you will make your AWK programs more robust and easier for others to use.

Multi-dimensional Arrays

Some people ask if AWK can handle multi-dimensional arrays. It can. However, you don't use conventional two-dimensional arrays. Instead you use associative arrays. (Did I even mention how useful associative arrays are?) Remember, you can put anything in the index of an associative array. It requires a different way to think about problems, but once you understand, you won't be able to live without it. All you have to do is to create an index that combines two other indices. Suppose you wanted to effectively execute

 a[1,2] = y;

This is invalid in AWK. However, the following is perfectly fine:

 a[1 "," 2] = y;

Remember: the AWK string concatenation operator is the space. It combines the three strings into the single string "1,2." Then it uses it as an index into the array. That's all there is to it. There is one minor problem with associative arrays, especially if you use the for command to output each element: you have no control over the order of output. You can create an algorithm to generate the indices to an associative array, and control the order this way. However, this is difficult to do. Since UNIX provides an excellent sort utility, more programmers separate the information processing from the sorting. I'll show you what I mean.

Example of using AWK's Associative Arrays

I often find myself using certain techniques repeatedly in AWK. This example will demonstrate these techniques, and illustrate the power and elegance of AWK. The program is simple and common. The disk is full. Who's gonna be blamed? I just hope you use this power wisely. Remember, you may be the one who filled up the disk.

Having resolved my moral dilemma, by placing the burden squarely on your shoulders, I will describe the program in detail. I will also discuss several tips you will find useful in large AWK programs. First, initialize all arrays used in a for loop. There will be four arrays for this purpose. Initialization is easy:

u_count[""]=0;
 g_count[""]=0;
 ug_count[""]=0;
 all_count[""]=0;

The second tip is to pick a convention for arrays. Selecting the names of the arrays, and the indices for each array is very important. In a complex program, it can become confusing to remember which array contains what. I suggest you clearly identify the indices and contents of each array. To demonstrate, I will use a "_count" to indicate the number of files, and "_sum" to indicate the sum of the file sizes. In addition, the part before the "_" specifies the index used for the array, which will be either "u" for user, "g" for group, "ug" for the user and group combination, and "all" for the total for all files. In other programs, I have used names like

username_to_directory[username]=directory;

Follow a convention like this, and it will be hard for you to forget the purpose of each associative array. Even when a quick hack comes back to haunt you three years later. I've been there.

The third suggestion is to make sure your input is in the correct form. It's generally a good idea to be pessimistic, but I will add a simple but sufficient test in this example.

if (NF != 10) {
 #  ignore
 } else {
etc.

I placed the test and error clause up front, so the rest of the code won't be cluttered. AWK doesn't have user defined functions. NAWK, GAWK and PERL do.

The next piece of advice for complex AWK scripts is to define a name for each field used. In this case, we want the user, group and size in disk blocks. We could use the file size in bytes, but the block size corresponds to the blocks on the disk, a more accurate measurement of space. Disk blocks can be found by using "ls -s." This adds a column, so the username becomes the fourth column, etc. Therefore the script will contain:

size=$1;
 user=$4;
 group=$5;

This will allow us to easily adapt to changes in input. We could use "$1" throughout the script, but if we changed the number of fields, which the "-s" option does, we'd have to change each field reference. You don't want to go through an AWK script, and change all the "$1" to "$2," and also change the "$2" to "$3" because those are really the "$1" that you just changed to "$2." Of course this is confusing. That's why it's a good idea to assign names to the fields. I've been there too.

Next the AWK script will count how many times each combination of users and groups occur. That is, I am going to construct a two-part index that contains the username and groupname. This will let me count up the number of times each user/group combination occurs, and how much disk space is used.

Consider this: how would you calculate the total for just a user, or for just a group? You could rewrite the script. Or you could take the user/group totals, and total them with a second script.

You could do it, but it's not the AWK way to do it. If you had to examine a bazillion files, and it takes a long time to run that script, it would be a waste to repeat this task. It's also inefficient to require two scripts when one can do everything. The proper way to solve this problem is to extract as much information as possible in one pass through the files. Therefore this script will find the number and size for each category:

Each user
 Each group
 Each user/group combination
 All users and groups

This is why I have 4 arrays to count up the number of files. I don't really need 4 arrays, as I can use the format of the index to determine which array is which. But this does maake the program easier to understand for now. The next tip is subtle, but you will see how useful it is. I mentioned the indices into the array can be anything. If possible, select a format that allows you to merge information from several arrays. I realize this makes no sense right now, but hang in there. All will become clear soon. I will do this by constructing a universal index of the form

<user><group>

This index will be used for all arrays. There is a space between the two values. This covers the total for the user/group combination. What about the other three arrays? I will use a "*" to indicate the total for all users or groups. Therefore the index for all files would be "* *" while the index for all of the file owned by user daemon would be "daemon *." The heart of the script totals up the number and size of each file, putting the information into the right category. I will use 8 arrays; 4 for file sizes, and 4 for counts:

u_count[user " *"]++;
 g_count["* " group]++;
 ug_count[user " " group]++;
 all_count["* *"]++;
 u_size[user " *"]+=size;
 g_size["* " group]+=size;
 ug_size[user " " group]+=size;
 all_size["* *"]+=size;
</p>

This particular universal index will make sorting easier, as you will see. Also important is to sort the information in an order that is useful. You can try to force a particular output order in AWK, but why work at this, when it's a one line command for sort? The difficult part is finding the right way to sort the information. This script will sort information using the size of the category as the first sort field. The largest total will be the one for all files, so this will be one of the first lines output. However, there may be several ties for the largest number, and care must be used. The second field will be the number of files. This will help break a tie. Still, I want the totals and sub-totals to be listed before the individual user/group combinations. The third and fourth fields will be generated by the index of the array. This is the tricky part I warned you about. The script will output one string, but the sort utility will not know this. Instead, it will treat it as two fields. This will unify the results, and information from all 4 arrays will look like one array. The sort of the third and fourth fields will be dictionary order, and not numeric, unlike the first two fields. The "*" was used so these sub-total fields will be listed before the individual user/group combination.

The arrays will be printed using the following format:

for (i in u_count) {
 if (i != "") {
 print u_size[i], u_count[i], i;
 }
 }

I only showed you one array, but all four are printed the same way. That's the essence of the script. The results is sorted, and I converted the space into a tab for cosmetic reasons.

Output of the script

I changed my directory to /usr/ucb, used the script in that directory. The following is the output:

size   count   user   group
3173   81      *      *
3173   81      root   *
2973   75      *      staff
2973   75      root   staff
88     3       *      daemon
88     3       root   daemon
64     2       *      kmem
64     2       root   kmem
48     1       *      tty
48     1       root   tty

This says there are 81 files in this directory, which takes up 3173 disk blocks. All of the files are owned by root. 2973 disk blocks belong to group staff. There are 3 files with group daemon, which takes up 88 disk blocks.

As you can see, the first line of information is the total for all users and groups. The second line is the sub-total for the user "root." The third line is the sub-total for the group "staff." Therefore the order of the sort is useful, with the sub-totals before the individual entries. You could write a simple AWK or grep script to obtain information from just one user or one group, and the information will be easy to sort.

There is only one problem. The /usr/ucb directory on my system only uses 1849 blocks; at least that's what du reports. Where's the discrepancy? The script does not understand hard links. This may not be a problem on most disks, because many users do not use hard links. Still, it does generate inaccurate results. In this case, the program vi is also e, ex, edit, view, and 2 other names. The program only exists once, but has 7 names. You can tell because the link count (field 2) reports 7. This causes the file to be counted 7 times, which causes an inaccurate total. The fix is to only count multiple links once. Examining the link count will determine if a file has multiple links. However, how can you prevent counting a link twice? There is an easy solution: all of these files have the same inode number. You can find this number with the -i option to ls. To save memory, we only have to remember the inodes of files that have multiple links. This means we have to add another column to the input, and have to renumber all of the field references. It's a good thing there are only three. Adding a new field will be easy, because I followed my own advice.

The final script should be easy to follow. I have used variations of this hundreds of times and find it demonstrates the power of AWK as well as provide insight to a powerful programming paradigm. AWK solves these types of problems easier than most languages. But you have to use AWK the right way.

This is a fully working version of the program, that accurately counts disk space, appears below:

#!/bin/sh
find . -type f -print | xargs /usr/bin/ls -islg |
awk '
BEGIN {
# initialize all arrays used in for loop
u_count[""]=0;
g_count[""]=0;
ug_count[""]=0;
all_count[""]=0;
}
{
# validate your input
if (NF != 11) {
#  ignore
} else {
# assign field names
inode=$1;
size=$2;
linkcount=$4;
user=$5;
group=$6;
 
# should I count this file?
 
doit=0;
if (linkcount == 1) {
# only one copy - count it
  doit++;
} else {
# a hard link - only count first one
seen[inode]++;
if (seen[inode] == 1) {
    doit++;
    }
  }
# if doit is true, then count the file
if (doit ) {
 
# total up counts in one pass
# use description array names
# use array index that unifies the arrays
 
# first the counts for the number of files
 
u_count[user " *"]++;
g_count["* " group]++;
ug_count[user " " group]++;
all_count["* *"]++;
 
# then the total disk space used
 
u_size[user " *"]+=size;
g_size["* " group]+=size;
ug_size[user " " group]+=size;
all_size["* *"]+=size;
  }
}
}
END {
# output in a form that can be sorted
for (i in u_count) {
  if (i != "") {
print u_size[i], u_count[i], i;
      }
}
for (i in g_count) {
  if (i != "") {
print g_size[i], g_count[i], i;
      }
}
for (i in ug_count) {
  if (i != "") {
print ug_size[i], ug_count[i], i;
      }
}
for (i in all_count) {
  if (i != "") {
print all_size[i], all_count[i], i;
      }
}
} ' |
# numeric sort - biggest numbers first
# sort fields 0 and 1 first (sort starts with 0)
# followed by dictionary sort on fields 2 + 3
sort +0nr -2 +2d |
# add header
(echo "size count user group";cat -) |
# convert space to tab - makes it nice output
# the second set of quotes contains a single tab character
tr ' ' '  '
# done - I hope you like it

View file: count_users3.awk

Remember when I said I didn't need to use 4 different arrays? I can use just one. This is more confusing, but more concise

#!/bin/sh
find . -type f -print | xargs /usr/bin/ls -islg |
awk '
BEGIN {
# initialize all arrays used in for loop
count[""]=0;
}
{
# validate your input
if (NF != 11) {
#  ignore
} else {
# assign field names
inode=$1;
size=$2;
linkcount=$4;
user=$5;
group=$6;
 
# should I count this file?
 
doit=0;
if (linkcount == 1) {
# only one copy - count it
  doit++;
} else {
# a hard link - only count first one
seen[inode]++;
if (seen[inode] == 1) {
    doit++;
    }
  }
# if doit is true, then count the file
if (doit ) {
 
# total up counts in one pass
# use description array names
# use array index that unifies the arrays
 
# first the counts for the number of files
 
count[user " *"]++;
count["* " group]++;
count[user " " group]++;
count["* *"]++;
 
# then the total disk space used
 
size[user " *"]+=size;
size["* " group]+=size;
size[user " " group]+=size;
size["* *"]+=size;
  }
}
}
END {
# output in a form that can be sorted
for (i in count) {
if (i != "") {
print size[i], count[i], i;
  }
}
} ' |
 
# numeric sort - biggest numbers first
# sort fields 0 and 1 first (sort starts with 0)
# followed by dictionary sort on fields 2 + 3
sort +0nr -2 +2d |
# add header
(echo "size count user group";cat -) |
# convert space to tab - makes it nice output
# the second set of quotes contains a single tab character
tr ' ' '  '
# done - I hope you like it

View file: count_users.awk

Picture Perfect PRINTF Output

So far, I described several simple scripts that provide useful information, in a somewhat ugly output format. Columns might not line up properly, and it is often hard to find patterns or trends without this unity. As you use AWK more, you will be desirous of crisp, clean formatting. To achieve this, you must master the printf function.

PRINTF - formatting output

The printf is very similar to the C function with the same name. C programmers should have no problem using printf function.

Printf has one of these syntactical forms:

printf ( format);
 printf ( format, arguments...);
 printf ( format) >expression;
 printf ( format, arguments...) > expression;

The parenthesis and semicolon are optional. I only use the first format to be consistent with other nearby printf statements. A print statement would do the same thing. Printf reveals it's real power when formatting commands are used.

The first argument to the printf function is the format. This is a string, or variable whose value is a string. This string, like all strings, can contain special escape sequences to print control characters.

Escape Sequences

The character "" is used to "escape" or mark special characters. The list of these characters is in table below:

+-------------------------------------------------------+
|      AWK Table 5        |
|          Escape Sequences        |
|Sequence   Description          |
+-------------------------------------------------------+
|a        ASCII bell (NAWK only)        |
|b        Backspace            |
|f        Formfeed            |
|n        Newline            |
|r        Carriage Return          |
|t        Horizontal tab          |
|v        Vertical tab (NAWK only)        |
|ddd        Character (1 to 3 octal digits) (NAWK only) |
|xdd        Character (hexadecimal) (NAWK only)    |
|        Any character c          |
+-------------------------------------------------------+

It's difficult to explain the differences without being wordy. Hopefully I'll provide enough examples to demonstrate the differences.

With NAWK, you can print three tab characters using these three different representations:

printf("t11x9n");

A tab character is decimal 11, or hexadecimal 9. See the man page ascii(7) for more information. Similarly, you can print three double-quote characters (decimal 42, hexadecimal 22) using

printf(""x2242n");

You should notice a difference between the printf function and the print function. Print terminates the line with the OFS character, and divides each field with the ORS separator. Printf does nothing unless you specify the action. Therefore you will frequently end each line with the newline character "n," and you must specify the separating characters explicitly.

Format Specifiers

The power of the printf statement lies in the format specifiers, which always start with the character "%." The format specifiers are described in table 6:

+----------------------------------------+
|    AWK Table 6      |
|       Format Specifiers      |
|Specifier   Meaning        |
+----------------------------------------+
|c        ASCII Character      |
|d        Decimal integer      |
|e        Floating Point number    |
|        (engineering format)    |
|f        Floating Point number    |
|        (fixed point format)    |
|g        The shorter of e or f,    |
|        with trailing zeros removed |
|o        Octal        |
|s        String        |
|x        Hexadecimal      |
|%        Literal %        |
+----------------------------------------+

Again, I'll cover the differences quickly. Table 3 illustrates the differences. The first line states "printf(%cn",100.0)"" prints a "d."

+--------------------------------+
|    AWK Table 7        |
| Example of format conversions  |
|Format   Value   Results      |
+--------------------------------+
|%c       100.0   d        |
|%c       "100.0"   1 (NAWK?)    |
|%c       42   "        |
|%d       100.0   100        |
|%e       100.0   1.000000e+02 |
|%f       100.0   100.000000   |
|%g       100.0   100        |
|%o       100.0   144        |
|%s       100.0   100.0        |
|%s       "13f"   13f        |
|%d       "13f"   0 (AWK)      |
|%d       "13f"   13 (NAWK)    |
|%x       100.0   64        |
+--------------------------------+

This table reveals some differences between AWK and NAWK. When a string with numbers and letters are coverted into an integer, AWK will return a zero, while NAWK will convert as much as possible. The second example, marked with "NAWK?" will return "d" on some earlier versions of NAWK, while later versions will return "1."

Using format specifiers, there is another way to print a double quote with NAWK. This demonstrates Octal, Decimal and Hexadecimal conversion. As you can see, it isn't symmetrical. Decimal conversions are done differently.

printf("%s%s%s%cn", """, "x22", "42", 34);

Between the "%" and the format character can be four optional pieces of information. It helps to visualize these fields as:

%
<sign>
<zero>
<width>
.
<precision>
format

I'll discuss each one separately.

Width - specifying minimum field size

If there is a number after the "%," this specifies the minimum number of characters to print. This is the width field. Spaces are added so the number of printed characters equal this number. Note that this is the minimum field size. If the field becomes to large, it will grow, so information will not be lost. Spaces are added to the left.

This format allows you to line up columns perfectly. Consider the following format:

printf("%st%dn", s, d);

If the string "s" is longer than 8 characters, the columns won't line up. Instead, use

printf("%20s%dn", s, d);

As long as the string is less than 20 characters, the number will start on the 21st column. If the string is too long, then the two fields will run together, making it hard to read. You may want to consider placing a single space between the fields, to make sure you will always have one space between the fields. This is very important if you want to pipe the output to another program.

Adding informational headers makes the output more readable. Be aware that changing the format of the data may make it difficult to get the columns aligned perfectly. Consider the following script:

#!/usr/bin/awk -f
 BEGIN {
 printf("String     Numbern");
 }
 {
 printf("%10s %6dn", $1, $2); }

View file: awk_example9.awk It would be awkward (forgive the choice of words) to add a new column and retain the same alignment. More complicated formats would require a lot of trial and error. You have to adjust the first printf to agree with the second printf statement. I suggest

#!/usr/bin/awk -f
 BEGIN {
 printf("%10s %6sn", "String", "Number");
 }
 {
 printf("%10s %6dn", $1, $2); }

View file: awk_example10.awk

or even better

#!/usr/bin/awk -f
 BEGIN {
 format1 ="%10s %6sn";
 format2 ="%10s %6dn";
 printf(format1, "String", "Number");
 }
 {
 printf(format2, $1, $2); }

View file: awk_example11.awk

The last example, by using string variables for formatting, allows you to keep all of the formats together.

Left Justification

The last example places spaces before each field to make sure the minimum field width is met. What do you do if you want the spaces on the right? Add a negative sign before the width:

 printf("%-10s %-6dn", $1, $2);

This will move the printing characters to the left, with spaces added to the right.

The Field Precision Value

The precision field, which is the number between the decimal and the format character, is more complex. Most people use it with the floating point format (%f), but surprisingly, it can be used with any format character. With the octal, decimal or hexadecimal format, it specifies the minimum number of characters. Zeros are added to met this requirement. With the %e and %f formats, it specifies the number of digits after the decimal point. The %e "e+00" is not included in the precision. The %g format combines the characteristics of the %d and %f formats. The precision specifies the number of digits displayed, before and after the decimal point. The precision field has no effect on the %c field. The %s format has an unusual, but useful effect: it specifies the maximum number of significant characters to print.

If the first number after the "%," or after the "%-," is a zero, then the system adds zeros when padding. This includes all format types, including strings and the %c character format. This means "%010d" and "%.10d" both adds leading zeros, giving a minimum of 10 digits. The format "%10.10d" is therefore redundant. Table 8 gives some examples:

+--------------------------------------------+
|    AWK Table 8        |
|      Examples of complex formatting      |
|Format    Variable     Results      |
+--------------------------------------------+
|%c    100       "d"        |
|%10c    100       "       d"      |
|%010c    100       "000000000d"      |
+--------------------------------------------+
|%d    10       "10"        |
|%10d    10       "      10"      |
|%10.4d    10.123456789     "    0010"      |
|%10.8d    10.123456789     "  00000010"      |
|%.8d    10.123456789     "00000010"      |
|%010d    10.123456789     "0000000010"      |
+--------------------------------------------+
|%e    987.1234567890   "9.871235e+02"   |
|%10.4e    987.1234567890   "9.8712e+02"      |
|%10.8e    987.1234567890   "9.87123457e+02" |
+--------------------------------------------+
|%f    987.1234567890   "987.123457"      |
|%10.4f    987.1234567890   "  987.1235"      |
|%010.4f    987.1234567890   "00987.1235"      |
|%10.8f    987.1234567890   "987.12345679"   |
+--------------------------------------------+
|%g    987.1234567890   "987.123"      |
|%10g    987.1234567890   "   987.123"      |
|%10.4g    987.1234567890   "   987.1"      |
|%010.4g    987.1234567890   "00000987.1"      |
|%.8g    987.1234567890   "987.12346"      |
+--------------------------------------------+
|%o    987.1234567890   "1733"      |
|%10o    987.1234567890   "    1733"      |
|%010o    987.1234567890   "0000001733"      |
|%.8o    987.1234567890   "00001733"      |
+--------------------------------------------+
|%s    987.123     "987.123"      |
|%10s    987.123     "   987.123"      |
|%10.4s    987.123     "    987."      |
|%010.8s    987.123     "000987.123"      |
+--------------------------------------------+
|%x    987.1234567890   "3db"      |
|%10x    987.1234567890   "     3db"      |
|%010x    987.1234567890   "00000003db"      |
|%.8x    987.1234567890   "000003db"      |
+--------------------------------------------+

There is one more topic needed to complete this lesson on printf.

Explicit File output

Instead of sending output to standard output, you can send output to a named file. The format is

printf("stringn") > "/tmp/file";

You can append to an existing file, by using ">>:"

printf("stringn") >> "/tmp/file";

Like the shell, the double angle brackets indicates output is appended to the file, instead of written to an empty file. Appending to the file does not delete the old contents. However, there is a subtle difference between AWK and the shell.

Consider the shell program:

#!/bin/sh
 while x=`line`
 do
 echo got $x >>/tmp/a
 echo got $x >/tmp/b
 done

This will read standard input, and copy the standard input to files "/tmp/a" and "/tmp/b." File "/tmp/a" will grow larger, as information is always appended to the file. File "/tmp/b," however, will only contain one line. This happens because each time the shell see the ">" or ">>" characters, it opens the file for writing, choosing the truncate/create or appending option at that time.

Now consider the equivalent AWK program:

#!/usr/bin/awk -f
 {
 print $0 >>"/tmp/a"
 print $0 >"/tmp/b"
 }

This behaves differently. AWK chooses the create/append option the first time a file is opened for writing. Afterwards, the use of ">" or ">>" is ignored. Unlike the shell, AWK copies all of standard input to file "/tmp/b."

Instead of a string, some versions of AWK allow you to specify an expression:

# [note to self] check this one - it might not work
 printf("stringn") > FILENAME ".out";

The following uses a string concatenation expression to illustrate this:

#!/usr/bin/awk -f
 END {
 for (i=0;i<30;i++) {
 printf("i=%dn", i) > "/tmp/a" i;
 } }

View file: awk_example12.awk This script never finishes, because AWK can have 10 additional files open, and NAWK can have 20. If you find this to be a problem, look into PERL.

I hope this gives you the skill to make your AWK output picture perfect.

AWK Numerical Functions

In previous tutorials, I have shown how useful AWK is in manipulating information, and generating reports. When you add a few functions, AWK becomes even more, mmm, functional.

There are three types of functions: numeric, string and whatever's left. Table9 lists all of the numeric functions:

+----------------------------------+
|    AWK Table 9         |
+----------------------------------+
|       Numeric Functions         |
|Name    Function     Variant     |
+----------------------------------+
|cos     cosine     AWK         |
|exp     Exponent     AWK         |
|int     Integer     AWK         |
|log     Logarithm     AWK         |
|sin     Sine     AWK         |
|sqrt    Square Root   AWK         |
|atan2   Arctangent     NAWK         |
|rand    Random     NAWK         |
|srand   Seed Random   NAWK         |
+----------------------------------+

Trigonometric Functions

Oh joy. I bet millions, if not dozens, of my readers have been waiting for me to discuss trigonometry. Personally, I don't use trigonometry much at work, except when I go off on a tangent.

Sorry about that. I don't know what came over me. I don't usually resort to puns. I'll write a note to myself, and after I sine the note, I'll have my boss cosine it.

Now stop that! I hate arguing with myself. I always lose. Thinking about math I learned in the year 2 B.C. (Before Computers) seems to cause flashbacks of high school, pimples, and (shudder) times best left forgotten. The stress of remembering those days must have made me forget the standards I normally set for myself. Besides, no-one appreciates obtuse humor anyway, even if I find acute way to say it.

I better change the subject fast. Combining humor and computers is a very serious matter.

Here is a NAWK script that calculates the trigonometric functions for all degrees between 0 and 360. It also shows why there is no tangent, secant or cosecant function. (They aren't necessary). If you read the script, you will learn of some subtle differences between AWK and NAWK. All this in a thin veneer of demonstrating why we learned trigonometry in the first place. What more can you ask for? Oh, in case you are wondering, I wrote this in the month of December.

#!/usr/bin/nawk -f
#
# A smattering of trigonometry...
#
# This AWK script plots the values from 0 to 360
# for the basic trigonometry functions
# but first - a review:
#
# (Note to the editor - the following diagram assumes
# a fixed width font, like Courier.
# otherwise, the diagram  looks very stupid, instead of slightly stupid)
#
# Assume the following right triangle
#
#       Angle Y
#
#       |
#       |
#       |
#     a |     c
#       |
#       |
#       +-------   Angle X
#          b
#
# since the triangle is a right angle, then
#  X+Y=90
#
# Basic Trigonometric Functions. If you know the length
# of 2 sides, and the angles, you can find the length of the third side.
# Also - if you know the length of the sides, you can calculate
# the angles.
#
# The formulas are
#
#  sine(X) = a/c
#  cosine(X) = b/c
#  tangent(X) = a/b
#
# reciprocal functions
#  cotangent(X) = b/a
#  secant(X) = c/b
#  cosecant(X) = c/a
#
# Example 1)
# if an angle is 30, and the hypotenuse (c) is 10, then
#   a = sine(30) * 10 = 5
#  b = cosine(30) * 10 =  8.66
#
# The second example will be more realistic:
#
#   Suppose you are looking for a Christmas tree, and
# while talking to your family, you smack into a tree
# because your head was turned, and your kids were arguing over who
# was going to put the first ornament on the tree.
#
# As you come to, you realize your feet are touching the trunk of the tree,
# and your eyes are 6 feet from the bottom of your frostbitten toes.
# While counting the stars that spin around your head, you also realize
# the top of the tree is located at a 65 degree angle, relative to your eyes.
 
# You suddenly realize the tree is 12.84 feet high! After all,
#   tangent(65 degrees) * 6 feet = 12.84 feet
 
# All right, it isn't realistic. Not many people memorize the
# tangent table, or can estimate angles that accurately.
# I was telling the truth about the stars spinning around the head, however.
 
#
BEGIN {
# assign a value for pi.
PI=3.14159;
# select an "Ed Sullivan" number - really really big
BIG=999999;
# pick two formats
# Keep them close together, so when one column is made larger
# the other column can be adjusted to be the same width
fmt1="%7s %8s %8s %8s %10s %10s %10s %10sn";
# print out the title of each column
fmt2="%7d %8.2f %8.2f %8.2f %10.2f %10.2f %10.2f %10.2fn";
# old AWK wants a backslash at the end of the next line
# to continue the print statement
# new AWK allows you to break the line into two, after a comma
printf(fmt1,"Degrees","Radians","Cosine","Sine",
"Tangent","Cotangent","Secant", "Cosecant");
 
for (i=0;i<=360;i++) {
# convert degrees to radians
r = i * (PI / 180 );
# in new AWK, the backslashes are optional
# in OLD AWK, they are required
printf(fmt2, i, r,
# cosine of r
cos(r),
# sine of r
sin(r),
#
# I ran into a problem when dividing by zero.
# So I had to test for this case.
#
# old AWK finds the next line too complicated
# I don't mind adding a backslash, but rewriting the
# next three lines seems pointless for a simple lesson.
# This script will only work with new AWK, now - sigh...
# On the plus side,
#   I don't need to add those back slashes anymore
#
# tangent of r
(cos(r) == 0) ? BIG : sin(r)/cos(r),
# cotangent of r
(sin(r) == 0) ? BIG : cos(r)/sin(r),
# secant of r
(cos(r) == 0) ? BIG : 1/cos(r),
# cosecant of r
(sin(r) == 0) ? BIG : 1/sin(r));
}
# put an exit here, so that standard input isn't needed.
exit;
}

View file: trigonometry.awk

NAWK also has the arctangent function. This is useful for some graphics work, as

arc tangent(a/b) = angle (in radians)

Therefore if you have the X and Y locations, the arctangent of the ratio will tell you the angle. The atan2() function returns a value from negative pi to positive pi.

Exponents, logs and square roots

The following script uses three other arithmetic functions: log, exp, and sqrt. I wanted to show how these can be used together, so I divided the log of a number by two, which is another way to find a square root. I then compared the value of the exponent of that new log to the built-in square root function. I then calculated the difference between the two, and converted the difference into a positive number.

#!/bin/awk -f
# demonstrate use of exp(), log() and sqrt in AWK
# e.g. what is the difference between using logarithms and regular arithmetic
# note - exp and log are natural log functions - not base 10
#
BEGIN {
# what is the about of error that will be reported?
ERROR=0.000000000001;
# loop a long while
for (i=1;i<=2147483647;i++) {
# find log of i
logi=log(i);
# what is square root of i?
# divide the log by 2
logsquareroot=logi/2;
# convert log of i back
squareroot=exp(logsquareroot);
# find the difference between the logarithmic calculation
# and the built in calculation
diff=sqrt(i)-squareroot;
# make difference positive
if (diff < 0) {
  diff*=-1;
  }
if (diff > ERROR) {
printf("%10d, squareroot: %16.8f, error: %16.14fn",
i, squareroot, diff);
  }
}
exit;
}

View file: awk_example13.awk Yawn. This example isn't too exciting, except to those who enjoy nitpicking. Expect the program to reach 3 million before you see any errors. I'll give you a more exciting sample soon.

Truncating Integers

All version of AWK contain the int function. This truncates a number, making it an integer. It can be used to round numbers by adding 0.5:

printf("rounding %8.4f gives %8dn", x, int(x+0.5));

Random Numbers

NAWK has functions that can generate random numbers. The function rand returns a random number between 0 and 1. Here is an example that calculates a million random numbers between 0 and 100, and counts how often each number was used:

#!/usr/bin/nawk -f
# old AWK doesn't have rand() and srand()
# only new AWK has them
# how random is the random function?
BEGIN {
#  srand();
i=0;
while (i++<1000000) {
x=int(rand()*100 + 0.5);
y[x]++;
}
for (i=0;i<=100;i++) {
printf("%dt%dn",y[i],i);
}
exit;
}

View file: random.awk

If you execute this script several times, you will get the exact same results. Experienced programmers know random number generators aren't really random, unless they use special hardware. These numbers are pseudo-random, and calculated using some algorithm. Since the algorithm is fixed, the numbers are repeatable unless the numbers are seeded with a unique value. This is done using the srand function above, which is commented out. Typically the random number generator is not given a special seed until the bugs have been worked out of the program. There's nothing more frustrating than a bug that occurs randomly. The srand function may be given an argument. If not, it uses the current time and day to generate a seed for the random number generator.

The Lotto script

I promised a more useful script. This may be what you are waiting for. It reads two numbers, and generates a list of random numbers. I call the script "lotto.awk."

#!/usr/bin/nawk -f
BEGIN {
# Assume we want 6 random numbers between 1 and 36
# We could get this information by reading standard input,
# but this example will use a fixed set of parameters.
#
# First, initialize the seed
srand();
# How many numbers are needed?
NUM=6;
# what is the minimum number
MIN=1;
# and the maximum?
MAX=36;
# How many numbers will we find? start with 0
Number=0;
while (Number < NUM) {
r=int(((rand() *(1+MAX-MIN))+MIN));
# have I seen this number before?
if (array[r] == 0) {
# no, I have not
Number++;
array[r]++;
}
}
 
# now output all numbers, in order
for (i=MIN;i<=MAX;i++) {
# is it marked in the array?
if (array[i]) {
# yes
printf("%d ",i);
}
}
printf("n");
exit;
}

View file: lotto.awk If you do win a lottery, send me a postcard.

String Functions

Besides numeric functions, there are two other types of function: strings and the whatchamacallits. First, a list of the string functions:

+-------------------------------------------------+
|      AWK Table 10           |
|          String Functions           |
|Name             Variant         |
+-------------------------------------------------+
|index(string,search)         AWK, NAWK, GAWK |
|length(string)           AWK, NAWK, GAWK |
|split(string,array,separator)    AWK, NAWK, GAWK |
|substr(string,position)         AWK, NAWK, GAWK |
|substr(string,position,max)      AWK, NAWK, GAWK |
|sub(regex,replacement)         NAWK, GAWK      |
|sub(regex,replacement,string)    NAWK, GAWK      |
|gsub(regex,replacement)         NAWK, GAWK      |
|gsub(regex,replacement,string)   NAWK, GAWK      |
|match(string,regex)         NAWK, GAWK      |
|tolower(string)           GAWK         |
|toupper(string)           GAWK         |
+-------------------------------------------------+

Most people first use AWK to perform simple calculations. Associative arrays and trigonometric functions are somewhat esoteric features, that new users embrace with the eagerness of a chain smoker in a fireworks factory. I suspect most users add some simple string functions to their repertoire once they want to add a little more sophistication to their AWK scripts. I hope this column gives you enough information to inspire your next effort.

There are four string functions in the original AWK: index(), length(), split(), and substr(). These functions are quite versatile.

The Length function

What can I say? The length() function calculates the length of a string. I often use it to make sure my input is correct. If you wanted to ignore empty lines, check the length of the each line before processing it with

if (length($0) > 1) {
 . . .
 }

You can easily use it to print all lines longer than a certain length, etc. The following command centers all lines shorter than 80 characters:

#!/bin/awk -f
 {
 if (length($0) < 80) {
 prefix = "";
 for (i = 1;i<(80-length($0))/2;i++)
 prefix = prefix " ";
 print prefix $0;
 } else {
 print;
 } }

View file: center.awk

The Index Function

If you want to search for a special character, the index() function will search for specific characters inside a string. To find a comma, the code might look like this:

sentence="This is a short, meaningless sentence.";
 if (index(sentence, ",") > 0) {
 printf("Found a comma in position %dn", index(sentence,","));
 }

The function returns a positive value when the substring is found. The number specified the location of the substring.

If the substring consists of 2 or more characters, all of these characters must be found, in the same order, for a non-zero return value. Like the length() function, this is useful for checking for proper input conditions.

The Substr function

The substr() function can extract a portion of a string. One common use is to split a string into two parts based on a special character. If you wanted to process some mail addresses, the following code fragment might do the job:

#!/bin/awk -f
 {
 # field 1 is the e-mail address - perhaps
 if ((x=index($1,"@")) > 0) {
 username = substr($1,1,x-1);
 hostname = substr($1,x+1,length($1));
 # the above is the same as
 #        hostname = substr($1,x+1);
 printf("username = %s, hostname = %sn", username, hostname);
 } }

View file: email.awk The substr() function takes two or three arguments. The first is the string, the second is the position. The optional third argument is the length of the string to extract. If the third argument is missing, the rest of the string is used.

The substr function can be used in many non-obvious ways. As an example, it can be used to convert upper case letters to lower case.

#!/usr/bin/awk -f
# convert upper case letters to lower case
BEGIN {
LC="abcdefghijklmnopqrstuvwxyz";
UC="ABCDEFGHIJKLMNOPQRSTUVWXYZ";
}
{
out="";
# look at each character
for(i=1;i<=length($0);i++) {
# get the character to be checked
char=substr($0,i,1);
# is it an upper case letter?
j=index(UC,char);
if (j > 0 ) {
# found it
out = out substr(LC,j,1);
} else {
out = out char;
}
}
printf("%sn", out);
}

View file: upper_to_lower.awk

GAWK's Tolower and Toupper function

GAWK has the toupper() and tolower() functions, for convenient conversions of case. These functions take strings, so you can reduce the above script to a single line:

#!/usr/local/bin/gawk -f
 {
 print tolower($0); }

View file: upper_to_lower.gawk

The Split function

Another way to split up a string is to use the split() function. It takes three arguments: the string, an array, and the separator. The function returns the number of pieces found. Here is an example:

#!/usr/bin/awk -f
 BEGIN {
 # this script breaks up the sentence into words, using
 # a space as the character separating the words
 string="This is a string, is it not?";
 search=" ";
 n=split(string,array,search);
 for (i=1;i<=n;i++) {
 printf("Word[%d]=%sn",i,array[i]);
 }
 exit; }

View file: awk_example14.sh

The third argument is typically a single character. If a longer string is used, only the first letter is used as a separator.

NAWK's string functions

NAWK (and GAWK) have additional string functions, which add a primitive SED-like functionality: sub(), match(), and gsub().

Sub() performs a string substitution, like sed. To replace "old" with "new" in a string, use

sub(/old/, "new", string)

If the third argument is missing, $0 is assumed to be string searched. The function returns 1 if a substitution occurs, and 0 if not. If no slashes are given in the first argument, the first argument is assumed to be a variable containing a regular expression. The sub() only changes the first occurrence. The gsub() function is similar to the g option in sed: all occurrence are converted, and not just the first. That is, if the patter occurs more than once per line (or string), the substitution will be performed once for each found pattern. The following script:

#!/usr/bin/nawk -f
 BEGIN {
 string = "Another sample of an example sentence";
 pattern="[Aa]n";
 if (gsub(pattern,"AN",string)) {
 printf("Substitution occurred: %sn", string);
 }
 
 exit; }

View file: awk_example15.awk

print the following when executed:

Substitution occurred: ANother sample of AN example sentence

As you can see, the pattern can be a regular expression.

The Match function

As the above demonstrates, the sub() and gsub() returns a positive value if a match is found. However, it has a side-effect of changing the string tested. If you don't wish this, you can copy the string to another variable, and test the spare variable. NAWK also provides the match() function. If match() finds the regular expression, it sets two special variables that indicate where the regular expression begins and ends. Here is an example that does this:

#!/usr/bin/nawk -f
 # demonstrate the match function
 
 BEGIN {
 regex="[a-zA-Z0-9]+";
 }
 {
 if (match($0,regex)) {
 #           RSTART is where the pattern starts
 #           RLENGTH is the length of the pattern
 before = substr($0,1,RSTART-1);
 pattern = substr($0,RSTART,RLENGTH);
 after = substr($0,RSTART+RLENGTH);
 printf("%s<%s>%sn", before, pattern, after);
 } }

View file: awk_example16.awk

Lastly, there are the whatchamacallit functions. I could use the word "miscellaneous," but it's too hard to spell. Darn it, I had to look it up anyway.

+-----------------------------------------------+
|      AWK Table 11          |
|      Miscellaneous Functions        |
|Name            Variant        |
+-----------------------------------------------+
|getline            AWK, NAWK, GAWK |
|getline <file          NAWK, GAWK      |
|getline variable          NAWK, GAWK      |
|getline variable <file      NAWK, GAWK      |
|"command" | getline        NAWK, GAWK      |
|"command" | getline variable   NAWK, GAWK      |
|system(command)          NAWK, GAWK      |
|close(command)          NAWK, GAWK      |
|systime()          GAWK        |
|strftime(string)          GAWK        |
|strftime(string, timestamp)    GAWK        |
+-----------------------------------------------+

The System function

NAWK has a function system() that can execute any program. It returns the exit status of the program.

if (system("/bin/rm junk") != 0)
 print "command didn't work";

The command can be a string, so you can dynamically create commands based on input. Note that the output isn't sent to the NAWK program. You could send it to a file, and open that file for reading. There is another solution, however.

The Getline function

AWK has a command that allows you to force a new line. It doesn't take any arguments. It returns a 1, if successful, a 0 if end-of-file is reached, and a -1 if an error occurs. As a side effect, the line containing the input changes. This next script filters the input, and if a backslash occurs at the end of the line, it reads the next line in, eliminating the backslash as well as the need for it.

#!/usr/bin/awk -f
 # look for a  as the last character.
 # if found, read the next line and append
 {
 line = $0;
 while (substr(line,length(line),1) == "") {
 # chop off the last character
 line = substr(line,1,length(line)-1);
 i=getline;
 if (i > 0) {
 line = line $0;
 } else {
 printf("missing continuation on line %dn", NR);
 }
 }
 print line; }

View file: awk_example17.awk

Instead of reading into the standard variables, you can specify the variable to set:

getline a_line
 print a_line;

NAWK and GAWK allow the getline function to be given an optional filename or string containing a filename. An example of a primitive file preprocessor, that looks for lines of the format

#include filename

and substitutes that line for the contents of the file:

#!/usr/bin/nawk -f
 {
 # a primitive include preprocessor
 if (($1 == "#include") && (NF == 2)) {
 # found the name of the file
 filename = $2;
 while (i = getline < filename ) {
 print;
 }
 } else {
 print;
 } }

View file: include.nawk

NAWK's getline can also read from a pipe. If you have a program that generates single line, you can use

"command" | getline;
 print $0;

or

"command" | getline abc;
 print abc;

If you have more than one line, you can loop through the results:

while ("command" | getline) {
cmd[i++] = $0;
}
for (i in cmd) {
printf("%s=%sn", i, cmd[i]);
}

Only one pipe can be open at a time. If you want to open another pipe, you must execute

close("command");

This is necessary even if the end of file is reached.

The systime function

The systime() function returns the current time of day as the number of seconds since Midnight, January 1, 1970. It is useful for measuring how long portions of your GAWK code takes to execute.

#!/usr/local/bin/gawk -f
# how long does it take to do a few loops?
BEGIN {
LOOPS=100;
# do the test twice
start=systime();
for (i=0;i<= 0) {
printf("%d loops were not enough to test, increase itn",
LOOPS);
exit;
} else {
printf("%d loops took %6.4f seconds to executen",
LOOPS, newtime);
printf("That's %10.8f seconds per loopn",
(newtime)/LOOPS);
# since the clock has an accuracy of +/- one second, what is the error
printf("accuracy of this measurement = %6.2f%%n",
(1/(newtime))*100);
}
exit;
}

View file: awk_example17.gawk

The Strftime function

GAWK has a special function for creating strings based on the current time. It's based on the strftime(3c) function. If you are familiar with the "+" formats of the date(1) command, you have a good head-start on understanding what the strftime command is used for. The systime() function returns the current date in seconds. Not very useful if you want to create a string based on the time. While you could convert the seconds into days, months, years, etc., it would be easier to execute "date" and pipe the results into a string. (See the previous script for an example). GAWK has another solution that eliminates the need for an external program.

The function takes one or two arguments. The first argument is a string that specified the format. This string contains regular characters and special characters. Special characters start with a backslash or the percent character. The backslash characters with the backslash prefix are the same I covered earlier. In addition, the strftime() function defines dozens of combinations, all of which start with "%." The following table lists these special sequences:

+---------------------------------------------------------------+
|        AWK Table 12        |
|        GAWK's strftime formats      |
+---------------------------------------------------------------+
|%a   The locale's abbreviated weekday name      |
|%A   The locale's full weekday name        |
|%b   The locale's abbreviated month name      |
|%B   The locale's full month name        |
|%c   The locale's "appropriate" date and time representation  |
|%d   The day of the month as a decimal number (01--31)    |
|%H   The hour (24-hour clock) as a decimal number (00--23)  |
|%I   The hour (12-hour clock) as a decimal number (01--12)  |
|%j   The day of the year as a decimal number (001--366)  |
|%m   The month as a decimal number (01--12)      |
|%M   The minute as a decimal number (00--59)      |
|%p   The locale's equivalent of the AM/PM      |
|%S   The second as a decimal number (00--61).      |
|%U   The week number of the year (Sunday is first day of week) |
|%w   The weekday as a decimal number (0--6).  Sunday is day 0  |
|%W   The week number of the year (Monday is first day of week) |
|%x   The locale's "appropriate" date representation    |
|%X   The locale's "appropriate" time representation    |
|%y   The year without century as a decimal number (00--99)  |
|%Y   The year with century as a decimal number      |
|%Z   The time zone name or abbreviation      |
|%%   A literal %.            |
+---------------------------------------------------------------+

Depending on your operating system, and installation, you may also have the following formats:

+-----------------------------------------------------------------------+
|            AWK Table 13        |
|         Optional GAWK strftime formats      |
+-----------------------------------------------------------------------+
|%D   Equivalent to specifying %m/%d/%y          |
|%e   The day of the month, padded with a blank if it is only one digit |
|%h   Equivalent to %b, above            |
|%n   A newline character (ASCII LF)          |
|%r   Equivalent to specifying %I:%M:%S %p        |
|%R   Equivalent to specifying %H:%M          |
|%T   Equivalent to specifying %H:%M:%S          |
|%t   A TAB character              |
|%k   The hour as a decimal number (0-23)        |
|%l   The hour (12-hour clock) as a decimal number (1-12)    |
|%C   The century, as a number between 00 and 99      |
|%u   is replaced by the weekday as a decimal number [Monday == 1]  |
|%V   is replaced by the week number of the year (using ISO 8601)  |
|%v   The date in VMS format (e.g. 20-JUN-1991)        |
+-----------------------------------------------------------------------+

One useful format is

strftime("%y_%m_%d_%H_%M_%S")

This constructs a string that contains the year, month, day, hour, minute and second in a format that allows convenient sorting. If you ran this at noon on Christmas, 1994, it would generate the string

94_12_25_12_00_00

Here is the GAWK equivalent of the date command:

#! /usr/local/bin/gawk -f
 #
 
 BEGIN {
 format = "%a %b %e %H:%M:%S %Z %Y";
 print strftime(format); }

View file: date.gawk

You will note that there is no exit command in the begin statement. If I was using AWK, an exit statement is necessary. Otherwise, it would never terminate. If there is no action defined for each line read, NAWK and GAWK do not need an exit statement.

If you provide a second argument to the strftime() function, it uses that argument as the timestamp, instead of the current system's time. This is useful for calculating future times. The following script calculates the time one week after the current time:

#!/usr/local/bin/gawk -f
 BEGIN {
 #  get current time
 ts = systime();
 # the time is in seconds, so
 one_day = 24 * 60 * 60;
 next_week = ts + (7 * one_day);
 format = "%a %b %e %H:%M:%S %Z %Y";
 print strftime(format, next_week);
 exit; }

View file: one_week_later.gawk

User Defined Functions

Finally, NAWK and GAWK support user defined functions. This function demonstrates a way to print error messages, including the filename and line number, if appropriate:

#!/usr/bin/nawk -f
 {
 if (NF != 4) {
 error("Expected 4 fields");
 } else {
 print;
 }
 }
 function error ( message ) {
 if (FILENAME != "-") {
 printf("%s: ", FILENAME) > "/dev/tty";
 }
 printf("line # %d, %s, line: %sn", NR, message, $0) >> "/dev/tty"; }

View file: awk_example18.nawk

AWK patterns

In my first tutorial on AWK, I described the AWK statement as having the form

pattern {commands}

I have only used two patterns so far: the special words BEGIN and END. Other patterns are possible, yet I haven't used any. There are several reasons for this. The first is that these patterns aren't necessary. You can duplicate them using an if statement. Therefore this is an "advanced feature." Patterns, or perhaps the better word is conditions, tend to make an AWK program obscure to a beginner. You can think of them as an advanced topic, one that should be attempted after becoming familiar with the basics.

A pattern or condition is simply an abbreviated test. If the condition is true, the action is performed. All relational tests can be used as a pattern. The "tail -10" command, which prints the first 10 lines and stops, can be duplicated with

{if (NR <= 10 ) {print}}

Changing the if statement to a condition shortens the code:

NR <= 10 {print}

Besides relational tests, you can also use containment tests, i. e. do strings contain regular expressions? Printing all lines that contain the word "special" can be written as

{if ($0 ~ /special/) {print}}

or more briefly

$0 ~ /special/ {print}

This type of test is so common, the authors of AWK allow a third, shorter format:

/special/ {print}

These tests can be combined with the AND (&&) and OR (||) commands, as well as the NOT (!) operator. Parenthesis can also be added if you are in doubt, or to make your intention clear.

The following condition prints the line if it contains the word "whole" or columns 1 and 2 contain "part1" and "part2" respectively.

($0 ~ /whole/) || (($1 ~ /part1/) && ($2 ~ /part2/)) {print}

This can be shortened to

/whole/ || $1 ~ /part1/ && $2 ~ /part2/ {print}

There is one case where adding parenthesis hurts. The condition

/whole/ {print}

works, but

(/whole/) {print}

does not. If parenthesis are used, it is necessary to explicitly specify the test:

($0 ~ /whole) {print}

A murky situation arises when a simple variable is used as a condition. Since the variable NF specifies the number of fields on a line, one might think the statement

NF {print}

would print all lines with one of more fields. This is an illegal command for AWK, because AWK does not accept variables as conditions. To prevent a syntax error, I had to change it to

NF != 0 {print}

I expected NAWK to work, but on some SunOS systems it refused to print any lines at all. On newer Solaris systems it did behave properly. Again, changing it to the longer form worked for all variations. GAWK, like the newer version of NAWK, worked properly. After this experience, I decided to leave other, exotic variations alone. Clearly this is unexplored territory. I could write a script that prints the first 20 lines, except if there were exactly three fields, unless it was line 10, by using

NF == 3 ? NR == 10 : NR < 20 { print }

But I won't. Obscurity, like puns, is often unappreciated.

There is one more common and useful pattern I have not yet described. It is the comma separated pattern. A common example has the form:

/start/,/stop/ {print}

This form defines, in one line, the condition to turn the action on, and the condition to turn the action off. That is, when a line containing "start" is seen, it is printed. Every line afterwards is also printed, until a line containing "stop" is seen. This one is also printed, but the line after, and all following lines, are not printed. This triggering on and off can be repeated many times. The equivalent code, using the if command, is:

{
 if ($0 ~ /start/) {
 triggered=1;
 }
 if (triggered) {
 print;
 if ($0 ~ /stop/) {
 triggered=0;
 }
 }
 }

The conditions do not have to be regular expressions. Relational tests can also be used. The following prints all lines between 20 and 40:

(NR==20),(NR==40) {print}

You can mix relational and containment tests. The following prints every line until a "stop" is seen:

(NR==1),/stop/ {print}

There is one more area of confusion about patterns: each one is independent of the others. You can have several patterns in a script; none influence the other patterns. If the following script is executed:

NR==10 {print}
 (NR==5),(NR==15) {print}
 /xxx/ {print}
 (NR==1),/NeVerMatchThiS/ {print}

and the input file's line 10 contains "xxx," it would be printed 4 times, as each condition is true. You can think of each condition as cumulative. The exception is the special BEGIN and END conditions. In the original AWK, you can only have one of each. In NAWK and GAWK, you can have several BEGIN or END actions.

Formatting AWK programs

Many readers have questions my style of AWK programming. In particular, they ask me why I include code like this:

# Print column 3
 print $3;

when I could use

print $3 # print column 3

After all, they reason, the semicolon is unnecessary, and comments do not have to start on the first column. This is true. Still I avoid this. Years ago, when I started writing AWK programs, I would find myself confused when the nesting of conditions were too deep. If I moved a complex if statement inside another if statement, my alignment of braces became incorrect. It can be very difficult to repair this condition, especially with large scripts. Nowadays I use emacs to do the formatting for me, but 10 years ago I didn't have this option. My solution was to use the program cb, which is a "C beautifier." By including optional semicolons, and starting comments on the first column of the line, I could send my AWK script through this filter, and properly align all of the code.

Environment Variables

I've described the 7 special variables in AWK, and briefly mentioned some others NAWK and GAWK. The complete list follows:

+--------------------------------+
|         AWK Table 14        |
|Variable      AWK    NAWK   GAWK |
+--------------------------------+
|FS      Yes    Yes   Yes  |
|NF      Yes    Yes   Yes  |
|RS      Yes    Yes   Yes  |
|NR      Yes    Yes   Yes  |
|FILENAME      Yes    Yes   Yes  |
|OFS      Yes    Yes   Yes  |
|ORS      Yes    Yes   Yes  |
+--------------------------------+
|ARGC      Yes   Yes  |
|ARGV      Yes   Yes  |
|ARGIND       Yes  |
|FNR      Yes   Yes  |
|OFMT      Yes   Yes  |
|RSTART      Yes   Yes  |
|RLENGTH      Yes   Yes  |
|SUBSEP      Yes   Yes  |
|ENVIRON       Yes  |
|IGNORECASE     Yes  |
|CONVFMT       Yes  |
|ERRNO       Yes  |
|FIELDWIDTHS     Yes  |
+--------------------------------+

Since I've already discussed many of these, I'll only cover those that I missed earlier.

ARGC - Number or arguments (NAWK/GAWK)

The variable ARGC specifies the number of arguments on the command line. It always has the value of one or more, as it counts its own program name as the first argument. If you specify an AWK filename using the "-f" option, it is not counted as an argument. If you include a variable on the command line using the form below, NAWK does not count as an argument.

nawk -f file.awk x=17

GAWK does, but that is because GAWK requires the "-v" option before each assignment:

gawk -f file.awk -v x=17

GAWK variables initialized this way do not affect the ARGC variable.

ARGV - Array of arguments (NAWK/GAWK)

The ARGV array is the list of arguments (or files) passed as command line arguments.

ARGIND - Argument Index (GAWK only)

The ARGIND specifies the current index into the ARGV array, and therefore ARGV[ARGIND] is always the current filename. Therefore

FILENAME == ARGV[ARGIND]

is always true. It can be modified, allowing you to skip over files, etc.

FNR (NAWK/GAWK)

The FNR variable contains the number of lines read, but is reset for each file read. The NR variable accumulates for all files read. Therefore if you execute an awk script with two files as arguments, with each containing 10 lines:

nawk '{print NR}' file file2
 nawk '{print FNR}' file file2

the first program would print the numbers 1 through 20, while the second would print the numbers 1 through 10 twice, once for each file.

OFMT (NAWK/GAWK)

The OFMT variable Specifies the default format for numbers. The default value is "%.6g."

RSTART, RLENGTH and match (NAWK/GAWK)

I've already mentioned the RSTART and RLENGTH variables. After the match() function is called, these variables contain the location in the string of the search pattern. RLENGTH contains the length of this match.

SUBSEP - Multi-dimensional array separator (NAWK/GAWK)

Earlier I described how you can construct multi-dimensional arrays in AWK. These are constructed by concatenating two indexes together with a special character between them. If I use an ampersand as the special character, I can access the value at location X, Y by the reference

array[ X "&" Y ]

NAWK (and GAWK) has this feature built in. That is, you can specify the array element

array[X,Y]

It automatically constructs the string, placing a special character between the indexes. This character is the non-printing character "034." You can control the value of this character, to make sure your strings do not contain the same character.

ENVIRON - environment variables (GAWK only)

The ENVIRON array contains the environment variables the current process. You can print your current search path using

print ENVIRON["PATH"]

IGNORECASE (GAWK only)

The IGNORECASE variable is normally zero. If you set it to non-zero, then all pattern matches ignore case. Therefore the following is equivalent to "grep -i match:"

BEGIN {IGNORECASE=1;}
 /match/ {print}

CONVFMT - conversion format (GAWK only)

The CONVFMT variable is used to specify the format when converting a number to a string. The default value is "%.6g." One way to truncate integers is to convert an integer to a string, and convert the string to an integer - modifying the actions with CONVFMT:

a = 12;
 b = a "";
 CONVFMT = "%2.2f";
 c = a "";

Variables b and c are both strings, but the first one will have the value "12.00" while the second will have the value "12."

ERRNO - system errors (GAWK only)

The ERRNO variable describes the error, as a string, after a call to the getline command fails.

FIELDWIDTHS - fixed width fields (GAWK only)

The FIELDWIDTHS variable is used when processing fixed width input. If you wanted to read a file that had 3 columns of data; the first one is 5 characters wide, the second 4, and the third 7, you could use substr to split the line apart. The technique, using FIELDWIDTHS, would be:

BEGIN {FIELDWIDTHS="5 4 7";}
 { printf("The three fields are %s %s %sn", $1, $2, $3);}

AWK, NAWK, GAWK, or PERL

This concludes my series of tutorials on AWK and the NAWK and GAWK variants. I originally intended to avoid extensive coverage of NAWK and GAWK. I am a PERL user myself, and still believe PERL is worth learning, yet I don't intend to cover this very complex language. Understanding AWK is very useful when you start to learn PERL, and a part of me felt that teaching you extensive NAWK and GAWK features might encourage you to bypass PERL.

I found myself going into more depth than I planned, and I hope you found this useful. I found out a lot myself, especially when I discussed topics not covered in other books. Which reminds me of some closing advice: if you don't understand how something works, experiment and see what you can discover. Good luck, and happy AWKing...

Other of my Unix shell tutorials can be found here. Other shell tutorials can be found at Heiner's SHELLdorado and Chris F. A. Johnson's Unix Shell Page This document was translated by troff2html v0.21 on September 22, 2001 and then manually edited to make it compliant with:

Awk Tutorial and Introduction originally appeared on AskApache.com