WARNING! This is for advanced users of the shell, this is NOT a howto or tutorial. The thing is, I googled how to uninstall cpanel and for once in my life I came up empty.. And certainly the cpanel official sites themselves don't provide any instructions other than to say "Dont uninstall it, reinstall your entire operating system without it.".. they sure don't seem confident that they know exactly what their code is doing. Cpanel is great for most people, perfect for many situations, so don't get me wrong.. this is just for fun.

Why Uninstall?

Why? Because I have always built my servers, php installations, perl installs, ruby, iptables, everything from source. I read the INSTALL/README docs, I read the man pages, and I read the info pages as well. I google for configuration advice, I google for tips, and I don't need a web-based perl script messing my stuff up!

The main problems I had with cpanel, which really is a great bit of software for millions of website developers, is that it was incredibly sneaky! I used it for about 6 months and spent that entire time trying to figure out what the heck it was doing. A couple issues that I really disliked, it takes over your bind install, it takes over your apache install, it takes over your php install. And although it does let you configure some things (very few) for custom configurations and the like, I just don't need any of that. By removing the darn thing I am saving GIGS of space on my server, tons of bandwidth, and most importantly to me I am saving CPU and processing time along with RAM and IO speed.

Anything Else

Please.. I could go on for DAYS! Another reason I have wanted to be rid of cpanel is that I like my servers to be as lean and mean as possible. This means I like as few files and processes as possible. There are many benefits to this, like it's much easier for my integrity checking software and rootkit/antivirus software to run and drastically reduces the false positives. And there is that glaring security issue of constantly having cpanel run it's own software to create the WHM/Cpanel web interface, which is accessible online. I haven't researched cpanel security at all, it's possible that it never has security problems that are published, but for me, why take the chance?

Warning - Caution!

The following is just a quick reference of some of the commands I used to uninstall cpanel. This is for advanced users of the shell. If you aren't that advanced and you run a single one of these commands without fully understanding it, you will probably kill your server, probably lose everything on it permanently, probably not have a website or email for weeks.. So backup all your data FIRST. Also, if you aren't 100% sure you won't run into problems, you should contact your hosts technical support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for hosts, and the last thing tech support wants is to fix a server broken by someone who doesn't know what they are doing.

Last Warning!

Unless you understand what theses commands do and the purpose they serve, do not try any of this. These are not the exact commands I used verbatim, they are also not in order. I only put them up here because I was so amazed that google didn't have any uninstall cpanel intructions. Hopefully it's not a conspiracy that will get my site taken down.. ;)

That said and out of the way, it really only took me about 10 minutes to uninstall cpanel completely. But keep in mind I have been closely monitoring and debugging cpanel for 6 months, so I knew what I was doing. And finally, I do apologize for not having better instructions.. but hey, if you don't get this then you have no business trying to figure out how to uninstall cpanel! It's great software and shouldn't be removed unless you are fully capable of managing email/dns/www/ftp and any/all other servers and services on your machine by hand.

Do This First

I have a few drafts I'm working on at the moment with specifics, but for now you will have to figure it out with google. Basically you want to make sure you don't totally knock your machine offline without being able to reconnect. What I do is compile a static version of openssh and a few other security-type shell tools, and configure this binary sshd to run by using inittab, which is the file run by init (pid 1) and makes sure if it dies it is restarted. Here is my /etc/inittab to run the static sshd binary:

hh:12345:respawn:/failover/os/sbin/aassh -D -q -u0 -f /failover/os/etc/aassh

Another trick is to keep a detached screen logged in to root. That way if you mess up your sudoers or securetty or pam or whatever, you can just reattach and fix it.

Finally, you may want to setup your syslog to start earlier than usual, and set up more than normal verbosity. ( I take it to the max ). Then you should setup a 2nd server or machine somewhere to act as a syslog server. Lastly, configure your web server syslog to copy all messages to the remote syslog you set up. I use a reverse ssh tunnel to encrypt the syslog packets, but when I do something serious like reboot after uninstalling cpanel, I prepare for it by adding additional networking routes on my machine to make sure I will get some logs even if sshd cant start or even if my network addresses aren't brought up correctly.

If that sounds easy to you, please continue. If you are saying: Wha??? Continue in read-only mode.

Also, you can't just uninstall cpanel, I have replaced a lot of cpanel already, like building my own bind, apache, php, syslog and making sure they work and aren't being tampered with by cpanel. Basically cpanel runs everything on your server in most cases, so you should prepare by creating your own static software to replace cpanel, and make sure it works.

Find files Accessing /var/cpanel

More than likely these will need to be killed.

lsof +w -Rg -nP +c15 -x f +D /var/cpanel
lsof +w -Rg -nP +c15 -x f +D /usr/local/cpanel

Killing cpanel

Just an example, your machine may have a lot more than these, I have been slowly taking control of my machine back from cpanel for 6 months, so it was easier for me.

for P in tailwatchd queueprocd cpanellogd exim; do pkill -9 $P; done

Commands and Shortcuts

alias NF='nice find $PWD -mount -depth ! -type d'
alias NFF='nice find $PWD -mount -depth ! -type d | xargs -IF87 file F87'
alias NA='nice find $PWD -mount -depth'
alias NAF='nice find $PWD -mount -depth | xargs -IF87 file F87'

Watch out for crontab

An example of the sneakiness (from my POV, from most it's called builtin robustness) that cpanel does is automagically adding crontab entries that make it behave similarly to a self-propagating virus. If you don't disable the cronjobs and kill the right processes within a short period of time, be prepared for a magic resurrection.

Here's my awesome crontab information function, you will need to check every file, it lists the default crons on my box, and every users crontab, but it can't account for other cron software like at and other crons.

function askapache_crontab()
{
  local GG i;
  for i in `getent passwd|cut -d ":" -f1`;
  do
    GG=$(sudo crontab -u $i -l 2>$N6 | tr -s '\n\000' | sed '/^#/d');
    [[ ${#GG} -gt 3 ]] && sleep 1 && echo -e "$i \n\n${GG}"
  done;
   sleep 4;
   ls -aLls1ch --color=always /etc/cron.{hourly,daily,weekly,monthly,d} | sed '/^total/d; /\ drwxr-xr-x/d';
}

grep -ir /var/spool cpan

#6 3 * * * /scripts/upcp
#0 1 * * * /scripts/cpbackup
#0 2 * * * /scripts/mailman_chown_archives
#35 * * * * /usr/bin/test -x /usr/local/cpanel/bin/tail-check && /usr/local/cpanel/bin/tail-check
#11,26,41,56 * * * * /usr/local/cpanel/whostmgr/bin/dnsqueue > /dev/null 2>&1
#30 */4 * * * /usr/bin/test -x /scripts/update_db_cache && /scripts/update_db_cache
#45 */8 * * * /usr/bin/test -x /usr/local/cpanel/bin/optimizefs && /usr/local/cpanel/bin/optimizefs
#*/5 * * * * /usr/local/cpanel/bin/dcpumon >/dev/null 2>&1
#25 1 * * * /usr/local/cpanel/whostmgr/docroot/cgi/cpaddons_report.pl --notify

Delete Crontabs

sudo crontab -u mailman -r

Find INIT scripts with cpanel

This is the main startup script: /usr/local/cpanel/etc/init/startup

I had no idea ruby-on-rails was being controlled by cpanel.. sneaky bugger. You can tell by all of these advanced unix commands just how difficult it would be to uninstall cpanel, its totally like the Alien!

(1:3744)# find . ! -type d -print0|xargs -0 -I'F87' grep -Hi "cpan\|tailwat\|chkser" F87
./fastmail:# Author:       cPanel, Inc. <nick@cpanel.net>
./httpd:        HTTPD=/usr/local/cpanel/bin/chroothttpd
./cpanel:# cpanel8       Start Cpanel Services
./cpanel:# Author:       cPanel, Inc. <nick@cpanel.net>
./cpanel:# description: This is the cpanel webserver and chat.
./cpanel:# processname: cpaneld
./cpanel:# pidfile: /var/run/cpanel.pid
./cpanel:[ -f /usr/local/cpanel/etc/init/startup ] || exit 0
./cpanel:       if [ -f "/var/cpanel/smtpgidonlytweak" ]; then
./cpanel:       echo -n "Starting cPanel services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpsrvd
./cpanel:       echo -n "Starting cPanel brute force detector services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcphulkd
./cpanel:    echo -n "Starting cPanel dav services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpdavd
./cpanel:               daemon /usr/local/cpanel/etc/init/startcppop
./cpanel:       echo -n "Starting cPanel Chat services: "
./cpanel:               daemon /usr/local/cpanel/entropychat/entropychat
./cpanel:               daemon /usr/local/cpanel/bin/startmelange
./cpanel:                       /usr/local/cpanel/bin/startinterchange
./cpanel:       echo -n "Starting cPanel ssl services: "
./cpanel:       daemon /usr/local/cpanel/startstunnel
./cpanel:    echo -n "Starting cPanel Queue services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startqueueprocd
./cpanel:    echo -n "Starting tailwatchd: "
./cpanel:    daemon /usr/local/cpanel/libexec/tailwatchd --start
./cpanel:       echo -n "Starting cPanel Log services: "
./cpanel:       daemon /usr/local/cpanel/cpanellogd
./cpanel:    action "Starting mailman services: " /usr/local/cpanel/etc/init/startmailman
./cpanel:    action "Stopping tailwatchd: " /usr/local/cpanel/libexec/tailwatchd --stop
./cpanel:    action "Stopping cPanel services: " /usr/local/cpanel/etc/init/stopcpsrvd
./cpanel:       action "Stopping cPanel dav services: " /usr/local/cpanel/etc/init/stopcpdavd
./cpanel:       action "Stopping cPanel queue services: " /usr/local/cpanel/etc/init/stopqueueprocd
./cpanel:       action "Stopping cPanel brute force detector services: " /usr/local/cpanel/etc/init/stopcphulkd
./cpanel:               action "Stopping pop3 services: " /usr/local/cpanel/etc/init/stopcppop
./cpanel:       echo -n "Stopping cPanel log services: "
./cpanel:       killproc cpanellogd
./cpanel:       echo -n "Stopping cPanel Chat services: "
./cpanel:       action "Stopping cPanel ssl services: " /usr/local/cpanel/etc/init/stopstunnel
./cpanel:       action "Stopping mailman services: " /usr/local/cpanel/etc/init/stopmailman
./cpanel:       if [ -e "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl" ]; then
./exim:if [ -e "/etc/chkserv.d" ]; then
./exim:        for file in `ls /etc/chkserv.d`
./exim:            if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./exim:    if [ -x "/usr/local/cpanel/etc/init/startspamd" ]; then
./exim:        /usr/local/cpanel/etc/init/startspamd
./exim:        if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --statusall
./securetmp:# Author:       cPanel, Inc. <copyright@cpanel.net>

Turn off cpanel services

You should remove the below delete command and start by just disabling the inits by turning them off. Then reboot. Then delete. If your machine won't reboot, I told you so, Cpanel told you so, and likely your host told you so.

for S in cpanel ror securetmp fastmail exim; do R=$(command chkconfig --level 123456 $S off ||echo); R=$(command chkconfig --del $S ||echo); done

HTTPD

If you are running chrooted httpd then you'll need to make sure you don't delete your entire webserver on accident. Here's a relevant part from the /etc/init.d/httpd script.

# the path to your httpd binary, including options if necessary
if [ -e "/etc/chroothttpd" ]; then
        HTTPD=/usr/local/cpanel/bin/chroothttpd
else
        HTTPD=/usr/local/apache/bin/httpd
fi

Finding files owned by cpanel

Some super cool bash commands in this post.. let's start with one to find all the files and folders on your machine owned by cpanel. Check your /etc/passwd file for your machines specific usernames and groups. This command saves all the filenames to ~/cpanel-files-backup.txt, which is used by tar next to create a backup of all of them.

{ find / -mount -depth -maxdepth 150 \( -group cpanel -o   -group cpanel-phpmyadmin -o -group cpanel-phppgadmin   -o -group cpanelphpmyadmin   -o -group cpanelphppgadmin   -o -group cpanelhorde   -o -group cpanelroundcube \) -print; find / -mount -depth -maxdepth 150 \( -user cpanel -o   -user cpanel-phpmyadmin -o -user cpanel-phppgadmin   -o -user cpanelphpmyadmin   -o -user cpanelphppgadmin   -o -user cpanelhorde   -o -user cpanelroundcube \) -print; } > ~/cpanel-files-backup.txt

Here's another way to search directories.

 grep --color=always -Hir cpanel /var

Create the Backup

Note that you must have the latest version of tar for this exact command, also you should backup /var/cpanel and /usr/local/cpanel and /etc and heck the whole machine why dontcha!

tar -T ~/cpanel-files-backup.txt -cvz --checkpoint=1000 --checkpoint-action="ttyout=\rHit %s checkpoint #%u" -f /cpanel-files-backup.tgz --totals

Remove Files

Once you do this your upstream without a paddle, you better make sure you know what you're doing with this. This removes all those files.

cat ~/cpanel-files-backup.txt | xargs -I'F87' rm -vfr F87

Additionally you will want to remove /usr/local/cpanel and /var/cpanel - What I always do when running as root is alias my rm command to instead simply move the files to a .trash folder. That way if something goes bork you have a better chance at fixing it.

Find Group-Owned Files

find / -mount -depth -maxdepth 150 \
\(  -group cpanel -o \
  -group cpanel-phpmyadmin \
  -o -group cpanel-phppgadmin \
  -o -group cpanelphpmyadmin \
  -o -group cpanelphppgadmin \
  -o -group mailman \
  -o -group cpanelhorde \
  -o -group cpanelroundcube \
\) -fprintf /root/cpanel-group-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'

Find User-Owned Files

find / -mount -depth -maxdepth 150 \(
  -user cpanel \
  -o -user cpanel-phpmyadmin \
  -o -user cpanel-phppgadmin \
  -o -user cpanelphpmyadmin \
  -o -user cpanelphppgadmin \
  -o -user mailman \
  -o -user cpanelhorde \
  -o -user cpanelroundcube
\) -fprintf /root/cpanel-users-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'

       4  0755  drwxr-xr-x     cpanel:cpanel     32002:32004 /var/cpanel/userhomes/cpanel cpanel d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/tmp tmp d reiserfs
       4  0644  -rw-r--r-- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin cpanel-phpmyadmin d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/sessions sessions d reiserfs
       4  0644  -rw-r--r-- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin cpanelphppgadmin d reiserfs
       4  0750  drwxr-x--- cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/mail mail d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/sessions sessions d reiserfs
       4  0711  drwx--x--x cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube cpanelroundcube d reiserfs
       4  0644  -rw-r--r--     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches caches d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel .cpanel d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/tmp tmp d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/log log d reiserfs

Find Permissions

cat ~/cpanel-group-files.log ~/cpanel-users-files.log |tr -s '\000 \t'|cut -d' ' -f3|sort -u

Find files tailwatchd

(1:3732)# $NICE find ${1:-`pwd`} -mount -name '*tailwatch*'
/usr/local/cpanel/libexec/tailwatchd
/usr/local/cpanel/libexec/tailwatch
/usr/local/cpanel/libexec/tailwatch/tailwatchd
/usr/local/cpanel/etc/init/scripts/freebsd/tailwatchd.sh
/usr/local/cpanel/etc/init/scripts/trustix/tailwatchd
/usr/local/cpanel/etc/init/scripts/centos/tailwatchd
/usr/local/cpanel/etc/init/scripts/suse/tailwatchd
/usr/local/cpanel/etc/init/scripts/caos/tailwatchd
/usr/local/cpanel/etc/init/scripts/whitebox/tailwatchd
/usr/local/cpanel/etc/init/scripts/mandrake/tailwatchd
/usr/local/cpanel/etc/init/scripts/debian/tailwatchd
/usr/local/cpanel/etc/init/scripts/redhat/tailwatchd
/usr/local/cpanel/etc/init/scripts/fedora/tailwatchd
/usr/local/cpanel/etc/init/stoptailwatchd
/usr/local/cpanel/etc/init/starttailwatchd
/usr/local/cpanel/bin/tailwatchd
/usr/local/cpanel/logs/tailwatchd_log
/var/log/cpanel/tailwatchd_log
/var/cpanel/log_rotation/cp_tailwatchd_log.cpanellogd
/var/cpanel/tailwatch.positions
/var/run/tailwatchd.pid
/etc/chkserv.d/tailwatchd
/scripts/restartsrv_tailwatchd

Delete cpanel Users/Groups

for U in cpanel-phpmyadmin cpanel-phppgadmin cpanelphpmyadmin cpanelphppgadmin cpanelhorde cpanelroundcube machbuild; do userdel -fr $U; groupdel $U; done

Check for broken symlinks

find / -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'

Especially check /etc

$ find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'
/etc/ftpd-rsa.pem                   broken symbolic link to `/var/cpanel/ssl/ftp/ftpd-rsa.pem'
/etc/rc.d/rc1.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc1.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc1.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc3.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc6.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc5.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc2.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc2.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc2.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc4.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc4.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc4.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc0.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/authlib/authProg               broken symbolic link to `/usr/local/cpanel/bin/courier-auth'

And delete if you are sure

find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p' |cut -d' ' -f1|xargs -I'F87' rm -rvf 'F87'

Reinstall CSF

The only thing I actually used that came with cpanel is the CSF/LFD Firewall package, which is a fantastic piece of software. I had to reinstall this, and to get it working without cpanel add the following line to the csf.conf

GENERIC = "1"

Thats It

Now once you've cleaned up everything, you should try everything conceivable to get an error before rebooting. Like you should start and stop every service in /etc/init.d/, you should use telinit to check various runlevels (which keeps your sshd connection still live). Go all out, should take at least a full hour.

Another thing I like to do is rebuild alot of my source-built software again in case anything got messed up. I upgrade perl from cpanels 5.8.8 to 5.10, which is pretty thorough, and you know, reinstall anything else I think I might need. One of the benefits of compiling your own software is all I have to do is cd to the source directory and type make -B && ( { make test || make check || make checks || make tests; } || echo ) && sudo make install and that's it. The tests/checks are optional of course.

If anyone actually ever reads this and does it, please share your advice here.. everybody knows we need it! Good Luck

HOWTO: Uninstall CPANEL over SSH originally appeared on AskApache.com

I am almost ready to publish the final tutorial on using ssh tunnels from a truecrypt partition on a Windows machine. Of course I choose to go way overboard in my research and the tutorial is full of some pretty awesome windows tricks.. I will come back to this post soon and post all the other advanced tools I use for debugging windows, for now though you MUST know how to debug the kernel and use these basic debugging tools.

Debugging Tools for Windows

Target Computer and Host Computer

Kernel-mode debugging requires a target computer and a host computer. The target computer is used to run the kernel-mode application. The host computer is used to run the debugger. The following diagram shows the typical Microsoft Windows setup that you can use to perform kernel debugging and diagnose system failures. Typical Windows debugging setup This diagram shows the typical setup. However, the current versions of KD and WinDbg (which you installed with this documentation) are flexible. KD and WinDbg can do the following

• Debug a target computer that is running Windows.
• Debug a target computer that is running on an x86-based platform, an Itanium-based platform, or an x64-based platform.
• Can be started from a host computer that is running Windows.
• Can be started from a host computer that is on an x86-based platform, an Itanium-based platform, or an x64-based platform.

The target computer and host computer do not have to use the same platform or the same version of Windows. Kernel debugging does not require specific combinations of the free or checked builds. You can debug a free system from a free or checked system, and you can debug a checked system from a free or checked system. However, typically, there is no reason for the host computer to run the slower checked build. Note If you are running the debuggers from an Itanium-based host computer, make sure that you are using the correct version of the binaries. For more information about which version of the debugger package to use, see Choosing a 32-bit or 64-bit Debugger Package.

Debuggers

WinDbg  (Windbg.exe)

A user-mode and kernel-mode debugger with a graphical interface.

KD  (Kd.exe)

A kernel-mode debugger with a console interface.

CDB  (Cdb.exe)

A user-mode debugger with a console interface.

NTSD  (Ntsd.exe)

A user-mode debugger with a console interface. CDB and NTSD are virtually identical. In this documentation, whenever a reference is made to "CDB", it applies to both CDB and NTSD. When these two debuggers differ, it is noted. (See CDB and NTSD for details.)

Using Debugging Tools for Windows Download Windows Symbol Packages Symbol packages for a variety of Windows versions. Install Debugging Tools for Windows 32-bit Version Download page for the latest 32-bit packages. Install Debugging Tools for Windows 64-bit Versions Download page for the latest 64-bit packages. What's New for Debugging Tools for Windows Debugging Tools and Symbols: Getting Started Tips for using Debugging Tools for Windows, with links to presentations and papers on best practices Debugger How-Tos Driver Tip: Debugging Windows Vista Improve Driver Debugability 1394 Kernel Debugging Tips and Tricks [WinHEC 2004; 373 KB] Advanced Driver Debugging [336 KB] Driver Debugging Basics [WinHEC 2007; 633 KB] Debugging Kernel-Mode Driver Framework Drivers Debugging User-Mode Driver Framework Drivers Training Sessions Debugging Tools for Windows (MSDN) HOWTO: Enable Verbose Debug Tracing in Various Drivers and Subsystems (314743) WDK Online: Tools for Debugging Drivers Resources WDK Documentation Download Driver Developer Community Resources Driver Signing Requirements for Windows Notes from Windows Development Teams Support for Developer Kits and Tools Driver Developer Blogs Windows Driver Kit (WDK) Version 7.1.0 Release Notes WDK Documentation on MSDN See Also Microsoft Application Verifier

Additional Tools and Utilities

Logger  (Logger.exe and Logexts.dll)

A tool and an extension DLL that record the function calls and other actions of a program. Logger is described in this documentation; see Logger and LogViewer.

LogViewer  (Logviewer.exe)

A tool that displays the logs created by Logger. LogViewer is described in this documentation; see Logger and LogViewer.

ADPlus  (Autodump+, Adplus.vbs)

A console-based Microsoft Visual Basic script that can automatically create memory dump files and log files with debug output from one or more processes. ADPlus is described in this documentation; see ADPlus.

DbgRpc  (Dbgrpc.exe)

A tool used to display Microsoft Remote Procedure Call (RPC) state information. DbgRpc is described in this documentation; see RPC Debugging and Using the DbgRpc Tool.

KDbgCtrl  (Kernel Debugging Control, Kdbgctrl.exe)

A tool that controls and configures the kernel debugging connection. KDbgCtrl is described in this documentation; see Using KDbgCtrl.

SrcSrv  (Srcsrv.dll)

A source server that can be used to deliver source files while debugging. SrcSrv is described in this documentation; see SrcSrv.

SymSrv  (Symsrv.dll)

A symbol server that the debugger can use to connect to a symbol store. SymSrv is described in this documentation; see SymSrv.

SymStore  (Symstore.exe)

A tool used to create a symbol store. SymSrv is described in this documentation; see Using SymStore.

SymProxy

A tool used to create a single HTTP symbol server on your network that all your debuggers can point to. This has the benefit of pointing to multiple symbol servers (both internal and external) with a single symbol path, handling all authentication, and increasing performance via symbol caching. SymProxy is described in this documentation; see SymProxy.

AgeStore  (Agestore.exe)

A tool that removes old entries in the downstream store of a symbol server or a source server. AgeStore is described in this documentation; see AgeStore.

DBH  (Dbh.exe)

A tool that displays information about the contents of a symbol file. DBH is described in this documentation; see DBH.

PDBCopy  (Pdbcopy.exe)

A tool that removes private symbol information from a symbol file, and controls which public symbols are included in the file. PDBCopy is described in this documentation; see PDBCopy.

DumpChk  (Dump File Checking Utility, Dumpchk.exe)

A tool used to validate a memory dump file. DumpChk is described in this documentation; see DumpChk.

DbgSrv  (Dbgsrv.exe)

A process server used for remote debugging. DbgSrv is described in this documentation; see Process Servers (User Mode).

KdSrv  (Kdsrv.exe)

A KD connection server used for remote debugging. KDSrv is described in this documentation; see KD Connection Servers (Kernel Mode).

DbEngPrx  (Dbengprx.exe)

A repeater (small proxy server) used for remote debugging. DbgSrv is described in this documentation; see Repeaters.

The Remote tool  (Remote.exe)

A remoting tool that can be used to remotely control any console program, including KD, CDB, and NTSD. The Remote tool is described in this documentation; see Remote Tool and Remote Debugging Through Remote.exe.

GFlags  (Global Flags Editor, Gflags.exe)

A tool used to control registry keys and other settings. GFlags is described in this documentation; see GFlags.

The Kill tool  (Kill.exe)

A tool used to terminate a process. The Kill tool is described in this documentation; see Kill Tool.

The Breakin tool  (Breakin.exe)

A tool used to cause a user-mode break to occur in a process. Breakin.exe is not described in this documentation. Use the breakin /? command for help with this tool.

The List tool  (File List Utilit y, List.exe)

List.exe is not described in this documentation. Use the list /? command for help with this tool.

TList  (Task List Viewer, Tlist.exe)

A tool used to list all running processes. TList is described in this documentation; see TList.

RTList  (Remote Task List Viewer, Rtlist.exe)

A tool used to list running processes via a DbgSrv process server. RTList is not described in this documentation. Use the rtlist /? command for help with this tool.

UMDH  (User-Mode Dump Heap utility, Umdh.exe)

A tool used to analyze heap allocations. UMDH is described in this documentation; see UMDH.

USBView  (Universal Serial Bus Viewer, Usbview.exe)

A tool used to display the USB devices connected to a computer. USBView is described in this documentation; see USBView.

Documentation

"Debugging Tools for Windows"  (Debugger.chm)

This is the documentation you are currently reading. It is the central documentation for Debugging Tools for Windows.

"Debug Help Library"  (Dbghelp.chm)

This documentation describes the DbgHelp API and the ImageHlp API, and also explains how to create your own symbol server. This is installed when you peform a custom install of Debugging Tools for Windows and select the SDK feature and its subfeatures.

Tools Outside the Debugging Tools for Windows Package

Dr. Watson  (Drwtsn32.exe)

A tool used for automatically creating dump files and sending error reports to Microsoft Online Crash Analysis (OCA). Dr. Watson is partially described in this documentation; see Dr. Watson. The other features of Dr. Watson are described in the help file associated with drwtsn32.exe.

Build utility  (Build.exe)

A compiler and linker used to build debugger extensions and other programs. The Build utility and its documentation can be found in the Windows Driver Kit, and in earlier versions of the Windows DDK.

BinPlace  (Binplace.exe)

A tool used to control symbol files for build products. BinPlace and its documentation can be found in the Windows Driver Kit, and in earlier versions of the Windows DDK.

Application Verifier  (AppVerif.exe and !avrf)

A tool used to test user-mode applications. This tool consists of two components: the AppVerif.exe utility and the !avrf extension command. All the features of Application Verifier that are debugger-related are described in Application Verifier. The other features of Application Verifier are described in the help file associated with AppVerif.exe.

Debugging different target platforms

CDB

NTSD

NTSD in the System32 Directory

• This version of NTSD cannot be used for Remote Debugging Through the Debugger.
• This version of NTSD may not match the version of the documentation you are currently reading.

Controlling CDB or NTSD from the Kernel Debugger

Debugging Tools for Windows originally appeared on AskApache.com

There are so many WordPress plugins out there now that I wanted to post my favorite 3 plugins for speeding up a WP-Powered blog (including one of my plugins). These are the 3 plugins that I install for pretty much all of my WP-Powered sites, which I run about 300 now. They work together to provide a very optimized blog for speed.

Top 3 WordPress Speed Plugins

1. DB Cache Reloaded
2. WP Super Cache
3. AskApache Crazy Cache

WP Caching Overview

Each request to your blog has to fire up the php interpreter and query the mysql database to create the output that you see in your browser. Using a plugin like WP Super Cache simply saves the output of that request as a static HTML file and serves that to a request instead of using php and mysql every time. DB Cache Reloaded takes this a step further by optimizing the mysql queries. Finally, AskApache Crazy Cache is used to keep a cache fully primed and ready.

Why Caching?

If you have a private server, or you want to keep your MEMORY, BANDWIDTH, and CPU usage down for your server, these plugins will be dramatic. If you have a site that is updated maybe once a month and gets a very small amount of traffic, then the AskApache Crazy Cache would be redundant. That plugin is geared for the heaviest traffic sites.

Request and Response

Most HTTP communication is initiated by a user agent (browser) and consists of a request to a resource on some origin server. In the simplest case, this may be accomplished via a single connection (v) between the user agent (UA) and the origin server (O).

             request chain ------------------------>
          UA -------------------v------------------- O
             <----------------------- response chain

If a browser requests a WP-driven page, the server generates the response (the outputted html) by loading a php interpreter or module to read the WP php files and load all the settings. WP stores the settings in a mysql database and has to query the mysql database for all the data (like the content of your post that becomes html). Finally php sends the output through the server back to the browser. This is the norm for most PHP applications. Every time an interpreter is loaded additional CPU and Memory are used. And perhaps even more troublesome for shared-hosting using a virtual or network filesystem, each load causes many hard-disk accesses, additional processes, etc.

Cache PHP

By saving the php-generated output of a page to a static html file, your server can entirely skip loading a php interpreter or other process. Originally servers were needed and created to essentially open a static file on disk and send that file back to the requesting user. So servers are specialized for this as it's their core function.

WP-Super cache does this for you by saving the output into a static html file, and by instructing the server to skip loading php. It basically only uses php for creating the cached version, in the same way that you can save this webpage as an html file, WP-Super cache saves all the WP-blog permalinks to static files.

DB-Cache Reloaded does something entirely different, it saves the mysql queries that are made to the WP-database, as well as the mysql results to static files, and then through php serves those cached-files instead of re-querying the mysql database. Most mysql databases are stored on separate servers, and although many are on the same local network there is a limit to how many queries, and how many connections can take place. But mysql is maybe the fastest thing I've seen, so your bottlenecks almost never happen there (if configured correctly).

So DB-Cache Reloaded basically makes WP-Super Cache work alot faster when generating the cache files, and DB-Cache Reloaded helps in a number of areas un-related to WP-Super Cache, like in the admin panel. And DB-Cache without WP-Super-Cache is a joke because it still uses the application-level and php for everything. Gotta use both (or just WPSC).

AskApache Crazy Cache is a plugin I wrote to do one thing very well, it runs at intervals via the WP-cron and forces WP-Super-Cache to create a static cache file for all the posts, pages, etc. on your site. Without this WPSC likes to do dumb things like try to manage it's own cache with stale files and expired files, which equals a lot more php interpreters getting loaded instead of cached static files. For sites with more than a visit/page/10minutes this plugin keeps a full primed cache built by WPSC.

Compression and WP-Super Cache

Enabling compression in WP-Super Cache is almost always a great idea. I've never had a problem other than some php compat issues with not-updated php installations. This option basically lets WP-Super Cache compress the generated output of a page and save that to the static file. Normally Apache, Lighttpd, Nginx, etc. open the static file and compress it before it is sent to the browser, then the browser automatically decompresses it to view. This happens so fast because it is run by the server.

This lets WPSC instruct your server to send the compressed version to all browsers that accept compression, and send the uncompressed static file to any other browsers. So this is helpful because it eliminates your server having to do any transparent compressing, it can instead just focus on what it does best, serving static files.

PHP is an application so it requires memory, hard-drive access, and CPU time. Check out the protocol hierarchy:

       +------+ +-----+ +-----+       +-----+
       |Telnet| | FTP | |Voice|  ...  |     |  Application Level
       +------+ +-----+ +-----+       +-----+
             |   |         |             |
            +-----+     +-----+       +-----+
            | TCP |     | RTP |  ...  |     |  Host Level
            +-----+     +-----+       +-----+
               |           |             |
            +-------------------------------+
            |    Internet Protocol & ICMP   |  Gateway Level
            +-------------------------------+
                           |
              +---------------------------+
              |   Local Network Protocol  |    Network Level
              +---------------------------+

DB Cache Reloaded

DB Cache Reloaded Plugin Page - News

This plugin caches every database query with given lifetime. It is much faster than other html caching plugins and uses less disk space for caching.

I think you've heard of WP-Cache or WP Super Cache, they are both top plugins for WordPress, which make your site faster and responsive. Forget about them - with DB Cache Reloaded your site will work much faster and will use less disk space for cached files. Your visitors will always get actual information in sidebars and server CPU loads will be as low as possible.

Why is DB Cache Reloaded better than WP Super Cache?

This plugin is based on a fundamentally different principle of caching queries to database instead of full pages, which optimises WordPress from the very beginning and uses less disk space for cache files because it saves only useful information. It saves information separately and also caches hidden requests to database.

WP Super Cache

WP Super Cache Plugin Page - News

This plugin generates static html files from your dynamic WordPress blog. After a html file is generated your webserver will serve that file instead of processing the comparatively heavier and more expensive WordPress PHP scripts.

The static html files will be served to the vast majority of your users, but because a user's details are displayed in the comment form after they leave a comment those requests are handled by PHP. Static files are served to:

· Users who are not logged in.
· Users who have not left a comment on your blog.
· Or users who have not viewed a password protected post.

99% of your visitors will be served static html files. Those users who don't see the static files will still benefit because they will see regular WP-Cache cached files and your server won't be as busy as before. This plugin will help your server cope with a front page appearance on digg.com or other social networking site.

Why is WP-Super-Cache better than WP-Cache?

This plugin is based on the excellent WP-Cache plugin and therefore brings all the benefits of that plugin to WordPress. On top of that it creates copies of every page that is accessed on a blog in a form that is quickly served by the web server. It's almost as quick as if the you had saved a page in your browser and uploaded it to replace your homepage.

Will the Super Cache compression slow down my server?

No, it will do the opposite in fact. Super Cache files are compressed and stored that way so the heavy compression is done only once. These files are generally much smaller and are sent to a visitor's browser much more quickly than uncompressed html. As a result, your server spends less time talking over the network which saves CPU time and bandwidth, and can also serve the next request much more quickly.

AskApache Crazy Cache

AskApache Crazy Cache Plugin Page - News

This sweet little plugin does one thing very well. It caches all the posts on your entire blog at the same time, if you are using WP-Cache, WP-Super-Cache, or Hyper-Cache.

                              +---------+ ---------\      active OPEN
                              |  CLOSED |            \    -----------
                              +---------+<---------\   \   create TCB
                                |     ^              \   \  snd SYN
                   passive OPEN |     |   CLOSE        \   \
                   ------------ |     | ----------       \   \
                    create TCB  |     | delete TCB         \   \
                                V     |                      \   \
                              +---------+            CLOSE    |    \
                              |  LISTEN |          ---------- |     |
                              +---------+          delete TCB |     |
                   rcv SYN      |     |     SEND              |     |
                  -----------   |     |    -------            |     V
 +---------+      snd SYN,ACK  /       \   snd SYN          +---------+
 |         |<-----------------           ------------------>|         |
 |   SYN   |                    rcv SYN                     |   SYN   |
 |   RCVD  |<-----------------------------------------------|   SENT  |
 |         |                    snd ACK                     |         |
 |         |------------------           -------------------|         |
 +---------+   rcv ACK of SYN  \       /  rcv SYN,ACK       +---------+
   |           --------------   |     |   -----------
   |                  x         |     |     snd ACK
   |                            V     V
   |  CLOSE                   +---------+
   | -------                  |  ESTAB  |
   | snd FIN                  +---------+
   |                   CLOSE    |     |    rcv FIN
   V                  -------   |     |    -------
 +---------+          snd FIN  /       \   snd ACK          +---------+
 |  FIN    |<-----------------           ------------------>|  CLOSE  |
 | WAIT-1  |------------------                              |   WAIT  |
 +---------+          rcv FIN  \                            +---------+
   | rcv ACK of FIN   -------   |                            CLOSE  |
   | --------------   snd ACK   |                           ------- |
   V        x                   V                           snd FIN V
 +---------+                  +---------+                   +---------+
 |FINWAIT-2|                  | CLOSING |                   | LAST-ACK|
 +---------+                  +---------+                   +---------+
   |                rcv ACK of FIN |                 rcv ACK of FIN |
   |  rcv FIN       -------------- |    Timeout=2MSL -------------- |
   |  -------              x       V    ------------        x       V
    \ snd ACK                 +---------+delete TCB         +---------+
     ------------------------>|TIME WAIT|------------------>| CLOSED  |
                              +---------+                   +---------+
 
                      TCP Connection State Diagram

Vetted – Top 3 WordPress Speed Plugins originally appeared on AskApache.com

Advanced Web Development by AskApache is a Firefox Collection I created for myself to make finding and installing my favorite Firefox Add-ons simple and easy.

My Setup

As this screenshot shows, I only use a handful of add-ons at a time. These buttons allow me to clear the DNS/Cookies/Cache for whichever site I'm on when I click it. Very very helpful for me as a web designer. The 4th button there is just a restart button. Other than those, Firebug, YSlow, LastPass, and Web Developer are the only ones I always use regularly.

I like the idea of the last.fm but it's not as powerful as the site, which is awesome. Lately I've been listening to Kings of Leon Radio...

Slow Firefox

The more add-ons you have, disabled or not, the slower Firefox is. (unless you are running your profile folder in TMPFS). Also, bookmarks and settings like that have a performance hit. I have been using Firefox since it launched way back when, and I have always kept my bookmarks when moving to new machines and new installations.. So with over 5 thousand bookmarks I finally did some debugging and discovered that was a huge cause of Firefox acting slow. Now I am trying to migrate them all over to Google's Gmarks, which knowing Google will be awesome eventually.

Multiple Firefox Installations, Sorta

The solution to all these problems is to use separate Firefox profiles which use separate folders to store your profile-specific extensions and settings. So I have profiles with upwards of 40 Addons installed and enabled, and another profile that is built for speed... It's very very slow to be running Firebug and have multiple tabs open.. You can use the profilemanager to load them specific profiles from the command line. I personally use separate icons on my Windows Quick Launch that I just modified the shortcut pointing to firefox to also have the profile commandline. Note also that you can have multiple profiles open and running simultaneously.. This lets you do some crazy networking and other random hacks like having many Firefox instances running each of which is configured to use a separate SOCKS Proxy or network interface, so you can really open up those pipes for some intense txrx.

You can do some very powerful things with Firefox that most people are unaware of, if you are interested start with these:

• Setting up an extension development environment
• Firefox Command Line Options

About This Collection

Web Development Add-ons for Advanced Web Developers. I personally use these to work on Sites, Servers, WordPress, Javascript, PHP, CSS, XHTML, validation, page-loading, SEO, optimizing, and much more. Add this collection.

Created by:AskApache

Updated:September 22, 2009

Share this Collection

• Digg this!
• Post to Facebook
• Add to Delicious
• Post to MySpace
• Share on FriendFeed
• Post to Twitter

Firefox Add-ons for Web Developers originally appeared on AskApache.com

Hi there, do you want help with a project (SEO, site/server optimization, setup, wordpress plugin development, other programming, server issues, security issues, etc.) or need some expert consulting? I'm very comfortable working freelance for clients from all across the globe. I enjoy helping good causes or sites/webmasters similar to mine, and I enjoy making new friends and networking.

Personally, I love to chat on the phone or instant message with fellow webmasters and web entrepreneurs, and I like networking with like-minded individuals. Once my services are retained we will communicate any way that works best for you.. Billing is pretty flexible for all services other than SEO, which is billed aggressively: one word: skills.

Work Policy

I will never waste my time and your money by doing just enough to get paid or meet simple requirements. I deal only in results. I like to understand the root of the problem and discern what the task ultimately is meant to accomplish, and cut straight towards that. I dislike temporary fixes that will need to be re-addressed, preferring a "once-and-for-all" type of solution.

Refunds

If I can't meet or exceed your expectations initially agreed upon then I will instantly refund 100% of any payments made other then the retainer. I've never had anyone request a refund.

Retainer

If you'd like to hire me, please pay this $25 retainer by clicking the paypal image or here. This is non-refundable but if I can't help you and didn't spend time figuring that out I'll refund it. That also allows us to trust each other's identity as I don't reveal much about myself online to anonymous netizens. Once I have the retainer I will email you all my contact info so we can figure out all the details. The $25 verifies your identity and mine, and pays for the time spent figuring out the details of the job and when to start and that sort of thing.

My rate is $125/hour, though for quick jobs like mod_rewrite or htaccess it's usually only .5 or so of an hour, usually around $75 as I can finish Apache stuff with a quickness. For lengthy projects or hours we can work something out, alot of times for bigger jobs I prefer to charge for the total job, and not by the hour. And of course I'm always interested in performance-based or profit-sharing opportunities.

Ideal Candidate looking to work with me

1. The right attitudes
2. Willing to challenge and be challenged
3. Scrappy & will do whatever it takes
4. High attention to detail
5. Want to make history not read it
6. You have a history of achievement at whatever level you've been at.
7. You are the best person you know at what you do.
8. You are faster than fast
9. You can be competitive but nice at the same time
10. Skills

Hire AskApache originally appeared on AskApache.com

So my blog as been rather quiet for almost a year now, and very few updates if any have been released for my Password Protection PLugin, my Google 404 Plugin, and definately not for my AskApache CrazyCache plugin, which I will be releasing last... So for all of you who've helped me out by sending me suggestions and notifying me of errors and sticking with it... Just wanted to say sorry about that, and thanks for all the great ideas.. Well, I've been sticking with it as well believe it our not. I manage to get free days once in a while, and then its time to jam.

I've used just about every CMS/Blog/Forum/Trac/Gallery/etc) and really didn't like a lot of the way they coded... I could use php but I didn't KNOW php.. so I've had to learn php also, and it was tough to learn the advanced class usage and all the other language specific (but similar) constructs for php. It was especially difficult (but fun and challenging) to program so as to be compatible with php4 and php5 (Such is WordPress). But I kept at it, and soon you can decide for yourself what to make of it.

I can code in plenty of languages (bash, lua, windows .bat and vbs, ocaml, big fan of all things shell) and can work my way through C and even sorta somewhat with assembly. Assembly is the hardest, by far, I'm into easy and powerful languages like Python, Javascript, perl, php, ruby, and CGI. I've used PHP for a long time to do various things, but never to build software projects like this. Once I noticed WordPress's core .php files and the excellent programming I wanted to try and learn hot to do it. The WordPress code is some of the best I've seen. It appears the way they built it was planned, and not just dreamt up while typing that I can't help but do. Every time I read through the core code I learn a new trick or very nice way to do something. Those guys are really good, and I think WordPress is going to dominate for a long long time.

The Strategy

The Password Protection (passpro) plugin has a lot of complex stuff going on, especially for a newbie to PHP and WordPress like me, so after refactoring the whole thing at least 5 times I decided to modify my approach, and wrote the AskApache Google 404 Plugin as a way to practice on a simpler piece of code, while at the same time providing a plugin of value. Eventually I stopped thinking I could just code the whole thing in one sit-down with a stream-of-consciousness, and had to instead modularize the code and focus in on each part before moving to the next (I go without a plan because its fun, just not the most productive, but again, I'm not a programmer in the scientific sense.).

So I decided I had to really learn how WordPress Plugins work, filters, hooks, actions, and basically comfortability at reverse-engineering code, (Im a beginner for the last time), and so with the upcoming release of the AskApache Google 404 Plugin I have succeeded in making an incredibly stable plugin. That way I only have to worry about what the aapasspro plugin is doing, instead of trying to fit it into a framework.

AskApache Google 404 Upgrade

I think its rather unusual to develop a nice plugin like this 404 handler merely for the purpose of improving upon another plugin, but hey it worked. As of 08/03/2009 14:06PM EST I have about 1 hour left of finishing touches to release this upgrade. But as you cantell by my badly edited posts, I don't have a lot of time to myself. An hour here and there is about it. So it could be up to 2 weeks before I actually have the time to commit the release to the repo. On a sidenote, have you checked out Windows 7 News? I've been contracted to do some technical work for them and thought they had an excellent site.

But keep in mind, the 404 PLugin is just where I practice for the passpro plugin, which truly does have features that no other software like it has ever had. I understand the technology behind this plugin, and know it would really have a great impact on improving the Web (esp. WordPress) for all of us, I've just had to learn how to make it.

AskApache Password Protection

Probably still a couple weeks away, this plugin is the ultimate culmination of apache hackers dreams, at least those on shared servers (who may be interested in learning how to bypass security of said servers).. So this is something I have much too fun with doing what I like to do.. network/protocol-level security. I've examined the source code for many software packages that I use or have used to audit a server's security, and this simple php plugin in most instances can enumerate with accuraccy most of the server's setup in about a minute. The catch (and the file permission problems I had to find a workaround too) is that this software is launched on the server, not remotely against the server.

Some of the software I examined was whiskers, nessus, nmap, hping, mozilla source, wireshark, ncftp, netcat, etc.. The closest comparison to the socket-level class I've hacked together to those is wireshark. Except that wireshark only interprets (captures) the data passing over the wire, while this class does that and in fact sends and receives the data like netcat or nmap. Its really more similar to metasploit, and can easily be used to send hex, binary, ascii, or any type of payload to the remote or local host.

The Upgrades Begin

Well I started working on them a long time ago. Both the Password Protection plugin and the Google 404 plugin needed serious work. And I finally have it all figured out. Essentially I would work on one and finish an upgrade, but I just wasn't happy with it and I wold start all over again, refactoring the code. So as I put the finishing touches on those 2 plugins keep an eye out. They are major upgrades. I was able to meet all the goals I had for them, and came up with a lot of more improvements during the process.One of the main things I needed was a socket-level class to perform all kinds of checks and tests on. I need this also for my crazy cache plugin, which my blog is currently using , and I have a 2 more really nice pplugins I use that also needed access to a network class. I wrote about what I was doing with fsockopen, and I've been improving on that example ever since. I use this class to do some really powerful and exciting stuff, but you'll see it soon enough. As an indication of 'getting it right' for the Password Protection plugin, the plugin will now work on Windows, Apache, IIS, Lighthttpd, and will even work running on a blackberry web server. So now everyone using wordpress can at least get some security()

Many of the the other improvements focus on using the fsockopen class and .htaccess tricks to basically enumerate and discover all the different capabilities of your particular server; That way you can learn about all the features and security that are possible for your specific server, and the securty modules wi8ll be geared for that as well. FINALLY this plugin is going to be stable, and I just cant wait to see how people react when they learn all great capability their Apache-based Server has that they didn't have a clue about. Its amazing in that sense, and hackers will love theh way it works.. but your server admins will love it even more because its entirely 100% focused on helping you to set your site up (if you have Apache) to keep spammers out, to keep virii-serving robots and their log-hogging exploit requests and CPU/Mem robiing 404 errors off of your servers for real. This will have a noticeable affect to whoever is running the server. As you can tell.. I am pumped!

Apache can only be influenced by the main server configs and by .htaccess files. Not by php, not by perl, and the main configs are almost never accessible to the masses. But .htaccess files are. And many hosting providers allow and enable .htaccess files, a configuration file for your web server. The advanced features and capabilities of Apache were out of reach for most of us, it just wasn't possible to enumerate or access, and most hosting providers are infamous for their lack of .htaccess (customer) support. This plugin goes around those problems to give the power back to the people.

Here are a few examples of the capabilities of this plugin, some of which I believe no other software can do.. (Open source free to copy!).

1. Current Version of Apache (Down to the API Version)
2. List of ALL Modules currently enabled by Apache (Such as Mod_Rewrite)
3. List of ALL Directives enabled by EACH enabled Module.
4. Enumerate .htaccess Overrides, Context Permissions
5. Test for any builtin Handlers (like the status handler screenshot)
6. Configure SSI (http://www.askapache.com/htaccess/advanced-htaccess-ssi.html#htaccess-ssi-security)

March 1, 2009
I would focus on the method that WordPress uses. The code they have now (2.8 bleeding-edge) still isn't where it needs to be, but this is some difficult stuff and they have a brilliant start, it'll work.. just a question of when.

The main issue with the password protection plugin working for some people and not others is due to file permission configurations. The plugin attempts to write/modify files in your blog's root directory.

November 05, 2008
To make a long story short, I downloaded each major release of the apache httpd source code starting at version 1.3.0 and finishing with version 2.2.11, I then compiled each version and built a HTTPD from source for all these apache versions.

• 1.3.0
• 1.3.1
• 1.3.11
• 1.3.12
• 1.3.14
• 1.3.17
• 1.3.19
• 1.3.2
• 1.3.20
• 1.3.22
• 1.3.23
• 1.3.24
• 1.3.27
• 1.3.28

• 1.3.29
• 1.3.3
• 1.3.31
• 1.3.32
• 1.3.33
• 1.3.34
• 1.3.35
• 1.3.36
• 1.3.37
• 1.3.39
• 1.3.4
• 1.3.41
• 1.3.6
• 1.3.9

• 2.0.35
• 2.0.36
• 2.0.39
• 2.0.40
• 2.0.42
• 2.0.43
• 2.0.44
• 2.0.45
• 2.0.46
• 2.0.47
• 2.0.48
• 2.0.49
• 2.0.50
• 2.0.51

• 2.0.52
• 2.0.53
• 2.0.54
• 2.0.55
• 2.0.58
• 2.0.59
• 2.0.61
• 2.0.63
• 2.1.3-beta
• 2.1.6-alpha
• 2.1.7-beta
• 2.1.8-beta
• 2.1.9-beta

• 2.2.0
• 2.2.10
• 2.2.2
• 2.2.3
• 2.2.4
• 2.2.6
• 2.2.8
• 2.2.9
• 2.2.10
• 2.2.11

Then I went through each version and determined the compatible modules for that version, and I'm pretty confident that I was also able to find each and every directive allowed by the compatible modules for that version (including core directives). See .htaccess directive list. Basically I can now test a server using a variety of methods and determine almost 100% accurately what version of Apache (down to the API) is running, what modules (and versions) are enabled, and each and every directive that is allowed or disallowed for that version. So this is so awesome because now we can enable all sorts of additional security features.

Htaccess enabled Modules

Here are most of the modules that come with Apache. Each one can have new commands that can be used in .htaccess file scopes.

mod_actions, mod_alias, mod_asis, mod_auth_basic, mod_auth_digest, mod_authn_anon, mod_authn_dbd, mod_authn_dbm, mod_authn_default, mod_authn_file, mod_authz_dbm, mod_authz_default, mod_authz_groupfile, mod_authz_host, mod_authz_owner, mod_authz_user, mod_autoindex, mod_cache, mod_cern_meta, mod_cgi, mod_dav, mod_dav_fs, mod_dbd, mod_deflate, mod_dir, mod_disk_cache, mod_dumpio, mod_env, mod_expires, mod_ext_filter, mod_file_cache, mod_filter, mod_headers, mod_ident, mod_imagemap, mod_include, mod_info, mod_log_config, mod_log_forensic, mod_logio, mod_mem_cache, mod_mime, mod_mime_magic, mod_negotiation, mod_proxy, mod_proxy_ajp, mod_proxy_balancer, mod_proxy_connect, mod_proxy_ftp, mod_proxy_http, mod_rewrite, mod_setenvif, mod_speling, mod_ssl, mod_status, mod_substitute, mod_unique_id, mod_userdir, mod_usertrack, mod_version, mod_vhost_alias

Debugging HTTP protocol

Check this out! I'm particularly happy about this feature, which outputs an exact trace of any requests made by the plugin (such as during the testing phase) by saving the actual raw data sent out on the wire using fsockopen, RX and TX. This is useful for a number of reasons, viewing your headers, finding Redirect Loops, testing RewriteRules, and following the request hop-by-hop for debugging. The below example shows 2 requests for 2 URIs. The first URI is protected using Digest Authentication, the 2nd shows Basic.

 ______________
|  RAW TRACE   |
==================================================================================================================================
GET /htaccess/index.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
 
HTTP/1.1 401 Authorization Required
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
WWW-Authenticate: Digest realm="do or die", nonce="03328f3ec7c7b", algorithm=MD5, domain="/", qop="auth"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 882
Connection: close
Content-Type: text/html; charset=UTF-8
 
GET /htaccess/index.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
Authorization: Digest username="test",realm="do or die",nonce="03328f3ec7c7b",uri="/htaccess/index.txt?testing=query",
cnonce="82d057852a9dc497",nc=00000001,algorithm=MD5,response="9d476e9ea3",qop="auth"
 
HTTP/1.1 200 OK
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
Authentication-Info: rspauth="9051b01ee26dd62b3e2b40dada694f45", cnonce="82d057852a9dc497", nc=00000001, qop=auth
Last-Modified: Tue, 21 Jul 2009 23:56:00 GMT
Accept-Ranges: bytes
Cache-Control: max-age=3600
Expires: Wed, 22 Jul 2009 07:29:58 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 27
Connection: close
Content-Type: text/plain; charset=UTF-8
```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````
 
 ______________
|  RAW TRACE   |
==================================================================================================================================
GET /htaccess/po.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
 
HTTP/1.1 401 Authorization Required
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
WWW-Authenticate: Basic realm="Po Pimping"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 878
Connection: close
Content-Type: text/html; charset=UTF-8
 
GET /htaccess/po.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
Authorization: Basic adfAGAltcA==
 
HTTP/1.1 200 OK
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
Last-Modified: Wed, 22 Jul 2009 05:54:39 GMT
Accept-Ranges: bytes
Cache-Control: max-age=3600
Expires: Wed, 22 Jul 2009 07:29:58 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 27
Connection: close
Content-Type: text/plain; charset=UTF-8
```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````

.htaccess Directives

AcceptFilter, AcceptMutex, AcceptPathInfo, AccessFileName, Action, AddAlt, AddAltByEncoding, AddAltByType, AddCharset, AddDefaultCharset, AddDescription, AddEncoding, AddHandler, AddIcon, AddIconByEncoding, AddIconByType, AddInputFilter, AddLanguage, AddModuleInfo, AddOutputFilter, AddOutputFilterByType, AddType, Alias, AliasMatch, AllowCONNECT, AllowEncodedSlashes, AllowOverride, Anonymous, Anonymous_Authoritative, Anonymous_LogEmail, Anonymous_MustGiveEmail, Anonymous_NoUserID, Anonymous_NoUserId, Anonymous_VerifyEmail, AssignUserId, AuthAuthoritative, AuthBasicAuthoritative, AuthBasicProvider, AuthDBDUserPWQuery, AuthDBDUserRealmQuery, AuthDBM, AuthDBMAuthoritative, AuthDBMGroupFile, AuthDBMType, AuthDBMUserFile, AuthDefaultAuthoritative, AuthDigestAlgorithm, AuthDigestDomain, AuthDigestFile, AuthDigestGroupFile, AuthDigestNcCheck, AuthDigestNonceFormat, AuthDigestNonceLifetime, AuthDigestProvider, AuthDigestQop, AuthDigestShmemSize, AuthGroupFile, AuthLDAPAuthzEnabled, AuthLDAPBindDN, AuthLDAPBindON, AuthLDAPBindPassword, AuthLDAPCharsetConfig, AuthLDAPCompareDNOnServer, AuthLDAPDereferenceAliases, AuthLDAPEnabled, AuthLDAPFrontPageHack, AuthLDAPGroupAttribute, AuthLDAPGroupAttributeIsDN, AuthLDAPRemoteUserAttribute, AuthLDAPRemoteUserIsDN, AuthLDAPStartTLS, AuthLDAPURL, AuthLDAPUrl, AuthName, AuthType, AuthUserFile, AuthzDBMAuthoritative, AuthzDBMType, AuthzDefaultAuthoritative, AuthzGroupFileAuthoritative, AuthzLDAPAuthoritative, AuthzOwnerAuthoritative, AuthzUserAuthoritative, BS2000Account, BalancerMember, BrowserMatch, BrowserMatchNoCase, BufferedLogs, CGIMapExtension, CacheDefaultExpire, CacheDirLength, CacheDirLevels, CacheDisable, CacheEnable, CacheExpiryCheck, CacheFile, CacheForceCompletion, CacheGcClean, CacheGcDaily, CacheGcInterval, CacheGcMemUsage, CacheGcUnused, CacheIgnoreCacheControl, CacheIgnoreHeaders, CacheIgnoreNoLastMod, CacheLastModifiedFactor, CacheMaxExpire, CacheMaxFileSize, CacheMaxStreamingBuffer, CacheMinFileSize, CacheNegotiatedDocs, CacheRoot, CacheSize, CacheStoreNoStore, CacheStorePrivate, CacheTimeMargin, CharsetDefault, CharsetOptions, CharsetSourceEnc, CheckCaseOnly, CheckSpelling, ChildPerUserId, ContentDigest, CookieDomain, CookieExpires, CookieLog, CookieName, CookieStyle, CookieTracking, CoreDumpDirectory, CustomLog, DAV, DAVDepthInfinity, DAVGenericLockDB, DAVMinTimeout, DBDExptime, DBDKeep, DBDMax, DBDMin, DBDParams, DBDPersist, DBDPrepareSQL, DBDriver, Dav, DavDepthInfinity, DavGenericLockDB, DavLockDB, DavMinTimeout, DefaultIcon, DefaultLanguage, DefaultType, DeflateBufferSize, DeflateCompressionLevel, DeflateFilterNote, DeflateMemLevel, DeflateWindowSize, Directory, DirectoryIndex, DirectoryMatch, DirectorySlash, DocumentRoot, DumpIOInput, DumpIOOutput, EnableExceptionHook, EnableMMAP, EnableSendfile, ErrorDocument, ErrorLog, Example, ExpiresActive, ExpiresByType, ExpiresDefault, ExtFilterDefine, ExtFilterOptions, ExtendedStatus, FancyIndexing, FileETag, Files, FilesMatch, FilterChain, FilterDeclare, FilterProtocol, FilterProvider, FilterTrace, ForceLanguagePriority, ForceType, ForensicLog, GprofDir, GracefulShutdownTimeout, Group, Header, HeaderName, HostNameLookups, HostnameLookups, ISAIPFakeAsync, ISAPIAppendLogToErrors, ISAPIAppendLogToQuery, ISAPICacheFile, ISAPIFakeAsync, ISAPILogNotSupported, ISAPIReadAheadBuffer, IdentityCheck, IdentityCheckTimeout, IfDefine, IfModule, IfVersion, ImapBase, ImapDefault, ImapMenu, Include, IndexIgnore, IndexOptions, IndexOrderDefault, IndexStyleSheet, KeepAlive, KeepAliveTimeout, LDAPCacheEntries, LDAPCacheTTL, LDAPCertDBPath, LDAPConnectionTimeout, LDAPOpCacheEntries, LDAPOpCacheTTL, LDAPSharedCacheFile, LDAPSharedCacheSize, LDAPTrustedClientCert, LDAPTrustedGlobalCert, LDAPTrustedMode, LDAPVerifyServerCert, LanguagePriority, Limit, LimitExcept, LimitInternalRecursion, LimitRequestBody, LimitRequestFields, LimitRequestFieldsize, LimitRequestLine, LimitXMLRequestBody, Listen, ListenBacklog, LoadFile, LoadModule, Location, LocationMatch, LockFile, LogFormat, LogLevel, MCacheMaxObjectCount, MCacheMaxObjectSize, MCacheMaxStreamingBuffer, MCacheMinObjectSize, MCacheRemovalAlgorithm, MCacheSize, MMapFile, MaxClients, MaxKeepAliveRequests, MaxMemFree, MaxRequestsPerChild, MaxSpareServers, MaxSpareThreads, MaxSpareThreadsPerChild, MaxThreads, MetaDir, MetaFiles, MetaSuffix, MimeMagicFile, MinSpareServers, MinSpareThreads, ModMimeUsePathInfo, MultiviewsMatch, NWSSLTrustedCerts, NWSSLUpgradeable, NameVirtualHost, NoProxy, NumServers, Options, PassEnv, PerlAccessHandler, PerlAuthenHandler, PerlAuthzHandler, PerlChildExitHandler, PerlChildInitHandler, PerlCleanupHandler, PerlDispatchHandler, PerlFixupHandler, PerlFreshRestart, PerlHandler, PerlHeaderParserHandler, PerlInitHandler, PerlLogHandler, PerlModule, PerlPassEnv, PerlPostReadRequestHandler, PerlRequire, PerlRestartHandler, PerlSendHeader, PerlSetEnv, PerlSetVar, PerlSetupEnv, PerlTaintCheck, PerlTransHandler, PerlTypeHandler, PerlWarn, PidFile, Port, Protocol, ProtocolEcho, Proxy, ProxyBadHeader, ProxyBlock, ProxyDomain, ProxyErrorOverride, ProxyFtpDirCharset, ProxyIOBufferSize, ProxyMatch, ProxyMaxForwards, ProxyPass, ProxyPassInterpolateEnv, ProxyPassMatch, ProxyPassReverse, ProxyPassReverseCookieDomain, ProxyPassReverseCookiePath, ProxyPreserveHost, ProxyReceiveBufferSize, ProxyRemote, ProxyRemoteMatch, ProxyRequests, ProxySet, ProxyStatus, ProxyTimeout, ProxyVia, RLimitCPU, RLimitMEM, RLimitNPROC, ReadmeName, Redirect, RedirectMatch, RedirectPermanent, RedirectTemp, RemoveCharset, RemoveEncoding, RemoveHandler, RemoveInputFilter, RemoveLanguage, RemoveOutputFilter, RemoveType, RequestHeader, Require, RewriteBase, RewriteCond, RewriteEngine, RewriteLock, RewriteLog, RewriteLogLevel, RewriteMap, RewriteOptions, RewriteRule, SSIAccessEnable, SSIEndTag, SSIErrorMsg, SSIStartTag, SSITimeFormat, SSIUndefinedEcho, SSLCACertificateFile, SSLCACertificatePath, SSLCADNRequestFile, SSLCADNRequestPath, SSLCARevocationFile, SSLCARevocationPath, SSLCertificateChainFile, SSLCertificateFile, SSLCertificateKeyFile, SSLCipherSuite, SSLCryptoDevice, SSLEngine, SSLHonorCipherOrder, SSLLog, SSLLogLevel, SSLMutex, SSLOptions, SSLPassPhraseDialog, SSLProtocol, SSLProxyCACertificateFile, SSLProxyCACertificatePath, SSLProxyCARevocationFile, SSLProxyCARevocationPath, SSLProxyCipherSuite, SSLProxyEngine, SSLProxyMachineCertificateFile, SSLProxyMachineCertificatePath, SSLProxyProtocol, SSLProxyVerify, SSLProxyVerifyDepth, SSLRandomSeed, SSLRequire, SSLRequireSSL, SSLSessionCache, SSLSessionCacheTimeout, SSLUserName, SSLVerifyClient, SSLVerifyDepth, Satisfy, ScoreBoardFile, Script, ScriptAlias, ScriptAliasMatch, ScriptInterpreterSource, ScriptLog, ScriptLogBuffer, ScriptLogLength, ScriptStock, SecureListen, SendBufferSize, ServerAdmin, ServerAlias, ServerLimit, ServerName, ServerPath, ServerRoot, ServerSignature, ServerTokens, SetEnv, SetEnvIf, SetEnvIfNoCase, SetHandler, SetInputFilter, SetOutputFilter, StartServers, StartThreads, Substitute, SuexecUserGroup, ThreadLimit, ThreadStackSize, ThreadsPerChild, TimeOut, Timeout, TraceEnable, TransferLog, TypeAuthDBMUserFile, TypesConfig, UnsetEnv, UseCanonicalName, UseCanonicalPhysicalPort, User, UserDir, VirtualDocumentRoot, VirtualDocumentRootIP, VirtualHost, VirtualScriptAlias, VirtualScriptAliasIP, Win32DisableAcceptEx, XBitHack, allow, deny, order, php_admin_flag, php_admin_value, php_flag, php_value

You can view the plugins home page, old, or view it on the wordpress.org site.

An AskApache Plugin Upgrade to Rule them All originally appeared on AskApache.com

My traffic is growing, alot, and I need to plan how I'm going to maintain scalability, high availability, and redundancy. Scalability is an application's ability to support a growing number of users. High availability can be defined as redundancy or speed. I decided to setup Round Robin DNS for static.askapache.com, which is the "static" subdomain of AskApache that serves all the static assets like images, javascript, css, etc.. (BTW, the z stands for ZAP).. All I needed to attempt setting this up was another hosting account on a separate server. I have hosting accounts with around 10 different companies from working with various clients over the years, like Powweb and I don't use them because they suck in terms of the unix environment. Many of these web hosts are actually very fast bandwidth-wise..

Round Robin Concept

A few months ago I was given a free hosting account on HostGator to evaluate and tempt me away from DreamHost to become a Gator. I get a lot of these types of offers from time to time from the smaller Web companies who read AskApache.. but when I noticed the SSH access was jailed and saw how restrictive the shell was I felt like I was on a windows box.. I want a shell, cpanel sucks. I compile and run everything from the shell so thats was a deal-breaker and I sorta forgot all about it.

The goal is to add the HostGator server to be an exact mirror of the static.askapache.com domain, then to add that server as a 2nd A record to my DNS zone. That way half the visitors to the size will be taking up resources and bandwidth on the HostGator server instead of mine.

Round Robin A records in DNS are intended to evenly distribute queries between each host of the same name. Using some tricks straight out of a hackers toolbox we can verify if the distribution is taking place. (It is.)

DNS - Domain Name System

The Domain Name System is often analogized as a "phone book" for the Internet because it translates human-friendly computer hostnames into IP addresses. For example, www.askapache.com translates to 208.113.134.190. Every request for a human-friendly hostname first needs to be translated to the IP before the server can be queried. Normally each hostname corresponds to exactly 1 IP address. But in a Round Robin setup, the idea is to have the hostname correspond to multiple IP addresses, which are different servers that house the exact same content, resulting in some hosts requesting files from one server, and another host requesting files from the other server, resulting in less CPU resources and bandwidth.

Here is an the zone for static.askapache.com Round Robin records:

Round Robin DNS

Round robin DNS is a technique of load distribution, load balancing, or fault-tolerance provisioning multiple, redundant Internet Protocol service hosts, e.g., Web servers, FTP servers, by managing the Domain Name System's (DNS) responses to address requests from client computers according to an appropriate statistical model.

In its simplest implementation Round-robin DNS works by responding to DNS requests not only with a single IP address, but a list of IP addresses of several servers that host identical services. The order in which IP addresses from the list are returned is the basis for the term round robin. With each DNS response, the IP address sequence in the list is permuted. Usually, basic IP clients attempt connections with the first address returned from a DNS query so that on different connection attempts clients would receive service from different providers, thus distributing the overall load among servers.

Round robin DNS is often used for balancing the load of geographically-distributed Web servers. For example, a company has one domain name and three identical web sites residing on three servers with three different IP addresses. When one user accesses the home page it will be sent to the first IP address. The second user who accesses the home page will be sent to the next IP address, and the third user will be sent to the third IP address. In each case, once the IP address is given out, it goes to the end of the list. The fourth user, therefore, will be sent to the first IP address, and so forth.

Although easy to implement, round robin DNS has problematic drawbacks, such as those arising from record caching in the DNS hierarchy itself, as well as client-side address caching and reuse, the combination of which can be difficult to manage. Round robin DNS should not solely be relied upon for service availability. If a service at one of the addresses in the list fails, the DNS will continue to hand out that address and clients will still attempt to reach the inoperable service.

Does Round Robin Work

Definately. I can look at the access logs for both servers and see that they are splitting the requests nicely. It is definately not an exact split however, look at these statistics and you'll see what I mean.

$ dig @ns1.dreamhost.com +authority +all static.askapache.com
 
;; ANSWER SECTION:
static.askapache.com.        14400   IN      A       69.56.174.114
static.askapache.com.        14400   IN      A       64.111.114.111
 
$ dig @ns1.dreamhost.com +authority +all static.askapache.com
 
;; ANSWER SECTION:
static.askapache.com.        14400   IN      A       64.111.114.111
static.askapache.com.        14400   IN      A       69.56.174.114

The effects of caching will distort the effectiveness of any IP address allocation algorithm unless a 0 TTL is used which has the effect of significantly increasing the load on the DNS (and is not always implemented consistently). In this case the cure may be worse than the disease Good news we have good load balancing on our web servers. Bad news we need 17 more DNS servers!. Intuitively, and without running any experiments to verify, we would suggest that given a normal TTL (12 hours or more) and ANY IP allocation algorithm other than a single static list, loads should be reasonably balanced .

Full root server query

Tracing to static.askapache.com[a] via A.ROOT-SERVERS.NET, maximum of 3 retries
A.ROOT-SERVERS.NET [.] (198.41.0.4)
 |\___ L.GTLD-SERVERS.NET [com] (192.41.162.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ D.GTLD-SERVERS.NET [com] (192.31.80.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ J.GTLD-SERVERS.NET [com] (192.48.79.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ H.GTLD-SERVERS.NET [com] (192.54.112.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ E.GTLD-SERVERS.NET [com] (192.12.94.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ F.GTLD-SERVERS.NET [com] (192.35.51.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ I.GTLD-SERVERS.NET [com] (192.43.172.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ G.GTLD-SERVERS.NET [com] (192.42.93.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ B.GTLD-SERVERS.NET [com] (2001:0503:231d:0000:0000:0000:0002:0030) Not queried
 |\___ B.GTLD-SERVERS.NET [com] (192.33.14.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ A.GTLD-SERVERS.NET [com] (2001:0503:a83e:0000:0000:0000:0002:0030) Not queried
 |\___ A.GTLD-SERVERS.NET [com] (192.5.6.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ C.GTLD-SERVERS.NET [com] (192.26.92.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ M.GTLD-SERVERS.NET [com] (192.55.83.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
  \___ K.GTLD-SERVERS.NET [com] (192.52.178.30)
       |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
       |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
        \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       static.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       static.askapache.com -> 69.56.174.114

Live Online DNS Tools

• DNSstuff.com - Your Destination for DNS and Networking Tools
• ZoneCheck
• Quick Check - Pingability.com
• Squishywishywoo: complete dns traversal checking
• ZoneCut DNS
• DNS Sleuth
• DNS test - Domain name DNS test - pweb.cz
• OpenDNS - Support - CacheCheck
• DNSReport
• DNSTrace
• intoDNS
• DNSthru.com
• DNS Doctor
• Power Check

More Reading

• HOWTO - Configure Load Balancing
• Load Sharing with DNS

RFC's related to DNS

• RFC 920: Specified original TLDs: .arpa, .com, .edu, .org, .gov, .mil and two-character country codes
• RFC 1032: Domain administrators guide
• RFC 1033: Domain administrators operations guide
• RFC 1034: Domain Names - Concepts and Facilities.
• RFC 1035: Domain Names - Implementation and Specification
• RFC 1101: DNS Encodings of Network Names and Other Types
• RFC 1123: Requirements for Internet Hosts -- Application and Support
• RFC 1912: Common DNS Operational and Configuration Errors
• RFC 1995: Incremental Zone Transfer in DNS
• RFC 1996: A Mechanism for Prompt Notification of Zone Changes (DNS NOTIFY)
• RFC 2136: Dynamic Updates in the domain name system (DNS UPDATE)
• RFC 2181: Clarifications to the DNS Specification
• RFC 2182: Selection and Operation of Secondary DNS Servers
• RFC 2308: Negative Caching of DNS Queries (DNS NCACHE)
• RFC 2317: Classless IN-ADDR.ARPA delegation
• RFC 2671: Extension Mechanisms for DNS (EDNS0)
• RFC 3597: Handling of Unknown DNS Resource Record (RR) Types
• RFC 3696: Application Techniques for Checking and Transformation of Names
• RFC 4343: Domain Name System (DNS) Case Insensitivity Clarification
• RFC 4592: The Role of Wildcards in the Domain Name System
• RFC 4892: Requirements for a Mechanism Identifying a Name Server Instance
• RFC 5001: DNS Name Server Identifier Option (NSID)
• RFC 5395: Domain Name System (DNS) IANA Considerations

This page contains content by Author of Article from Wikipedia and is licensed under the GNU FDL.

Table of Contents

• Round Robin Concept
• DNS - Domain Name System
• Round Robin DNS
• Efficacy of DNS Load Balancing
• Live Online DNS Tools
• More Reading
• RFC's related to DNS

DNS Round Robin Configuration using Rsync over SSH originally appeared on AskApache.com