AskApache » Web Hosting

Optimizing Servers and Processes for Speed with ionice, nice, ulimit

AskApache — Sat, 10 Oct 2009 05:41:28 +0000

Ok, sup. I really felt I had to get this out of the way, because I have a whole stack of drafts waiting to be published, but I realized that not many people will benefit from all the advanced optimizations and tricks I’m writing unless they get a basic understanding of some of the tools I’m using. I decided to write a series of articles explaining how I optimize servers for speed because lately I’ve been getting a lot more people wanting to hire me to do that. I take on projects when I can but there is clearly a need out here on the net for some self-help. The momentum is swinging more and more towards VPS type of web hosting, and I would say that 99% of those customers are getting supremely ripped off, which goes against the foundation of the web.

Keep in mind that this blog and my research is only a hobby of mine, my job is primarily marketing and sales, so I’m not some licensed expert or anything, or even an unlicensed expert! haha. But it does bother me that those who are tech-savvy enough to run web-hosting companies are happily ripping people off. So this article details the main tools that are used to speed up and optimize your machine by delegating levels of priority to specific processes. Future articles will use these tools alot, so this is meant as an intro.

CPU and Disk I/O

As most of you are aware, there are 2 variables that determine any computer or programs speed. CPU and Disk I/O. CPU determines how fast you can process data, crunch numbers, etc. while disk I/O determines how fast your disks can read and write data to the hard-drive. Wouldn’t it be great if you could easily configure your server to give your httpd, php, and other processes both greater CPU processing and disk IO than your non-important processes like backup scripts, ftp daemons, etc.? We are talking about Linux in this article, so of course YES not only can you do that, you should!

RAM

RAM is like a hard-drive in that data is stored on it, and read/written to it. The difference is that RAM is somewhere around 30x faster than disk I/O, but the cost of that incredible speed is that the data stored on it is only temporary in the sense that it won’t be stored permanently, it is completely erased when your machine is rebooted. RAM is also expensive, and there is a limit to how much a server or machine can have due to hardware limits.

SWAP

SWAP takes off when you run out of RAM but you still want certain data to be read/write quickly. Basically when you start running out of RAM your machine starts supplementing RAM with SWAP storage. SWAP is usually a partition on a second hard-drive disk. There is an upper limit on how much I/O can occur on a disk at one time, and the more I/O takes place, the slower all I/O becomes, so SWAP works well on a separate hard-drive as it will have much faster I/O. On Windows they opted to copy the SWAP mechanism but instead use a file named pagefile.sys, and that is just one reason people in the know do not care for Windows.

CPU

So lets do this, think of your CPU (your processor) as having an amount of 100% processing available when not being used, 0% when its maxed out. CPU’s handle multiple processing tasks simultaneously, so what we will discuss in this article is how to specify HOW MUCH of that processing amount each of your programs (heretofore “processes”) are able to use. Yes, very very cool.

That is correct, you can easily configure your server to provide more of the available processing time to certain programs over others, like you can configure apache and php to utilize 50% of your CPU processing time by themselves, so that all other processes (proftpd, sshd, rsync, etc.) combined can only utilize 50%. The terminology is we can give certain specific processes (like php.cgi, httpd, fast-cgi.cgi) a specific priority, where -19 is the most priority, and +19 is the least amount of priority, or CPU processing time. I know it seems backwards..

The Tools

If you run Windows, you are in the right place… because the following advice will save your life: GET LINUX! Ok, now that that is out of the way, the following are the tools dicussed on this page. All of them are free, open-source, and wonderful. The basic idea of these tools is to control how much CPU is devoted to each process, and also how much Disk IO/Disk traffic is given to each process.

nice: run a program with modified scheduling priority
renice: alter priority of running processes
ionice: set or retrieve the I/O priority for a given pid or execute a new task with a given I/O priority.
iostat: Report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions.
ulimit: Ulimit provides control over the resources available to processes started by the shell, on systems that allow such control.
chrt: set or retrieve real-time scheduling parameters for a given pid or execute a new task under given scheduling parameters.
taskset: set or retrieve task CPU affinity for a given pid or execute a new task under a given affinity mask.

Part 1: Process Processes Faster

Ok so lets tackle figuring out how to give your response-intensive processes (like apache, php, ruby, perl, java) meaning a request to your server/machine requires a response. For instance, when you requested this page that you are reading at this very second, several things on my server had to happen for you to be able to read this.

First your computer sends out a request to see what server the www.askapache.com domain name is. DNS servers respond with my server IP, so for servers dedicated as nameservers, optimizing the DNS processes like bind would speed that up. Now that your computer knows how to reach my server it sends an HTTP GET request for this url. This request is received by the httpd process that is apache, and apache determines this url should be handled by my custom compiled php5.3.0 binary, because this page is WordPress generated. So the php binary loads up the WordPress /index.php file, which chain-loads several other php files, including wp-config.php containing my MySql database settings. Now php connects to my MySql Server to fetch this articles content, comments, title, tags, etc. and then generates the HTML and hands that back to Apache.

Finally, Apache generates a HTTP RESPONSE and sends the RESPONSE and CONTENT back to your Browser, which then in turn renders the page for your eyes with the necessary javascript, images, css, and other files included in the HTML response.

Too much Processing

Now you see why I’ve opted to write my own caching plugin that takes the php and mysql processes OUT of that equation. Both the php binary and the mysql instance consume CPU processing, and disk IO, to load all their library files, make various network requests and sockets, check permissions, and on and on. And that’s completely ok, the thing is, unless you configure these processes (Apache, PHP, MySQL) they will use the same amount of CPU processing that other processes use, other processes that have very little to do with you reading this sentence. Processes to run my mail server, my FTP server, my SSH server, my cronjobs, cleanup scripts, atd daemon, etc.. and they will get the same amount of CPU!

Another even simpler example is what got me to look into this myself. I wrote a shell script that created hourly, daily, weekly, and monthly backups for all of my websites and sql databases, and set it up to run by cronjob at those set intervals. Eventually I noticed my sites were slower, my php even slower, and sometimes I even saw 503 errors that my host throws up when my server is overloaded. The research that I pursued to prevent that from happening has been hugely eye-opening. What does a backup script do? Mine just created tar archives of all the files in my web root, then gzipped the tar archive saving to a backup server using scp (a file transfer using ssh). This resulted in the following huge problems that seem to have nothing to do with a faster server and speedier website, but they have everything to with it.

CPU Bottleneck #1 – tar and gzip use compression algorithms at a low level to create a compressed version, and all that compressing uses a whole lot of crunching – CPU processing
DISK IO Bottleneck – Tarring the whole web root directory was creating a ton of disk io, and remember the more disk io that is going on, the less is available for everything else.
CPU Bottleneck #2 – Using scp to send my backups was security-smart, but these huge archive files had to be encrypted and sent over the net.

Breaking Bottles

I apologize for being a little long-winded there, but I think it’s important to make sure everyone understands those basic concepts, which are foreign to most people. Once you understand what is causing the bottlenecks, then you can understand the solutions, which actually are incredibly simple and even a novice linux user can easily do. Besides, the net gets a little bit faster every time someone implements this.

nice

Nice allows you to run a program with modified scheduling priority which specifies how much CPU is devoted to a particular process. Run COMMAND with an adjusted niceness, which affects process scheduling. With no COMMAND, print the current niceness.

Nicenesses range from -20 (most favorable scheduling) to 19 (least favorable). -n, --adjustment=N – add integer N to the niceness (default 10). nice +19 tasks get a HZ-independent 1.5%. Running a nice +10 and a nice +11 task means the first will get 55% of the CPU, the other 45%.

nice usage

nice [OPTION] [COMMAND [ARG]...]
 
-n, --adjustment=ADJUST   increment priority by ADJUST first

Examples of nice

Using nice to download a file

nice -n 17 curl -q -v -A 'Mozilla/5.0' -L -O http://wordpress.org/latest.zip

Unzipping a file with nice

nice -n 17 unzip latest.zip

Nice way to build from source

nice -n 2 ./configure
nice -n 2 make
nice -n 2 make install

It is sometimes useful to run non-interactive programs with reduced priority.

$ nice factor `echo '2^9 - 1'|bc`
511: 7 73

Since nice prints the current priority, we can invoke it through itself to demonstrate how it works: The default behavior is to reduce priority by 10.

 $ nice nice
10
$ nice -n 10 nice
10

The ADJUSTMENT is relative to the current priority. The first nice invocation runs the second one at priority 10, and it in turn runs the final one at a priority lowered by 3 more.

$ nice nice -n 3 nice
13

Specifying a priority larger than 19 is the same as specifying 19.

$ nice -n 30 nice
19

Only a privileged user may run a process with higher priority.

$ nice -n -1 nice
nice: cannot set priority: Permission denied
$ sudo nice -n -1 nice
-1

The new scheduler in v2.6.23 addresses all three types of complaints:

To address the first complaint (of nice levels being not “punchy” enough), the scheduler was decoupled from ‘time slice’ and HZ concepts (and granularity was made a separate concept from nice levels) and thus it was possible to implement better and more consistent nice +19 support: with the new scheduler nice +19 tasks get a HZ-independent 1.5%, instead of the variable 3%-5%-9% range they got in the old scheduler.

To address the second complaint (of nice levels not being consistent), the new scheduler makes nice(1) have the same CPU utilization effect on tasks, regardless of their absolute nice levels. So on the new scheduler, running a nice +10 and a nice 11 task has the same CPU utilization “split” between them as running a nice -5 and a nice -4 task. (one will get 55% of the CPU, the other 45%.) That is why nice levels were changed to be “multiplicative” (or exponential) – that way it does not matter which nice level you start out from, the ‘relative result’ will always be the same.

The third complaint (of negative nice levels not being “punchy” enough and forcing audio apps to run under the more dangerous SCHED_FIFO scheduling policy) is addressed by the new scheduler almost automatically: stronger negative nice levels are an automatic side-effect of the recalibrated dynamic range of nice levels.

renice

Renice is similar to the nice command, but it lets you modify the nice of a currently running process. This is nice for shell scripts where you can add this to the top of the script to nicify the whole script to 19.

renice usage

renice priority [ [ -p ] pids ] [ [ -g ] pgrps ] [ [ -u ] users ]
 
-g      Force who parameters to be interpreted as process group ID's.
-u      Force the who parameters to be interpreted as user names.
-p      Resets the who interpretation to be (the default) process ID's.

Examples of renice

From the shell, changes the priority of the shell and all children to 19. From a shell script, does the same but only for the script and its children.

renice 19 -p $$

This runs renice without any output

renice 19 -p $$ &>/dev/null

10 gets more CPU than 19

renice 10 -p $$

change the priority of process ID’s 987 and 32, and all processes owned by users daemon and root.

renice +1 987 -u daemon root -p 32

Part 2: Optimizing Disk I/O

Linux Scheduling Policies

The scheduler is the kernel component that decides which runnable process will be executed by the CPU next. Each process has an associated scheduling policy and a static scheduling priority, sched_priority

Processes scheduled under one of the real-time policies (SCHED_FIFO, SCHED_RR) have a sched_priority value in the range 1 (low) to 99 (high). (As the numbers imply, real-time processes always have higher priority than normal processes.) The following “real-time” policies are also supported, for special time-critical applications that need precise control over the way in which runnable processes are selected for execution:

Currently, Linux supports the following “normal” (i.e., non-real-time) scheduling policies:

SCHED_OTHER: Default Linux time-sharing scheduling: The standard round-robin time-sharing policy
SCHED_BATCH: Scheduling batch processes: This policy is useful for workloads that are non-interactive, but do not want to lower their nice value, and for workloads that want a deterministic scheduling policy without interactivity causing extra preemptions (between the workload’s tasks).
SCHED_IDLE: Scheduling very low priority jobs: This policy is intended for running jobs at extremely low priority (lower even than a +19 nice value with the SCHED_OTHER or SCHED_BATCH policies)
SCHED_FIFO: First In-First Out scheduling: A first-in, first-out policy
SCHED_RR: Round Robin scheduling: A round-robin policy.

Scheduling Classes

IOPRIO_CLASS_RT: This is the realtime io class. The RT scheduling class is given first access to the disk, regardless of what else is going on in the system. Thus the RT class needs to be used with some care, as it can starve other processes. As with the best effort class, 8 priority levels are defined denoting how big a time slice a given process will receive on each scheduling window. This scheduling class is given higher priority than any other in the system, processes from this class are given first access to the disk every time. Thus it needs to be used with some care, one io RT process can starve the entire system. Within the RT class, there are 8 levels of class data that determine exactly how much time this process needs the disk for on each service. In the future this might change to be more directly mappable to performance, by passing in a wanted data rate instead.
IOPRIO_CLASS_BE: This is the best-effort scheduling class, which is the default for any process that hasn’t set a specific io priority. This is the default scheduling class for any process that hasn’t asked for a specific io priority. Programs inherit the CPU nice setting for io priorities. This class takes a priority argument from 0-7, with lower number being higher priority. Programs running at the same best effort priority are served in a round-robin fashion. The class data determines how much io bandwidth the process will get, it’s directly mappable to the cpu nice levels just more coarsely implemented. 0 is the highest BE prio level, 7 is the lowest. The mapping between cpu nice level and io nice level is determined as: io_nice = (cpu_nice + 20) / 5.
IOPRIO_CLASS_IDLE: This is the idle scheduling class, processes running at this level only get io time when no one else needs the disk. A program running with idle io priority will only get disk time when no other program has asked for disk io for a defined grace period. The impact of idle io processes on normal system activity should be zero. This scheduling class does not take a priority argument. The idle class has no class data, since it doesn’t really apply here.

ionice

ionice – get/set program io scheduling class and priority. This program sets the io scheduling class and priority for a program. Since v3 (aka CFQ Time Sliced) CFQ implements I/O nice levels similar to those of CPU scheduling. These nice levels are grouped in three scheduling classes each one containing one or more priority levels:

ionice usage

If no arguments or just -p is given, ionice will query the current io scheduling class and priority for that process.

ionice [-c] [-n] [-p] [COMMAND [ARG...]]

-c – The scheduling class. 1 for real time, 2 for best-effort, 3 for idle.
-n – The scheduling class data. This defines the class data, if the class accepts an argument. For real time and best-effort, 0-7 is valid data.
-p – Pass in a process pid to change an already running process. If this argument is not given, ionice will run the listed program with the given parameters.

ionice Examples

Sets process with PID 89 as an idle io process.

ionice -c3 -p89

Runs ‘bash’ as a best-effort program with highest priority.

ionice -c2 -n0 bash

Returns the class and priority of the process with PID 89

ionice -p89

With the ionice command, you can set the IO priority for a process to one of three classes: Idle (3), Best Effort (2), and Real Time (1). The Idle class means that the process will only be able to read and write to the disk when all other processes are not using the disk. The Best Effort class is the default and has eight different priority levels from 0 (top priority) to 7 (lowest priority). The Real Time class results in the process having first access to the disk irregardless of other process and should never be used unless you know what you are doing.

If we wish to run the updatedb process in the background with an Idle IO class priority, we can run the following:
$ sudo date
$ sudo updatedb &
[1] 16324
$ sudo ionice -c3 -p16324
If we’d rather just lower the Best Effort class priority (defaults to 4) for the command so the process isn’t limited to idle IO periods, we can run the following:
$ sudo date
$ sudo updatedb &
[1] 16324
$ sudo ionice -c2 -n7 -p16324
Again, the Real Time class should not be used as it can prevent you from being able to interact with your system.

You may wonder where you can get the process ID if you don’t know it, can’t remember it, or didn’t start the process (an automatted script may have launched it). You can find process IDs with the ps command.

For example, if I had an updatedb program running in the background, and I wanted to find its process ID, I can run the following:
$ ps -C updatedb
PID TTY TIME CMD
4234 ? 00:00:42 updatedb
This tells me that the process’ process ID (PID) is 4234.

iostat

iostat Usage

iostat [ -c ] [ -d ] [ -N ] [ -n ] [ -h ] [ -k | -m ] [ -t ] [ -V ] [ -x ] [ -z ] [  [...] | ALL ] [ -p [  [,...] | ALL ] ] [  [  ] ]

-c     The -c option is exclusive of the -d option and displays only the CPU usage report.
-d     The -d option is exclusive of the -c option and displays only the device utilization report.
-k     Display statistics in kilobytes per second instead of blocks per second.  Data displayed are valid only with kernels 2.4 and newer.
-m     Display statistics in megabytes per second instead of blocks or kilobytes per second.  Data displayed are valid only with kernels 2.4 and newer.
-n     Displays the NFS-directory statistic.  Data displayed are valid only with kernels 2.6.17 and newer.  This option is exclusive ot the -x option.
-h     Display the NFS report more human readable.
-p [ { device | ALL } ]   The  -p  option  is  exclusive  of  the -x option and displays statistics for block devices and all their partitions that are used by the system.
-t     Print the time for each report displayed.
-x     Display extended statistics.

iostat Examples

iostat -p ALL 2 1000
avg-cpu:  %user   %nice    %sys %iowait   %idle
            8.34    0.08    1.26    2.27   88.05

Display a single history since boot report for all CPU and Devices.

$ iostat

Display a continuous device report at two second intervals.

$ iostat -d 2

Display six reports at two second intervals for all devices.

$ iostat -d 2 6

Display six reports of extended statistics at two second intervals for devices hda and hdb.

$ iostat -x hda hdb 2 6

Display six reports at two second intervals for device sda and all its partitions (sda1, etc.)

$ iostat -p sda 2 6

Schedule Utils

These are the Linux scheduler utilities – schedutils for short. These programs take advantage of the scheduler family of syscalls that Linux implements across various kernels. These system calls implement interfaces for scheduler-related parameters such as CPU affinity and real-time attributes. The standard UNIX utilities do not provide support for these interfaces — thus this package.

The programs that are included in this package are chrt and taskset. Together with nice and renice (not included), they allow full control of process scheduling parameters. Suggestions for related utilities are welcome, although it is believed (barring new interfaces) that all scheduling interfaces are covered.

I’ve found that quite a few servers do not have this package installed, indicating to you that they might not know what they are doing. Here is how you can install this incredible package, for non-root users. Root users know how to do this, or they shouldn’t be root. Download and install in 1 line provided you have curl. Or just use the following commands.

mkdir -pv $HOME/{dist,source,bin,share/man/man1} && cd ~/dist && curl -O http://ftp.de.debian.org/debian/pool/main/s/schedutils/schedutils_1.5.0.orig.tar.gz && cd ~/source && tar -xvzf ~/dist/sch*z && cd sch* && sed -i -e 's,= /usr/local,=${HOME},g' Makefile && make && make install && make installdoc

mkdir -pv $HOME/{dist,source,bin,share/man/man1}
cd ~/dist && curl -O http://ftp.de.debian.org/debian/pool/main/s/schedutils/schedutils_1.5.0.orig.tar.gz
cd ~/source && tar -xvzf ~/dist/schedutils_1.5.0.orig.tar.gz
cd ~/source/schedutils-1.5.0 && sed -i -e 's,= /usr/local,=${HOME},g' Makefile
make || make -d && make install || make install -d && make installdoc || make installdoc -d

taskset

Taskset is used to set or retrieve the CPU affinity of a running process given its PID or to launch a new COMMAND with a given CPU affinity. CPU affinity is a scheduler property that “bonds” a process to a given set of CPUs on the system. The Linux scheduler will honor the given CPU affinity and the process will not run on any other CPUs. Note that the Linux scheduler also supports natural CPU affinity: the scheduler attempts to keep processes on the same CPU as long as practical for performance reasons. Therefore, forcing a specific CPU affinity is useful only in certain applications.

The CPU affinity is represented as a bitmask, with the lowest order bit corresponding to the first logical CPU and the highest order bit corresponding to the last logical CPU. Not all CPUs may exist on a given system but a mask may specify more CPUs than are present. A retrieved mask will reflect only the bits that correspond to CPUs physically on the system. If an invalid mask is given (i.e., one that corresponds to no valid CPUs on the current system) an error is returned. A user must possess CAP_SYS_NICE to change the CPU affinity of a process. Any user can retrieve the affinity mask.

taskset Usage

taskset [options] [mask | cpu-list] [pid | cmd [args...]]
 
-p, --pid            operate on existing given pid
-c, --cpu-list     display and specify cpus in list format

taskset-examples

The default behavior is to run a new command:

$ taskset 03 sshd -b 1024

You can retrieve the mask of an existing task or set it:

$ taskset -p 700
$ taskset -p 03 700

List format uses a comma-separated list instead of a mask:

$ taskset -pc 0,3,7-11 700

chrt

chrt sets or retrieves the real-time scheduling attributes of an existing PID or runs COMMAND with the given attributes. Both policy (one of SCHED_FIFO, SCHED_RR, or SCHED_OTHER) and priority can be set and retrieved. A user must possess CAP_SYS_NICE to change the scheduling attributes of a process. Any user can retrieve the scheduling information.

chrt Usage

chrt [options] [prio] [pid | cmd [args...]]
 
-p, --pid operate on an existing PID and do not launch a new task
-f, --fifo set scheduling policy to SCHED_FIFO
-m, --max show minimum and maximum valid priorities, then exit
-o, --other set policy scheduling policy to SCHED_OTHER
-r, --rr set scheduling policy to SCHED_RR (the default)

chrt Examples

The default behavior is to run a new command: chrt [prio] -- [command] [arguments]

You can also retrieve the real-time attributes of an existing task:

chrt -p [pid]

Or set them:

chrt -p [prio] [pid]

ulimit – get and set user limits

Ulimit provides control over the resources available to processes started by the shell, on systems that allow such control. One can set the resource limits of the shell using the built-in ulimit command. The shell’s resource limits are inherited by the processes that it creates to execute commands.

ulimit Usage

ulimit [-SHacdfilmnpqstuvx] [limit]

-S: use the `soft’ resource limit
-H: use the `hard’ resource limit
-a: all current limits are reported
-c: the maximum size of core files created
-d: the maximum size of a process’s data segment
-f: the maximum size of files created by the shell
-l: the maximum size a process may lock into memory
-m: the maximum resident set size
-n: the maximum number of open file descriptors
-p: the pipe buffer size
-s: the maximum stack size
-t: the maximum amount of cpu time in seconds
-u: the maximum number of user processes
-v: the size of virtual memory

If LIMIT is given, it is the new value of the specified resource; the special LIMIT values `soft’, `hard’, and `unlimited’ stand for the current soft limit, the current hard limit, and no limit, respectively. Otherwise, the current value of the specified resource is printed. If no option is given, then -f is assumed. Values are in 1024-byte increments, except for -t, which is in seconds, -p, which is in increments of 512 bytes, and -u, which is an unscaled number of processes.

RLIMIT_AS: The maximum size of the process’s virtual memory (address space) in bytes. This limit affects calls to brk(2), mmap(2) and mremap(2), which fail with the error ENOMEM upon exceeding this limit. Also automatic stack expansion will fail (and generate a SIGSEGV that kills the process if no alternate stack has been made available via sigaltstack(2)). Since the value is a long, on machines with a 32-bit long either this limit is at most 2 GiB, or this resource is unlimited.
RLIMIT_CORE: Maximum size of core file. When 0 no core dump files are created. When non-zero, larger dumps are truncated to this size.
RLIMIT_CPU CPU: time limit in seconds. When the process reaches the soft limit, it is sent a SIGXCPU signal. The default action for this signal is to terminate the process. However, the signal can be caught, and the handler can return control to the main program. If the process continues to consume CPU time, it will be sent SIGXCPU once per second until the hard limit is reached, at which time it is sent SIGKILL. (This latter point describes Linux 2.2 through 2.6 behavior. Implementations vary in how they treat processes which continue to consume CPU time after reaching the soft limit. Portable applications that need to catch this signal should perform an orderly termination upon first receipt of SIGXCPU.)
RLIMIT_DATA: The maximum size of the process’s data segment (initialized data, uninitialized data, and heap). This limit affects calls to brk(2) and sbrk(2), which fail with the error ENOMEM upon encountering the soft limit of this resource.
RLIMIT_FSIZE: The maximum size of files that the process may create. Attempts to extend a file beyond this limit result in delivery of a SIGXFSZ signal. By default, this signal terminates a process, but a process can catch this signal instead, in which case the relevant system call (e.g., write(2), truncate(2)) fails with the error EFBIG.
RLIMIT_LOCKS: (Early Linux 2.4 only) A limit on the combined number of flock(2) locks and fcntl(2) leases that this process may establish.
RLIMIT_MEMLOCK: The maximum number of bytes of memory that may be locked into RAM. In effect this limit is rounded down to the nearest multiple of the system page size. This limit affects mlock(2) and mlockall(2) and the mmap(2) MAP_LOCKED operation. Since Linux 2.6.9 it also affects the shmctl(2) SHM_LOCK operation, where it sets a maximum on the total bytes in shared memory segments (see shmget(2)) that may be locked by the real user ID of the calling process. The shmctl(2) SHM_LOCK locks are accounted for separately from the per-process memory locks established by mlock(2), mlockall(2), and mmap(2) MAP_LOCKED; a process can lock bytes up to this limit in each of these two categories. In Linux kernels before 2.6.9, this limit controlled the amount of memory that could be locked by a privileged process. Since Linux 2.6.9, no limits are placed on the amount of memory that a privileged process may lock, and this limit instead governs the amount of memory that an unprivileged process may lock.
RLIMIT_MSGQUEUE: (Since Linux 2.6.8) Specifies the limit on the number of bytes that can be allocated for POSIX message queues for the real user ID of the calling process. This limit is enforced for mq_open(3). Each message queue that the user creates counts (until it is removed) against this limit according to the formula: bytes = attr.mq_maxmsg * sizeof(struct msg_msg *) + attr.mq_maxmsg * attr.mq_msgsize where attr is the mq_attr structure specified as the fourth argument to mq_open(3). The first addend in the formula, which includes sizeof(struct msg_msg *) (4 bytes on Linux/i386), ensures that the user cannot create an unlimited number of zero-length messages (such messages nevertheless each consume some system memory for bookkeeping overhead).
RLIMIT_NICE: (since Linux 2.6.12, but see BUGS below) Specifies a ceiling to which the process’s nice value can be raised using setpriority(2) or nice(2). The actual ceiling for the nice value is calculated as 20 – rlim_cur. (This strangeness occurs because negative numbers cannot be specified as resource limit values, since they typically have special meanings. For example, RLIM_INFINITY typically is the same as -1.)
RLIMIT_NOFILE: Specifies a value one greater than the maximum file descriptor number that can be opened by this process. Attempts (open(2), pipe(2), dup(2), etc.) to exceed this limit yield the error EMFILE. (Historically, this limit was named RLIMIT_OFILE on BSD.)
RLIMIT_NPROC: The maximum number of processes (or, more precisely on Linux, threads) that can be created for the real user ID of the calling process. Upon encountering this limit, fork(2) fails with the error EAGAIN.
RLIMIT_RSS: Specifies the limit (in pages) of the process’s resident set (the number of virtual pages resident in RAM). This limit only has effect in Linux 2.4.x, x < 30, and there only affects calls to madvise(2) specifying MADV_WILLNEED.
RLIMIT_RTPRIO: (Since Linux 2.6.12, but see BUGS) Specifies a ceiling on the real-time priority that may be set for this process using sched_setscheduler(2) and sched_setparam(2).
RLIMIT_RTTIME: (Since Linux 2.6.25) Specifies a limit on the amount of CPU time that a process scheduled under a real-time scheduling policy may consume without making a blocking system call. For the purpose of this limit, each time a process makes a blocking system call, the count of its consumed CPU time is reset to zero. The CPU time count is not reset if the process continues trying to use the CPU but is preempted, its time slice expires, or it calls sched_yield(2). Upon reaching the soft limit, the process is sent a SIGXCPU signal. If the process catches or ignores this signal and continues consuming CPU time, then SIGXCPU will be generated once each second until the hard limit is reached, at which point the process is sent a SIGKILL signal. The intended use of this limit is to stop a runaway real-time process from locking up the system.
RLIMIT_SIGPENDING: (Since Linux 2.6.8) Specifies the limit on the number of signals that may be queued for the real user ID of the calling process. Both standard and real-time signals are counted for the purpose of checking this limit. However, the limit is only enforced for sigqueue(2); it is always possible to use kill(2) to queue one instance of any of the signals that are not already queued to the process.
RLIMIT_STACK: The maximum size of the process stack, in bytes. Upon reaching this limit, a SIGSEGV signal is generated. To handle this signal, a process must employ an alternate signal stack (sigaltstack(2)).

ulimit Examples

Turn off core dumps

ulimit -S -c 0

Man Pages

Kernel Documentation

Future Discussions:

IO Benchmarking: How, Why and With What

Optimizing Servers and Processes for Speed with ionice, nice, ulimit originally appeared on AskApache.

Advanced WordPress wp-config.php Tweaks

AskApache — Sat, 03 Oct 2009 07:23:37 +0000

The bottom line for this article is that I want to make WordPress as fast, secure, and easy to install, run, and manage because I am using it more and more for client production sites, I will work for days in order to solve an issue so that I never have to spend time on that issue again. Time is money in this industry and that is ultimately (time) what there is to gain by tweaking WordPress.

Note: I spent no time on readability, this is primarily a read the code and figure it out article.. This is for advanced users looking for a reference or discussion and for those of you looking to advance. Feedback would be great if you make it that far..

For a better handle on the way I like to structure web site directories, see Optimize a Website for Speed, Security, and Easy Management but note it is a bit outdated compared to what I’m doing now. I don’t have the luxury of using only one type of server, or hosting provider anymore, so I have been working towards making things even more portable in order to move from host to host from server to server without issues i.e. my portable .bash_profile.

So I’ve been basically experimenting various ways to accomplish that and thought I would share what I am currently doing for my benefit and hopefully get some input. All of my WP installs run the development version, and one main idea with my setups is that upgrading is automated. So I really keep the WordPress install clean and use plugins and wp-config.php to do all the customization.

Portability – Hands-free upgrades and easy to move
Security – Additional security and protection
Speed – Less CPU and Disk I/O
Customization – All my favorite customizations

wp-config.php

These are the main settings I use.. Seriously this is more like an interactive article, because to understand it you will need to do some code grepping. You may want to grab a jolt.

ASKAPACHE_ROOT

The ASKAPACHE_ROOT variable is just a better way for me to be able to include and access all the different files in my site tree. For instance, in my non-wp php files, I can do this:

!defined('ASKAPACHE_ROOT') && require $_SERVER['DOCUMENT_ROOT'] . '/wp-config.php';
include(ASKAPACHE_ROOT . '/includes/custom-download.inc.php');

ASKAPACHE_LOCK

This is one of my all-time favorite hacks, that I think is one of the most useful methods I employ as a web developer. This allows me to use far-future-expire headers for optimum caching, while still forcing browsers to re-validate every day or so automatically, or forcing them to re-validate whenever I change the suffix. This takes advantage of the mod_rewrite trick that I use on EVERY site I run, definately worth learning. Because I practice best-practice web-standards, for every web site I create a single css file and javascript file, which I then add to the template like:

 Settings > General panel.
 */
!defined('WP_SITEURL') && define('WP_SITEURL', 'http://'.$_SERVER['SERVER_NAME']);
 
/**
 * WP_HOME is another wp-config.php option added in WordPress Version 2.2. Similar to WP_SITEURL,
 * WP_HOME overrides the wp_options table value for home but does not change it permanently.
 * home is the address you want people to type in their browser to reach your WordPress blog. It should include the http:// part. Also, do not put a slash "/" at the end.
 */
!defined('WP_HOME') && define('WP_HOME', WP_SITEURL);
 
/** no trailing slash, full paths only */
!defined('WP_CONTENT_DIR') && define( 'WP_CONTENT_DIR', ABSPATH . 'wp-content' );
 
// full url - WP_CONTENT_DIR is defined further up
!defined('WP_CONTENT_URL') && define( 'WP_CONTENT_URL', WP_SITEURL . '/wp-content');
 
/** Allows for the plugins directory to be moved from the default location. @since 2.6.0 */
// full path, no trailing slash
!defined('WP_PLUGIN_DIR') && define( 'WP_PLUGIN_DIR', WP_CONTENT_DIR . '/plugins' );
 
/** Allows for the plugins directory to be moved from the default location. @since 2.6.0 */
// full url, no trailing slash
!defined('WP_PLUGIN_URL') && define( 'WP_PLUGIN_URL', WP_CONTENT_URL . '/plugins' );
 
/** Allows for the plugins directory to be moved from the default location. @since 2.1.0 */
// Relative to ABSPATH.  For back compat.
//!defined('PLUGINDIR') && define( 'PLUGINDIR', 'wp-content/plugins' );
 
/** Number of autosaves to save. TRUE is default and enables post revisions, FALSE disables revisions completely. */
!defined('WP_POST_REVISIONS') && define('WP_POST_REVISIONS', 150);
 
/* ini_set('memory_limit', WP_MEMORY_LIMIT); */
!defined('WP_MEMORY_LIMIT') && define('WP_MEMORY_LIMIT', '64M');
 
/** Only check at this interval for new messages. Default is 5min */
/** @since 2.9  */
!defined('WP_MAIL_INTERVAL') && define('WP_MAIL_INTERVAL', 3600); // 1 hour
 
/** Saves updated post values to post from edit window every x seconds. (default 60)
 * When editing a post, WordPress uses Ajax to auto-save revisions to the post as you edit. You may want to increase this setting for longer delays in between auto-saves, or decrease the setting to make sure you never lose changes.
 * @since 2.5.0 */
!defined( 'AUTOSAVE_INTERVAL' ) && define( 'AUTOSAVE_INTERVAL', 60 );
 
/** @since 2.9.0  */
/** Permanently deletes posts, pages, attachments, and comments which have been in the trash for EMPTY_TRASH_DAYS. */
!defined( 'EMPTY_TRASH_DAYS' ) && define( 'EMPTY_TRASH_DAYS', 300 );

Debugging WordPress

One of my secrets for getting really good at this stuff is to master debugging. There is really not ever a time when I am working on a site that I don’t have color-highlighted logs scrolling automatically in an ssh window. It’s really almost impossible to fix problems with wordpress or do any kind of advanced anything without being able to view debugging info. At first I relied heavily on a custom php.ini being available on the server, but after having to deal with many hosts who don’t allow php.ini files I now rely completely on setting values using ini_set for ultimate portability. Detailed towards the end of this article and is also included in this wp-config.php

/**#@+
 * DEBUGGING STUFF
 */
/** display of notices during development. if false, error_reporting is E_ERROR | E_WARNING | E_PARSE | E_USER_ERROR | E_USER_WARNING | E_RECOVERABLE_ERROR otherwise E_ALL */
!defined('WP_DEBUG') && define('WP_DEBUG', false);
 
/** The SAVEQUERIES definition saves the database queries to a array and that array can be displayed to help analyze those queries.
 *  The information saves each query, what function called it, and how long that query took to execute.  */
!defined('SAVE_QUERIES') && define('SAVE_QUERIES', WP_DEBUG);
 
!defined('ACTION_DEBUG') && define('ACTION_DEBUG', WP_DEBUG);
 
/** This will allow you to edit the scriptname.dev.js files in the wp-includes/js and wp-admin/js directories.  */
!defined('SCRIPT_DEBUG') && define('SCRIPT_DEBUG', WP_DEBUG);

 
/** Add define('WP_DEBUG_LOG', true); to enable php debug logging to WP_CONTENT_DIR/debug.log */
//!defined('WP_DEBUG_LOG') && define('WP_DEBUG_LOG', true);
 
/** This determines whether errors should be printed to the screen as part of the output or if they should be hidden from the user.
 *  Add define('WP_DEBUG_DISPLAY', false); to wp-config.php to use the globally configured setting for display_errors and not force it to On */
!defined('WP_DEBUG_DISPLAY') && define('WP_DEBUG_DISPLAY', false);

Ultimate Security Tweaks

Well, ultimate for WP’s built-in keys and password functions, this is all for wp-config.php keep in mind. This is a very neccessary and recommended step, and is one of the only things I modify for each new installation.

Security KEYS

If like me you are familiar with password-cracking software like John the ripper, rainbow hash tables, l0pht-crack, etc.. then you will like to know that you can specify your own keys and salts for the encryption used by WP. They are AUTH_KEY, AUTH_SALT, SECURE_AUTH_KEY, SECURE_AUTH_SALT, LOGGED_IN_KEY, LOGGED_IN_SALT, NONCE_KEY, NONCE_SALT, SECRET_KEY and SECRET_SALT.

A random and long key gives you better encryption, and exponentially increasing that is using a random and long salt for the encryption. Encryptions with known salts are incredibly easy to decrypt compared to encryptions with secure salts, because the salt + key individually need to be guessed in order to find a matching hash, vs. just the key if the salt is known. See: Locating weak passwords.

A secret key is a hashing salt which makes your site harder to hack and access harder to crack by adding random elements to the password.

In simple terms, a secret key is a password with elements that make it harder to generate enough options to break through your security barriers. A password like “password” or “test” is simple and easily broken. A random, unpredictable password such as “88a7da62429ba6ad3cb3c76a09641fc” takes years to come up with the right combination.

For more information on the technical background and breakdown of secret keys and secure passwords, see:

I like to use the WordPress.org secret-key service 4 times. That’s because for each key and salt I like to do: (1 key from api +random keyboard input+1 key from api).

/**#@+
 * Authentication Unique Keys.
 *
 * Change these to different unique phrases!
 * You can generate these using the {@link https://api.wordpress.org/secret-key/1.1/ WordPress.org secret-key service}
 * You can change these at any point in time to invalidate all existing cookies.
 * This will force all users to have to log in again.
 *
 * @since 2.6.0
 *
 * Get salt to add to hashes to help prevent attacks.
 *
 * The secret key is located in two places: the database in case the secret key
 * isn't defined in the second place, which is in the wp-config.php file. If you
 * are going to set the secret key, then you must do so in the wp-config.php
 * file.
 *
 * The secret key in the database is randomly generated and will be appended to
 * the secret key that is in wp-config.php file in some instances. It is
 * important to have the secret key defined or changed in wp-config.php.
 *
 * If you have installed WordPress 2.5 or later, then you will have the
 * SECRET_KEY defined in the wp-config.php already. You will want to change the
 * value in it because hackers will know what it is. If you have upgraded to
 * WordPress 2.5 or later version from a version before WordPress 2.5, then you
 * should add the constant to your wp-config.php file.
 *
 * Below is an example of how the SECRET_KEY constant is defined with a value.
 * You must not copy the below example and paste into your wp-config.php. If you
 * need an example, then you can have a
 * {@link https://api.wordpress.org/secret-key/1.1/ secret key created} for you.
 *
 * Salting passwords helps against tools which has stored hashed values of
 * common dictionary strings. The added values makes it harder to crack if given
 * salt string is not weak.
 *
 * @since 2.5
 * @link https://api.wordpress.org/secret-key/1.1/ Create a Secret Key for wp-config.php
 *
 * @return string Salt value from either 'SECRET_KEY' or 'secret' option
 */
define('AUTH_KEY',        'jflkhaskljdfhkljasdhflkjashd;flkjhas;djfh;kajshdflkjashdlfkjhasdlkfhal?p[B+GR{@>{Yq`c|LnG;dvq#| %OA_cbBSU6,rICC1o/c)-|');
define('SECURE_AUTH_KEY', 'jflkhaskljdfhkljasdhflkjashd;flkjhas;djfh;kajshdflkjashdlfkjhasdlkfhal?Vp[Bb15baar8&R-r<[T|?(xhJJABGq+Ux+U$)-Hltp/');
define('LOGGED_IN_KEY',   'jflkhaskljdfhkljasdhflkjashd;flkjhas;djfh;kajshdflkjashdlfkjhasdlkfhal?Vp[B<5n6DG|YWnJ9tY2!M1L)`{-$LW~~Ia%.uCbn!P. 41o2$Z$4');
define('NONCE_KEY',       'jflkhaskljdfhkljasdhflkjashd;flkjhas;djfh;kajshdflkjashdlfkjhasdlkfhal?Vp[Bgu+wm 2)bS0Pd_+1rx0brX]ND8|');
define('SECRET_SALT',      '123423190847olqkfhladhfsldshafasdfasdf09a7f-90a87df98adfyapoiyaf9asd8f70a9s8d7f908a7sdf97W4qCdm2<>))U|sty)+4vpWooKls/^[vN');
/**#@-*/

Using SSL for Admin and Login

SSL is kinda required from my point of view, it is just way to easy to sniff data off the wire otherwise. At least with SSL you force them to use tools like burpsuite, paros proxy, webscarab, etc..

/** @since 2.6.0  */
!defined('FORCE_SSL_ADMIN') && define('FORCE_SSL_ADMIN', true);
 
/** @since 2.6.0  */
!defined('FORCE_SSL_LOGIN') && define('FORCE_SSL_LOGIN', true);

Mod_Rewrite to Force SSL

This is pretty cool, it forces non-https for all urls except for /wp-admin and wp-login.php, which both require https. It also checks for the logged_in_cookie, and if that is present in the request then it doesn't force non-https. Kinda confusing if you don't have a mod_rewrite cheatsheet.

RewriteCond %{THE_REQUEST} ^$ [OR]
RewriteCond %{REQUEST_URI} ^/(wp-admin|wp-login\.php).*$ [NC,OR]
RewriteCond %{HTTP_COOKIE} ^.*wp_li_sadfsdfasdf11b361cdsdfasdfasd=.*$ [NC]
RewriteRule .* - [S=1]
 
RewriteCond %{HTTPS} =on [OR]
RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC]
RewriteRule .* http://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]
 
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(wp-admin/.*|wp-login\.php.*)\ HTTP/ [NC]
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]

File System Permissions

You can get a basic and solid intro on file permissions by reading: Changing File Permissions, or you can check out some of my file permission research.

/** The permissions as octal number, usually 0644 for files, 0755 for dirs.
 *  http://codex.wordpress.org/Changing_File_Permissions
 *  if ( !$wp_filesystem->mkdir($remote_destination, FS_CHMOD_DIR) )
 */
!defined('FS_CHMOD_DIR') && define('FS_CHMOD_DIR', (0755 & ~ umask()));
!defined('FS_CHMOD_FILE') && define('FS_CHMOD_FILE', (0644 & ~ umask()));
/**#@-*/
 
/** Define the timeouts for the connections. Only available after the construct is called to allow for per-transport overriding of the default. */
//stream_set_timeout( $stream, FS_TIMEOUT );
//!defined('FS_TIMEOUT') && define('FS_TIMEOUT', 30);
 
//$this->link = @ftp_connect($this->options['hostname'], $this->options['port'], FS_CONNECT_TIMEOUT);
//!defined('FS_CONNECT_TIMEOUT') && define('FS_CONNECT_TIMEOUT', 30);
 
// function get_filesystem_method($args = array(), $context = false) {
//  $method = defined('FS_METHOD') ? FS_METHOD : false; //Please ensure that this is either 'direct', 'ssh', 'ftpext' or 'ftpsockets'
//!defined('FS_METHOD') && define('FS_METHOD', 'direct');
 
/** These methods for the WordPress core, plugin, and theme upgrades try to determine the WordPress path, as reported by PHP, but symlink trickery can sometimes
 * 'muck this up' so if you know the paths to the various folders on the server, as seen via your FTP user, you can manually define them in the wp-config.php file.
 * FS_METHOD forces the filesystem method. It should only be "direct", "ssh", "ftpext", or "ftpsockets".
 * FTP_BASE is the full path to the "base" folder of the WordPress installation.
 * FTP_CONTENT_DIR is the full path to the wp-content folder of the WordPress installation.
 * FTP_PLUGIN_DIR is the full path to the plugins folder of the WordPress installation.
 * FTP_PUBKEY is the full path to your SSH public key.
 * FTP_PRIKEY is the full path to your SSH private key.
 * FTP_USER is either user FTP or SSH username. Most likely these are the same, but use the appropriate one for the type of update you wish to do.
 * FTP_PASS is the password for the username entered for FTP_USER. If you are using SSH public key authentication this can be omitted.
 * FTP_HOST is the hostname:port combination for your SSH/FTP server. The standard FTP port is 21 and the standard SSH port is 22.
 */
//define('FS_METHOD', 'ftpext');
//define('FTP_BASE', '/path/to/wordpress/');
//define('FTP_CONTENT_DIR', '/path/to/wordpress/wp-content/');
//define('FTP_PLUGIN_DIR ', '/path/to/wordpress/wp-content/plugins/');
//define('FTP_PUBKEY', '/home/username/.ssh/id_rsa.pub');
//define('FTP_PRIKEY', '/home/username/.ssh/id_rsa');
//define('FTP_USER', 'username');
//define('FTP_PASS', 'password');
//define('FTP_HOST', 'ftp.example.org:21');
 
/**
 * Block requests through the proxy.
 *
 * Those who are behind a proxy and want to prevent access to certain hosts may do so. This will
 * prevent plugins from working and core functionality, if you don't include api.wordpress.org.
 *
 * You block external URL requests by defining WP_HTTP_BLOCK_EXTERNAL in your wp-config.php file
 * and this will only allow localhost and your blog to make requests.
 * The constant WP_ACCESSIBLE_HOSTS will allow additional hosts to go through for requests. The format of the
 * WP_ACCESSIBLE_HOSTS constant is a comma separated list of hostnames to allow.
 *
 * @since 2.8.0
 * @link http://core.trac.wordpress.org/ticket/8927 Allow preventing external requests.
/** @since 2.9  */
//!defined('WP_HTTP_BLOCK_EXTERNAL') && define( 'WP_HTTP_BLOCK_EXTERNAL', false );
 
/*
 * The constant WP_ACCESSIBLE_HOSTS will allow additional hosts to go through for requests. The format of the
 * WP_ACCESSIBLE_HOSTS constant is a comma separated list of hostnames to allow.
 *
 * @since 2.8.0
 * @link http://core.trac.wordpress.org/ticket/8927 Allow preventing external requests.
 * $accessible_hosts = preg_split('|,\s*|', WP_ACCESSIBLE_HOSTS);
 * return !in_array( $check['host'], $accessible_hosts ); //Inverse logic, If its in the array, then we can't access it.
 */
//!defined('WP_ACCESSIBLE_HOSTS') && define( 'WP_ACCESSIBLE_HOSTS', 'askapache.com,askapache.org' );

Cookies!

There’s always a little comfort in having non-default cookies for security (against auto-bots), and using shorter names also means smaller HTTP Packets.

The $cookie_hash is my hack to get around the fact that COOKIEHASH isn’t definable in wp-config.

/**#@+
 * COOKIES
 * Used to guarantee unique hash cookies @since 1.5 */
$cookie_hash=md5(WP_SITEURL);
 
/** Set a cookie now to see if they are supported by the browser.
 * setcookie(TEST_COOKIE, 'WP Cookie check', 0, COOKIEPATH, COOKIE_DOMAIN);
 * @since 2.3.0 */
!defined('TEST_COOKIE') && define('TEST_COOKIE', 'wp_tc');
 
/* @since 2.6.0 */
!defined('LOGGED_IN_COOKIE') && define('LOGGED_IN_COOKIE', 'wp_li_' . $cookie_hash);
 
/* @since 2.6.0 */
!defined('SECURE_AUTH_COOKIE') && define('SECURE_AUTH_COOKIE', 'wp_sa_' . $cookie_hash);
 
/* @since 2.5.0 */
!defined('AUTH_COOKIE') && define('AUTH_COOKIE', 'wp_a_' . $cookie_hash);
 
/* @since 2.0.0 */
!defined('PASS_COOKIE') && define('PASS_COOKIE', 'wp_p_' . $cookie_hash);
 
/* @since 2.0.0 */
!defined('USER_COOKIE') && define('USER_COOKIE', 'wp_u_' . $cookie_hash);
 
/* ok unset this var, its not needed as COOKIEHASH will have this value, but is not definable in wp-config.php */
unset($cookie_hash);
 
/** @since 1.2.0 */
!defined('COOKIEPATH') && define('COOKIEPATH', preg_replace('|https?://[^/]+|i', '', WP_HOME . '/' ) );
 
/** @since 1.5.0 */
!defined('SITECOOKIEPATH') && define('SITECOOKIEPATH', preg_replace('|https?://[^/]+|i', '', WP_SITEURL . '/' ) );
 
/** @since 2.6.0 */
!defined('ADMIN_COOKIE_PATH') && define( 'ADMIN_COOKIE_PATH', SITECOOKIEPATH . 'wp-admin' );
 
/** @since 2.6.0 */
!defined('PLUGINS_COOKIE_PATH') && define( 'PLUGINS_COOKIE_PATH', preg_replace('|https?://[^/]+|i', '', WP_PLUGIN_URL)  );
 
/** @since 2.0.0 */
!defined('COOKIE_DOMAIN') && define('COOKIE_DOMAIN', $_SERVER['SERVER_NAME']);

/**
  * The WP_CACHE setting, if true, includes the wp-content/advanced-cache.php script, when executing wp-settings.php.
  * For an advanced caching plugin to use, static because you would only want one
  * if ( defined('WP_CACHE') )@include WP_CONTENT_DIR . '/advanced-cache.php';
  */
!defined('WP_CACHE') && define('WP_CACHE', true);
 
/** WordPress Localized Language, defaults to en_US.
 *
 * Change this to localize WordPress.  A corresponding MO file for the chosen
 * language must be installed to wp-content/languages. For example, install
 * de.mo to wp-content/languages and set WPLANG to 'de' to enable German
 * language support. */
!defined('WPLANG') && define ('WPLANG', 'en_US');
 
/** Stores the location of the language directory. First looks for language folder in WP_CONTENT_DIR
 *   and uses that folder if it exists. Or it uses the "languages" folder in WPINC. @since 2.1.0 */
//!defined('WP_LANG_DIR') && define('WP_LANG_DIR', ABSPATH . WPINC . '/languages');
 
/** LANGDIR defines what directory the WPLANG .mo file resides. If LANGDIR is not defined WordPress looks first to wp-content/languages and then wp-includes/languages for the .mo defined by WPLANG file.  Old static relative path maintained for limited backwards compatibility - won't work in some cases*/
//!defined('LANGDIR') && define('LANGDIR', 'wp-content/languages');
 
/** Stores the location of the WordPress directory of functions, classes, and core content. @since 1.0.0 */
//!defined('WPINC') && define('WPINC', 'wp-includes');

WPMU Stuff

I personally don’t use.

/** Allows for the mu-plugins directory to be moved from the default location. @since 2.8.0 */
//!defined('WPMU_PLUGIN_DIR') && define( 'WPMU_PLUGIN_DIR', WP_CONTENT_DIR . '/mu-plugins' ); // full path, no trailing slash
 
/** Allows for the mu-plugins directory to be moved from the default location. @since 2.8.0 */
//!defined('WPMU_PLUGIN_URL') && define( 'WPMU_PLUGIN_URL', WP_CONTENT_URL . '/mu-plugins' ); // full url, no trailing slash
 
/** Allows for the mu-plugins directory to be moved from the default location. @since 2.8.0 */
//!defined( 'MUPLUGINDIR' ) && define( 'MUPLUGINDIR', 'wp-content/mu-plugins' ); // Relative to ABSPATH.  For back compat.

WordPress Database

This is usually the only thing I have to manually edit when creating a new site, unless I just use the same DB and modify the $table_prefix, (farther down). I run everything I possibly can in UTF-8, but if you don’t already know alot about character sets, wow it is one of the most confusing things so you may want to save learning about that topic for another day. Otherwise the following are helpful (and show how confusing character sets are!)

If you ever setup WP to use the builtin membership features, make sure you learn about the CUSTOM_USER_TABLE and CUSTOM_USER_META_TABLE constants, I’ve found them very helpful.

/**#@+
 * MySQL settings
 */
/** The name of the database for WordPress */
define('DB_NAME', 'askapachewpblog75');
 
/** The username to access the database */
define('DB_USER', 'askapache245d');
 
/** The password for the username to access the database */
define('DB_PASSWORD', 'asdfklj2340');
 
/** The hostname to connect to the database at */
define('DB_HOST', 'mysql.askapache.com');
 
/** The charset of the database */
define('DB_CHARSET', 'utf8');
 
/** The collation of the database */
define('DB_COLLATE', 'utf8_general_ci');

$table_prefix

The $table_prefix is the value placed in the front of your database tables. Change the value if you want to use something other than wp_ for your database prefix. Typically this is changed if you are installing multiple WordPress blogs in the same database, and also for enhanced security.

Its a safe and good idea to change this value pre-installation to add more security to your WordPress blog. Exploits attempted against your WordPress blog by malicious crackers often are built with the premise that your blog uses the prefix wp_, by changing the value you mitigate some attack vectors.

/**
 * WordPress Database Table prefix.
 *
 * You can have multiple installations in one database if you give each a unique
 * prefix. Only numbers, letters, and underscores please!
 */
$table_prefix  = 'ar15_';
 
/** CUSTOM_USER_TABLE and CUSTOM_USER_META_TABLE are used to designated that the user and usermeta tables normally utilized by WordPress are not used, instead these values/tables are used to store your user information. */
//!defined('CUSTOM_USER_TABLE') && define('CUSTOM_USER_TABLE', $table_prefix . 'my_users');
//!defined('CUSTOM_USER_META_TABLE') && define('CUSTOM_USER_META_TABLE', $table_prefix . 'my_usermeta');

Setup PHP Ini Settings

 
/** Turns the output of errors on or off, you really never want this on, you should only view errors by reading the log file. */
ini_set('display_errors', WP_DEBUG_DISPLAY);
 
/** Tells whether script error messages should be logged to the server's error log or error_log. */
ini_set('log_errors', 'On');
 
/** http://us.php.net/manual/en/timezones.php */
ini_set('date.timezone', 'America/Indianapolis');
 
/** Where to log php errors */
ini_set('error_log', ASKAPACHE_ROOT . '/logs/php_error.log');
 
/** Set the memory limit, otherwise defaults to '32M' */
ini_set('memory_limit', WP_MEMORY_LIMIT);

Sessions are slow

So I only use sessions when I have a specific use… In this case I need sessions only when one of the tools in the /online-tools/ directory is being used. And that is for the captcha image. In the future I won’t ever use sessions.

if(preg_match( '#^/online-tools/#',$_SERVER['REQUEST_URI'])) session_start();

Include Custom Files

Sure you could use the my-hacks.php that WP allows, or you can just stick your functions in your TEMPLATEPATH/functions.php file, but they are executed only after the wp-settings.php file, which may be too late for your file.

In the past I’ve also used the auto_prepend_file settings to run my script before anything (index.php) but I ran into some issues on different hosts, and it wasn’t as portable.

This is useful because you can have a file with globally available functions that you can use in non-WP areas as well as WP areas. I am moving away from this more and more as I learn more about classes and build plugins instead for portability.

include_once ASKAPACHE_ROOT . '/includes/myfunctions.inc';
 
/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');
?>

Some Useful PHP

I am constantly trying to make my sites and code more portable, so I am using plugins alot more to accomplish things that I use to do with separate php. Here are some examples of minimal php.

add_filter("the_generator", create_function('$a','return "";'));
add_filter('the_content', create_function('$a', 'return ((is_feed())? $a."".get_the_title()." originally appeared on ".get_bloginfo("name")."." : $a);'), 99999);
add_filter('excerpt_length', create_function('$a', 'return 300;'),99);
add_filter('excerpt_more', create_function('$a', 'return "…";'),99);
add_action( 'wp_head', create_function('$a','echo "\n";'), 95 );
add_action( 'wp_head', create_function('$a','echo "\n";'), 96 );
add_action( 'wp_head', create_function('$a','echo "\n";'), 97 );
add_action( 'wp_head', create_function('$a','echo "\n";'), 98 );
add_action( 'wp_head', create_function('$a','echo "\n";'), 99 );

Debugging Note

If you read this far than you probably know how important debugging is, but I sometimes like to stick the best tips deep in my articles to make sure only YOU find it. GRTFM isn’t used on this site, it’s mostly a requirement because my writing can get pretty bad.. The point, debugging is more than a crucial requirement if you want to do anything cool. Don’t worry I got you.. check my AskApache Debug Viewer Plugin from the official WP site. It’s pretty close to providing as verbose amount of information that I could possibly figure out how to get out of php, probably more than you have ever seen at least, I focused on quantity. I use it all the time on new installs as there is no setup required and it tells me advanced information about the setup of the server, hacker code for sure.

Here’s a quick function to see set global vars, I just think this is interesting code.

function askapache_global_debug(){
  global $_GET,$_POST,$_COOKIE,$_SESSION,$_ENV,$_FILES,$_SERVER,$_REQUEST,$HTTP_POST_FILES,$HTTP_POST_VARS,$HTTP_SERVER_VARS,$HTTP_RAW_POST_DATA,$HTTP_GET_VARS,$HTTP_COOKIE_VARS,$HTTP_ENV_VARS;
  $gv=create_function('$n','global $$n; ob_start(); if ( is_array($$n) && sizeof($$n)>0 && print("[{$n}]\n") ) print_r($$n);return ob_get_clean();');
  foreach (array('_GET','_POST','_COOKIE','_SESSION','_ENV','_FILES','_SERVER','_REQUEST','HTTP_POST_FILES','HTTP_POST_VARS','HTTP_SERVER_VARS','HTTP_RAW_POST_DATA','HTTP_GET_VARS','HTTP_COOKIE_VARS','HTTP_ENV_VARS') as $k)echo $gv($k);
  print_r(get_defined_constants());
}

Also check the WordPress Codex page: Editing wp-config.php

Advanced WordPress wp-config.php Tweaks originally appeared on AskApache.

mod_rewrite Fix for Caching Updated Files

AskApache — Sun, 30 Aug 2009 13:20:59 +0000

Web Developers sometimes use file.ext?v=137 as a version control system so they can force visitors to use an updated file. This is so terrible. Instead link to apache-003.css and set it to be cached forever. When you change the file you just change the links to apache-004.css.

Raw Speed Benefit

This eliminates millions of bandwidth and resource robbing 304 If-Modified-Since requests.

Renaming links vs. Renaming files

On the server my files are named apache.css and apache.js, but in the xhtml I point to them using the names apache-113.css and apache-113.js, after I change the file I just add 1 to the number, and the new file is cached. They are internally redirected to apache.css and apache.js (invisible to the user) The concept is similar to a “shortcut” in windows or a symlink in BSD. The trick is that I never actually rename the files on the server. I just rename them in the html. That means apache-135.css is served from the file apache.css but the browser/cache only see and know about apache-135.css.

XHTML

NOTE: You can do your own investigating of this sites source code and HTTP headers to see this whole system in action

mod_rewrite code for htaccess or httpd.conf

Updated: 10/20/2008

RewriteEngine On
RewriteBase /
RewriteCond %{THE_REQUEST} ^(GET|HEAD)\ /z/([a-z]+)/(.+)-([0-9]+)\.(js|css).*\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule .+ /z/%2/%3.%5 [NC,L]

Alternate mod_rewrite code

RewriteEngine On
RewriteBase /
RewriteRule ^([cij]+)(/?[a-z]*)/([a-z]+)-([0-9]+)\.([a-z]+)$ /$1$2/$3.$5 [L]

Ideal Caching Scheme

Ok so you want the xhtml to be the only file that isn’t cached without being validated, its simple to setup your own caching scheme.

Bad Cache information for a file with the `?v=foo` hack

This object will be fresh for 1 week. It can be validated with Last-Modified. This object requests that a Cookie be set; this makes it and other pages affected automatically stale; clients must check them upon every request. Because it contains a query (‘?’) in the URL, many caches will not keep this object.

mod_rewrite Fix for Caching Updated Files originally appeared on AskApache.

Custom bash_profile for Advanced Shell Users

AskApache — Sun, 23 Aug 2009 18:32:26 +0000

Looking for some advanced uses for the shell? Function examples? Advanced shell settings? Here is some of my best.

Want to get started without even having to read this page? Type the following in your shell.

curl -O http://z.askapache.com/askapache-bash-profile.txt && source askapache-bash-profile.txt

For those of you power users and server admins that use Bash, ksh, csh, vanilla sh, etc.., or if you are just passionate about shell scripting, because it allows you to get advanced tasks done fast and efficiently, not to mention automated automatically. I give you my .bash_profile file. You should edit it to fit your needs, (especially the exported vars like PATH, LDFLAGS, if you don’t understand something just comment it out) but it’s pretty universal because I work on alot of other people’s servers not to mention many different distros and platforms, and when I get hired to do some server work through a shell, I bring this script along for the ride.

askapache-bash-profile.txt

You can download the latest version: http://z.askapache.com/askapache-bash-profile.txt

The functions and variables below are the way bash sees them, using declare -f, and alias, to make it easier for you to read and understand them. The actual file at http://z.askapache.com/askapache-bash-profile.txt will always be the most updated version, as I use it personally. And it has the whole file the way I wrote it, meaning many extra notes and much simpler to follow. Enjoy!

#!/bin/bash
#
# To install the latest version
#
# curl -O http://z.askapache.com/askapache-bash-profile.txt && source askapache-bash-profile.txt
#
# To run automatically at login: In your ~/.bash_profile or similar login script do
# [[ -r path-to-askapache-bash-profile.txt ]] && source path-to-askapache-bash-profile.txt
#
 
AAPN='AskApache Bash Profile Script'
AAPV='5.6'
AAPT='2009-22-2009'
AAPS=`echo "$0" | sed -e 's,.*/,,'`
 
############################################################################################################################################################
# Copyright (C) 2009 www.AskApache.com
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see .
############################################################################################################################################################
 
# dont do anything for non-interactive shells
[[ -z "$PS1"  ]] && return
 
# bash defines the following built-in commands:
#   :, ., [, alias, bg, bind, break, builtin, case, cd, command, compgen, complete, continue, declare, dirs
#   disown, echo, enable, eval, exec, exit, export, fc, fg, getopts, hash, help, history, if, jobs, kill,
#   let, local, logout, popd, printf, pushd, pwd, read, readonly, return, set, shift, shopt, source, suspend
#   test, times, trap, type, typeset, ulimit, umask, unalias, unset, until, wait, while
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# The first digit selects the set user ID (4) and set group ID (2) and sticky (1) attributes.
# The second digit selects permissions for the user who owns the file: read (4), write (2), and execute (1)
# The third selects permissions for other users in the file's group, with the same values
# The fourth for other users not in the file's group, with the same values.
# [see man chmod, help umask]
#------------------------------------------------------------------------------------------------------------------------------------------------------------
umask 0022
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# Trapping Signals to Catch Errors
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# 1      2      3       4      5       6       7      8      9       10      11      12      13      14      15      17      18      19      20      21
# SIGHUP SIGINT SIGQUIT SIGILL SIGTRAP SIGABRT SIGBUS SIGFPE SIGKILL SIGUSR1 SIGSEGV SIGUSR2 SIGPIPE SIGALRM SIGTERM SIGCHLD SIGCONT SIGSTOP SIGTSTP SIGTTIN
# [see man bash, man signal, help trap]
#------------------------------------------------------------------------------------------------------------------------------------------------------------
trap 'echo -e "\n\n\n!!! TERMINATED SIG: [$?] AT LINE:$LINENO AFTER $SECONDS SECONDS !!!\n"|tee -a `tty 2>/dev/null`;' 15
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# Advanced Shell Limits
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# -S      use the `soft' resource limit
# -H      use the `hard' resource limit
# -a      all current limits are reported
# -c      the maximum size of core files created
# -d      the maximum size of a process's data segment
# -f      the maximum size of files created by the shell
# -l      the maximum size a process may lock into memory
# -m      the maximum resident set size
# -n      the maximum number of open file descriptors
# -p      the pipe buffer size
# -s      the maximum stack size
# -t      the maximum amount of cpu time in seconds
# -u      the maximum number of user processes
# -v      the size of v
# [see man getrlimit, help ulimit]
#------------------------------------------------------------------------------------------------------------------------------------------------------------
ulimit -S -c 0 # Don't want any coredumps
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# Advanced Shell (set)tings
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# e [errexit] Exit immediately if a command exits with a non-zero status.
# B [braceexpand] The shell will perform brace expansion.
# h [hashall] Remember the location of commands as they
# f [noglob]      Disable file name generation (globbing).
# H [histexpand]  Enable ! style history substitution.
# v [verbose]     Print shell input lines as they are read.
# x [xtrace]      Print commands and their arguments as they are executed.
# n [noexec]      Read commands but do not execute them.
# [history] Enable command history
# [see man bash, help set]
#------------------------------------------------------------------------------------------------------------------------------------------------------------
set +C +f +H +v +x +n -b -h -i -m -B
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# Advanced Shell (shopt)ions
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# cdable_vars             an argument to the cd builtin command that is not a directory is assumed to be the name of a variable dir to change to.
# cdspell                 minor errors in the spelling of a directory component in a cd command will be corrected.
# checkhash               bash checks that a command found in the hash table exists before execute it.  If no longer exists, a path search is performed.
# checkwinsize            bash checks the window size after each command and, if necessary, updates the values of LINES and COLUMNS.
# cmdhist                 bash attempts to save all lines of a multiple-line command in the same history entry.  Allows re-editing of multi-line commands.
# dotglob                 bash includes filenames beginning with a `.' in the results of pathname expansion.
# execfail                a non-int shell will not exit if it cannot execute the file specified as an argument to the exec builtin command, like int sh.
# expand_aliases          aliases are expanded as described above under ALIASES.  This option is enabled by default for interactive shells.
# extglob                 the extended pattern matching features described above under Pathname Expansion are enabled.
# histappend              the history list is appended to the file named by the value of the HISTFILE variable when shell exits, no overwriting the file.
# hostcomplete            and readline is being used, bash will attempt to perform hostname completion when a word containing a @ is being completed
# huponexit               bash will send SIGHUP to all jobs when an interactive login shell exits.
# interactive_comments    allow a word beginning with # to cause that word and all remaining characters on that line to be ignored in an interactive shell
# lithist                 if cmdhist option is enabled, multi-line commands are saved to the history with embedded newlines rather than using semicolon
# login_shell             shell sets this option if it is started as a login shell (see INVOCATION above).  The value may not be changed.
# mailwarn                file that bash is checking for mail has been accessed since the last checked, ``The mail in mailfile has been read'' is displayed.
# no_empty_cmd_completion bash will not attempt to search the PATH for possible completions when completion is attempted on an empty line.
# nocaseglob              bash matches filenames in a case-insensitive fashion when performing pathname expansion (see Pathname Expansion above).
# nullglob                bash allows patterns which match no files (see Pathname Expansion above) to expand to a null string, rather than themselves.
# progcomp                the programmable completion facilities (see Programmable Completion above) are enabled.  This option is enabled by default.
# promptvars              prompt strings undergo variable and parameter expansion after being expanded as described in PROMPTING above.
# shift_verbose           the shift builtin prints an error message when the shift count exceeds the number of positional parameters.
# sourcepath              the source (.) builtin uses the value of PATH to find the directory containing the file supplied as an argument.
# xpg_echo                the echo builtin expands backslash-escape sequences by default.
# [see man bash, help shopt]
#------------------------------------------------------------------------------------------------------------------------------------------------------------
shopt -s histappend cmdhist extglob no_empty_cmd_completion checkwinsize progcomp sourcepath cdspell cdable_vars checkhash histreedit histverify
shopt -u mailwarn; unset MAILCHECK # dont notify for new mail
 
#----------------------------
# CUSTOM SETTING VARIABLES
#----------------------------
 
# quicker, cleaner ref
export N6=/dev/null
 
# if a program exists in path
function have()
{ unset -v have; PATH=$PATH:/sbin:/usr/sbin:/usr/local/sbin; type $1 &>/$N6 && have="yes"; }
 
[[ -z ${USER:=""} ]] && export USER=`whoami 2>$N6 || id -un 2>$N6 || logname 2>$N6`
[[ -z ${HOME:=""} ]] && export HOME=$(command pwd -L)
 
export HOME=${HOME:-~};
export SHELL=${SHELL:-/bin/bash}
have tty && export SSHTTY=`tty 2>$N6`
 
export TZ='America/Indianapolis'
[[ -z "$HOST" ]] && export HOST=`hostname 2>$N6||uname -n 2>$N6`
 
# DEFINATELY EDIT THIS STUFF, is pretty good at setting path
#JAVA_HOME=/usr/local/jdk
#P=${PATH}:$HOME/bin:$HOME/sbin:/bin:/etc:/sbin:/usr/local/bin:/usr/sbin:/usr/X11R6/bin:/usr/bin:/usr/bin/mh:/usr/libexec:/etc/X11:/etc/X11/xinit:$HOME/.gem/ruby/1.8/bin
#P=${P}:/usr/local/dh/apache/template/bin:/usr/local/dh/apache2/template/bin:/usr/local/dh/apache2/template/build:/usr/local/dh/apache2/template/sbin
#P=${P}:/usr/local/dh/bin:/usr/local/dh/java/bin:/usr/local/dh/java/jre/bin:/usr/local/php5/bin:.
#PATH=$P
GP=; for p in ${PATH//:/ }; do [[ -d $p && -x $p ]] && [[ "${GP}s" == s ]] && GP=$p || GP=${GP}:$p; done; export PATH=$GP
 
#----------------------------
# LIBS / COMPILES - EDIT
#----------------------------
#LDFLAGS="-L${HOME}/lib -L/lib -L/usr/lib -L/usr/lib/libc5-compat -L/lib/libc5-compat -L/usr/i486-linuxlibc1/lib -L/usr/X11R6/lib"; LD_LIBRARY_PATH=$HOME/lib; CPPFLAGS="-I${HOME}/include"
#export LDFLAGS LD_LIBRARY_PATH CPPFLAGS
 
#----------------------------
# MAIL, PROGRAMS, EDITOR
#----------------------------
have nano && export EDITOR="`type -P nano`" && export VISUAL=$EDITOR
have lynx && export BROWSER="`type -P lynx`"
 
#alias man='man -H'
#[[ -r $HOME/.lynx/lynx.cfg ]] && export LYNX_CFG=$HOME/.lynx/lynx.cfg
#[[ -r $HOME/var/locatedb ]] && export LOCATE_PATH=$HOME/var/locatedb
 
alias more='less'
export PAGER=less
export LESSCHARSET='latin1'
have lesspipe && export LESSOPEN='|/usr/bin/lesspipe %s 2>&-' && export LESS='-i -N -w  -z-4 -g -e -M -X -F -R -P%t?f%f :stdin .?pb%pb\%:?lbLine %lb:?bbByte %bb:-...'
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# PROMPT
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# \a     an ASCII bell character (07)
# \d     the date in "Weekday Month Date" format (e.g., "Tue May 26")
# \D{format}  the format is passed to strftime(3) and the result is inserted into the prompt string;
# \e     an ASCII escape character (033)
# \h     the hostname up to the first `.'
# \H     the hostname
# \j     the number of jobs currently managed by the shell
# \l     the basename of the shell's terminal device name
# \n     newline
# \r     carriage return
# \s     the name of the shell, the basename of $0 (the portion following the final slash)
# \t     the current time in 24-hour HH:MM:SS format
# \T     the current time in 12-hour HH:MM:SS format
# \@     the current time in 12-hour am/pm format
# \A     the current time in 24-hour HH:MM format
# \u     the username of the current user
# \v     the version of bash (e.g., 2.00)
# \V     the release of bash, version + patchelvel (e.g., 2.00.0)
# \w     the current working directory
# \W     the basename of the current working directory
# \!     the history number of this command
# \#     the command number of this command
# \$     if the effective UID is 0, a #, otherwise a $
# \nnn   the character corresponding to the octal number nnn
# \\     a backslash
# \[     begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt
# \]     end a sequence of non-printing characters
#------------------------------------------------------------------------------------------------------------------------------------------------------------
 
case ${TERM:-dummy} in linux*|con80*|con132*|console|xterm*|vt*|screen*|putty|Eterm|dtterm|ansi|rxvt|gnome*|*color*)COLORTERM=yes; ;; *) COLORTERM=no; ;; esac; export COLORTERM
PS1="\n[`date +%m/%d`][\u@\h][\w]($SHLVL:\!)\n$ "; X=;R=;
[[ "$COLORTERM" == yes ]] && X='\033[1;37m' && R='\033[0;00m' && PS1="\n[`date +%m/%d`]\e[1;37m[\e[0;32m\u\e[0;35m@\e[0;32m\h\e[1;37m]\e[1;37m[\e[0;31m\w\e[1;37m]($SHLVL:\!)\n\[${X}\]\$ "
export PS1 R X
 
# If set, the value is executed as a command prior to issuing each primary prompt.
have tty && export SSHTTY=$(tty 2>$N6);
[[ -z "$SSHTTY" ]] || export PROMPT_COMMAND='echo -ne "\033]0;$USER:`id -gn 2>$N6`@$HOSTNAME `tty 2>$N6`  [`uptime 2>$N6|sed -e "s/.*: \([^,]*\).*/\1/" -e "s/ //g"` / `command ps aux|wc -l`]  `command w|wc -l` users \007"'
unset PROMPT_COMMAND
 
# used by functions below
export RJ=0;
declare -a CC;
export L1=$(seq -s_ 1 75|tr -d [0-9]);
export L2=$(echo $L1|tr '_' ' ')
 
#---------------
# ls aliases
#---------------
C=' --color=auto'
alias la="command ls -Al${C}"      # show hidden files
alias lx="command ls -lAXB${C}"    # sort by extension
alias lk="command ls -lASr${C}"    # sort by size
alias lc="command ls -lAcr${C}"    # sort by change time
alias lu="command ls -lAur${C}"    # sort by access time
alias lr="command ls -lAR${C}"     # recursive ls
alias lt="command ls -lAtr${C}"    # sort by date
alias lll="stat -c %a\ %N\ %G\ %U \$PWD/*|sort"
 
#---------------
# safety aliases
#---------------
alias chmod='command chmod -c'
alias mkdir='command mkdir -pv'
alias rm='command rm -v'
alias cp='command cp -v'
alias mv='command mv -v'
 
#---------------
# func aliases
#---------------
alias who='command who -ar -pld'
alias which='command type -path'
have tree || alias tree='command -lsFR'
have tree && alias tree='command tree -Csuflapi'
alias top='top -c'
alias vim='command vim --noplugin'
alias du='command du -kh'
alias df='command df -kTh'
[[ $UNAME != Linux ]] && have gsed && alias sed=gsed
#alias php='php -d display_errors=true'
#alias man='man -H'
#alias ldconfig="ldconfig -v -f ${HOME}/etc/ld.so.conf -C ${HOME}/etc/ld.so.cache"
alias path='echo -e ${PATH//:/\\n}'
 
alias pp='command ps -HAcl -F S -A f'
alias p='command ps -HAcl -F S -A f|uniq -w3'
alias ps2='command ps -H'
alias ps1='command ps -lFA'
 
alias df1='command df -iTa'
alias n="${EDITOR}"3
alias inice='ionice -c3 -n7 nice'
alias vdir="ls --format=long"
alias dir="ls --format=vertical"
alias killphp='pkill -9 -f php.cgi\|php5.cgi\|php-cgi\|php\|php4.cgi;pkill -13 -f php.cgi\|php5.cgi\|php-cgi\|php\|php4.cgi'
alias dsiz='du -sk * | sort -n --'
alias lessc='ccze -A |/usr/bin/less -R'
alias h='history'
alias j='jobs -l'
alias ..='cd ..'
alias wtf='watch -n 1 w -hs'
 
# spelling typos
alias kk='ll'
alias xs='cd'
alias vf='cd'
alias moer='more'
alias moew='more'
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# history setup
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# HISTCONTROL
#    If  set  to a value of ignorespace, lines which begin with a space character are not entered on the history list.
#    If set to a value of ignoredups, lines matching the last history line are not entered.  A value of ignoreboth combines the two options.
#    If unset, or if set to any other value than those above, all lines read by the parser are saved on the history list, subject to the value of HISTIGNORE.
# HISTFILE
#    The name of the file in which command history is saved (see HISTORY below).  The default value is ~/.bash_history.  If unset, the command history is not saved when an interactive shell exits.
# HISTFILESIZE
#    The  maximum number of lines contained in the history file.  When this variable is assigned a value, the history file is truncated, if necessary, to contain no more than that number of lines.
#    The default value is 500.  The history file is also trun-cated to this size after writing it when an interactive shell exits.
# HISTIGNORE
#    A colon-separated list of patterns used to decide which command lines should be saved on the history list.  Each pattern is anchored at the beginning of the line and must match the complete line (no implicit `*' is appended).
#    Each pattern is  tested against  the  line  after  the  checks  specified  by HISTCONTROL are applied.  In addition to the normal shell pattern matching characters, `&' matches the previous history line.
#    `&' may be escaped using a backslash; the backslash is removed before attempting a match.  The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTIGNORE.
# HISTSIZE
#    The number of commands to remember in the command history (see HISTORY below).  The default value is 500.
#
HISTMASTER=;HISTCONTROL=;HISTIGNORE=;HISTSIZE=;HISTFILESIZE=;HISTFILE=;HISTTIMEFORMAT=;
function asetup_history()
{
  HISTMASTER=${HISTMASTER:-$HOME/backups/.history/combined.log}
  HISTCONTROL=${HISTCONTROL:-'ignoreboth'}
  HISTTIMEFORMAT="%H:%M > "
  HISTIGNORE=${HISTIGNORE:-'clear:ll:ls:./_sbackup.sh:./_logview.sh:ps:updatedb:top'}
  HISTSIZE=${HISTSIZE:-5000000}
  HISTFILESIZE=${HISTFILESIZE:-5000000}
  HISTFILE=${HISTFILE:-$HOME/.bash_history}
  export HISTMASTER HISTCONTROL HISTIGNORE HISTSIZE HISTFILESIZE HISTFILE HISTTIMEFORMAT
}
 
function h1()
{ cat $HISTFILE $HISTMASTER | command grep $1 | sort | uniq | command grep --color=always $1; }
 
function h2()
{ cat $HISTFILE $HISTMASTER | command grep $1 | sort | uniq; }
 
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# personalized colors
#------------------------------------------------------------------------------------------------------------------------------------------------------------
# Attribute codes:        00=none 01=bold 04=underscore 05=blink 07=reverse 08=concealed
# Text color codes:       30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white
# Background color codes: 40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white
# NORMAL 00       # global default, although everything should be something.
# FILE 00         # normal file
# DIR 01;34       # directory
# LINK 01;36      # symbolic link.
# FIFO 40;33      # pipe
# SOCK 01;35      # socket
# DOOR 01;35      # door
# BLK 40;33;01    # block device driver
# CHR 40;33;01    # character device driver
# ORPHAN 40;31;01 # symlink to nonexistent file
# EXEC 01;32      # executables
LS_COLORS=;LS_OPTIONS=;
function asetup_colors(){
 local L; eval "`dircolors -b`";
 [[ "$COLORTERM" == no ]] && return
 export LS_COLORS
 unset LS_OPTIONS
}
 
function ascript_title(){
  clear; echo; echo; lin 0;
  echo -e "| ${CC[2]}             ___       __    ___                 __                     ${CC[0]} |"
  echo -e "| ${CC[2]}            / _ | ___ / /__ / _ | ___  ___ _____/ /  ___                ${CC[0]} |"
  echo -e "| ${CC[9]}           / __ |(_-$N6);export RMSO=$(tput rmso &>$N6);
#find . -type f -iname '*htaccess*' -print0 | xargs -0 grep -sn -i "THE_REQUEST" | sed "s/THE_REQUEST/${SMSO}\0$RMSO}/gI"
 
function repeat()
{ local i max=$1; shift; for ((i=1; i <= max ; i++)); do eval "$@"; done; }
 
function cuttail()
{ sed -n -e :a -e "1,${2:-10}!{P;N;D;};N;ba" $1; }
 
function quote()
{ echo \'${1//\'/\'\\\'\'}\'; }
 
function dequote()
{ eval echo "$1"; }
 
function get_pids_on_system()
{ command ps axo pid|sed 1d; }
 
function get_pgids_on_system()
{ command ps axo pgid|sed 1d; }
 
function get_users_on_system()
{ have getent &>$N6 && getent passwd|awk -F: '{print $1}' && return; awk 'BEGIN {FS=":"} {print $1}' /etc/passwd; }
 
function get_groups_on_system()
{ have getent &>$N6 && getent group|awk -F: '{print $1}' && return; awk 'BEGIN {FS=":"} {print $1}' /etc/group; }
 
function get_inet_interface_ips()
{ have ip && /bin/ip -o -f inet addr|sed -e '/inet 10./d' -e 's/\/24//g' |awk '{print $4}'|sed -e "s/\\n//g" -e '/.*\.0$/d'; }
 
function personalized_wget()
{
 exec 6>&1 ; exec > ${1:-$HOME/.wgetrc}
 echo 'header = Accept-Language: en-us,en;q=0.5'
 echo 'header = Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'
 echo 'header = Accept-Encoding: gzip,deflate'
 echo 'header = Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7'
 echo 'header = Keep-Alive: 300'
 echo 'user_agent = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6'
 echo 'referer = http://www.google.com'
 echo 'robots = off'
 exec 1>&6 6>&-
}
 
function mkdir()
{
 local e=0;
 for f in ${1+"$@"};do
 set fnord `echo ":$f"|sed -e 's/^:\//%/' -e 's/^://' -e 's/\// /g' -e 's/^%/\//'`;shift;pp=;for d in ${1+"$@"};do pp="$pp$d";
 case "$pp" in -*)pp=./$pp; ;; ?:)pp="$pp/";continue ;; esac; [[ ! -d "$pp" ]] && mkdir -v "$pp"||e=$?; pp="$pp/"; done;
 done; return  $e
}
 
function clean_exit()
{ lin 0;lin 1;lin 2 "COMPLETED SUCCESSFULLY";lin 1;lin 3; }
 
function dirty_exit()
{ echo "See ya..";sleep 1;exit; }
 
function set_window_title()
{ echo -n -e "\033]0;$*\007"; }
 
function pd()
{ echo -e  "\n ${CC[15]} ${1:-DONE} $R\n\n" >$SSHTTY; }
 
function cont()
{ local ans; echo -en "\n ${CC[15]}[ ${1:-Press any key to continue} ]$R\n"; read -n 1 ans; }
 
function do_sleep ()
{ echo -en "${CC[6]}${3:-.}" >$SSHTTY;while [[ -d /proc/$RJ ]];do sleep ${2:-3};echo -en "${3:-.}" >$SSHTTY;done;echo -e "${CC[0]}" >$SSHTTY&&sleep 1&&pd; }
 
function pwd()
{ command pwd -LP "$@"; }
 
function kill_jobs(){
 for i in `jobs -p`; do kill -9 $i; done; }
 
function beep_alarm()
{ local i; for i in `seq 0 ${1:-5}`;do echo -en "\a" && sleep 1; done; }
 
function lin()
{ case ${1:-1} in 0)echo -e "\n$R $L1"; ;; 1)echo -e "|$L2|"; ;; 2)echo -en "${R}|${CC[34]}";echo -en "${2:-1}"|sed -e :a -e 's/^.\{1,72\}$/ & /;ta' -e "s/\(.*\)/\1/";echo -e "${R} |${R}" ;; 3)echo -e "$R $L1$R$X\n\n"; ;; esac; }
 
function mp3info()
{ ls *.mp3 |xargs -ix 2>&1 ffmpeg -i x|grep -v "^Must" |grep -v "built\|libavutil\|libavcodec\|configuration\|FFmpeg\|libavformat"; }
 
function tailit()
{ clear;echo "p e a ma md all | FILENAME"; sh ${HOME}/scripts/logview.sh "${2:-askapache.com}" "${1:-all}"; }
 
function pm()
{ local S I=${1:-3};S=$SSHTTY;echo -en "$R\n";case ${2:-0} in 0)echo -en "${CC[6]}==> $X$I$R" >$S; ;; 3)echo -e "\n\n${CC[4]}:: $X$I$R\n" >$S; ;; esac; }
 
function yes_no()
{ local a YN=65; echo -en "${1:-Answer} [y/n] "; read -n 1 a; case $a in [yY]) YN=0; ;; esac; return $YN; }
 
function l()
{ command ls -AhFp --color=auto "$@"; }
 
function la()
{ du *|awk '{print $2,$1}'|sort -n|tr ' ' "\t"; }
 
function ll()
{ command ls -lABls1c --color=auto "$@"; }
 
function stat1()
{ local D=${1:-$PWD/*}; stat -c %a\ %A\ \ A\ %x\ \ M\ %y\ \ C\ %z\ \ %N ${D} |sed -e 's/ [0-9:]\{8\}\.[0-9]\{9\} -[0-9]\+//g' |tr  -d "\`\'"|sort -r; }
 
function stat2()
{ local D=${1:-$PWD/*}; stat -c %a\ %A\ \ A\ %x\ \ M\ %y\ \ C\ %z\ \ %N ${D} |sed -e 's/\.[0-9]\{9\} -[0-9]\+//g'|tr  -d "\`\'"|sort -r; }
 
#--=--=--=--=--=--=--=--=--=--=--#
# processes
#==-==-==-==-==-==-==-==-==-==-==#
function psu()
{ command ps -Hcl -F S f -u ${1:-$USER}; }
 
function ps()
{ [[ -z "$1" ]] && command ps -Hacl -F S -A f && return; command ps "$@"; }
 
function make_nice()
{ have nice || return; pm "Making Nice $$"; pm 0 0; command renice ${1:-19} -p $$; pd; }
 
function make_ionice()
{ have ionice || return; pm "Making IONice $$"; pm 0 0; command ionice -c${1:-3} -n7 -p $$; pd; }
 
function dump_ps_environment()
{ command ps aux | grep ${USER:0:3} | awk '{print $2}' | xargs -t -ipid cat /proc/pid/environ; }
 
function procinfo1()
{ PI=($(strace -s1 procinfo -a 2>&1|sed -e '/^op/!d' -e '/pro/!d' -e '/= -1/d'|sed -e 's%o.*"/proc/\(.*\)".*% \1%g')); for i in ${PI[*]}; do echo -e "\n---===[  /proc/$i  ]\n" && cat /proc/$i && echo -e "\n\n"; done; }
 
function pss(){
  local U PPS PL PX PXX UUS=( $(command ps uax|awk '{print $1}'|command tail -n +2|sort|uniq) ); UL=$((${#UUS[@]} - 1))
  exec 6>&1; exec > ~/proc.$$
  ps aux | grep ${USER:0:3} | awk '{print $2}' | xargs -t -ipid cat /proc/pid/environ
  for UX in $(seq 0 1 $UUS); do U=${UUS[$UX]}; PPS=( $(pgrep -u ${U}) ); PL=$((${#PPS[@]} - 1));
   for PX in $(seq 0 1 $PL); do PXX=${PPS[$PX]};echo -e "\n\n\n----- PROCESS ID: ${PXX} -----\n\n";cat /proc/${PXX}/cmdline 2>$N6 || echo;echo -e "\n\n"; command tree -Csuflapi /proc/Q/${PXX};done
  done
  exec 1>&6 6>&-; cat ~/proc.$$ | more
}
 
function ashow_motd()
{ [[ -r /etc/motd ]] && echo -e "\n\n${CC[2]}`head -n 7 /etc/motd | tail -n 6`${R}\n"; }
 
function ashow_calendar()
{ [[ -d /usr/share/calendar/ ]] && echo -en "\n${CC[5]}" && ( sed = $(echo /usr/share/calendar/calendar*) | sed -n "/$(date +%m\\/%d\\\|%b\*\ %d)/p" ) && echo -en "${R}"; }
 
function ashow_fortune()
{ [[ -x /usr/games/fortune ]] && echo -en "\n${CC[6]}" && /usr/games/fortune -s && echo -en "${R}"; }
 
function askapache()
{
  ascript_title
  ashow_motd
 
  have who && pm "Users" && pm "Logged In" 3 &&  command who -ar -pld
  pm "Current Limits" 3;command ulimit -a
 
  pm "Machine stats";
  have uptime && pm "uptime" 3 && command uptime
  [[ -d /proc ]] && [[ -f /proc/meminfo ]] && cat /proc/meminfo
  have who && pm "Users" 3 && command who
 
  pm "Networking"
  have ip && pm "interfaces" 3 && ip -o addr|sed -e 's/ \{1,\}/\t/g'
  [[ -r /proc/net/sockstat ]] && pm "Sockets" 3 && head -n 2 /proc/net/sockstat
  have ss && pm "Networking Stats" 3 && ss -s
  have netstat && pm "Routing Information" 3 && netstat -r
 
  pm "Disk"
  pm "Mounts" 3; command df -hai
  #[[ -d /sys/block/ ]] && pm "IO Scheduling" 3; for d in /sys/block/[a-z][a-z][a-z]*/queue/*; do [[ -d $d ]] && tree $d; echo "$d => $(cat $d)";done
  have iostat && pm "I/O on Disks" && iostat -p ALL
 
  pm "Processes";
  pm "process tree" 3;  command ps -HAcl -F S -A f | uniq -w3
  have procinfo && pm "procinfo" 3 && procinfo|head -n 13|tail -n 11
}
 
asetup_colors
asetup_history
askapache

Custom bash_profile for Advanced Shell Users originally appeared on AskApache.

An AskApache Plugin Upgrade to Rule them All

AskApache — Wed, 29 Jul 2009 17:59:07 +0000

So my blog as been rather quiet for almost a year now, and very few updates if any have been released for my Password Protection PLugin, my Google 404 Plugin, and definately not for my AskApache CrazyCache plugin, which I will be releasing last… So for all of you who’ve helped me out by sending me suggestions and notifying me of errors and sticking with it… Just wanted to say sorry about that, and thanks for all the great ideas.. Well, I’ve been sticking with it as well believe it our not. I manage to get free days once in a while, and then its time to jam.

I’ve used just about every CMS/Blog/Forum/Trac/Gallery/etc) and really didn’t like a lot of the way they coded… I could use php but I didn’t KNOW php.. so I’ve had to learn php also, and it was tough to learn the advanced class usage and all the other language specific (but similar) constructs for php. It was especially difficult (but fun and challenging) to program so as to be compatible with php4 and php5 (Such is WordPress). But I kept at it, and soon you can decide for yourself what to make of it.

I can code in plenty of languages (bash, lua, windows .bat and vbs, ocaml, big fan of all things shell) and can work my way through C and even sorta somewhat with assembly. Assembly is the hardest, by far, I’m into easy and powerful languages like Python, Javascript, perl, php, ruby, and CGI. I’ve used PHP for a long time to do various things, but never to build software projects like this. Once I noticed WordPress’s core .php files and the excellent programming I wanted to try and learn hot to do it. The WordPress code is some of the best I’ve seen. It appears the way they built it was planned, and not just dreamt up while typing that I can’t help but do. Every time I read through the core code I learn a new trick or very nice way to do something. Those guys are really good, and I think WordPress is going to dominate for a long long time.

The Strategy

The Password Protection (passpro) plugin has a lot of complex stuff going on, especially for a newbie to PHP and WordPress like me, so after refactoring the whole thing at least 5 times I decided to modify my approach, and wrote the AskApache Google 404 Plugin as a way to practice on a simpler piece of code, while at the same time providing a plugin of value. Eventually I stopped thinking I could just code the whole thing in one sit-down with a stream-of-consciousness, and had to instead modularize the code and focus in on each part before moving to the next (I go without a plan because its fun, just not the most productive, but again, I’m not a programmer in the scientific sense.).

So I decided I had to really learn how WordPress Plugins work, filters, hooks, actions, and basically comfortability at reverse-engineering code, (Im a beginner for the last time), and so with the upcoming release of the AskApache Google 404 Plugin I have succeeded in making an incredibly stable plugin. That way I only have to worry about what the aapasspro plugin is doing, instead of trying to fit it into a framework.

AskApache Google 404 Upgrade

I think its rather unusual to develop a nice plugin like this 404 handler merely for the purpose of improving upon another plugin, but hey it worked. As of 08/03/2009 14:06PM EST I have about 1 hour left of finishing touches to release this upgrade. But as you cantell by my badly edited posts, I don’t have a lot of time to myself. An hour here and there is about it. So it could be up to 2 weeks before I actually have the time to commit the release to the repo. On a sidenote, have you checked out Windows 7 News? I’ve been contracted to do some technical work for them and thought they had an excellent site.

But keep in mind, the 404 PLugin is just where I practice for the passpro plugin, which truly does have features that no other software like it has ever had. I understand the technology behind this plugin, and know it would really have a great impact on improving the Web (esp. WordPress) for all of us, I’ve just had to learn how to make it.

AskApache Password Protection

Probably still a couple weeks away, this plugin is the ultimate culmination of apache hackers dreams, at least those on shared servers (who may be interested in learning how to bypass security of said servers).. So this is something I have much too fun with doing what I like to do.. network/protocol-level security. I’ve examined the source code for many software packages that I use or have used to audit a server’s security, and this simple php plugin in most instances can enumerate with accuraccy most of the server’s setup in about a minute. The catch (and the file permission problems I had to find a workaround too) is that this software is launched on the server, not remotely against the server.

Some of the software I examined was whiskers, nessus, nmap, hping, mozilla source, wireshark, ncftp, netcat, etc.. The closest comparison to the socket-level class I’ve hacked together to those is wireshark. Except that wireshark only interprets (captures) the data passing over the wire, while this class does that and in fact sends and receives the data like netcat or nmap. Its really more similar to metasploit, and can easily be used to send hex, binary, ascii, or any type of payload to the remote or local host.

The Upgrades Begin

Well I started working on them a long time ago. Both the Password Protection plugin and the Google 404 plugin needed serious work. And I finally have it all figured out. Essentially I would work on one and finish an upgrade, but I just wasn’t happy with it and I wold start all over again, refactoring the code. So as I put the finishing touches on those 2 plugins keep an eye out. They are major upgrades. I was able to meet all the goals I had for them, and came up with a lot of more improvements during the process.One of the main things I needed was a socket-level class to perform all kinds of checks and tests on. I need this also for my crazy cache plugin, which my blog is currently using , and I have a 2 more really nice pplugins I use that also needed access to a network class. I wrote about what I was doing with fsockopen, and I’ve been improving on that example ever since. I use this class to do some really powerful and exciting stuff, but you’ll see it soon enough. As an indication of ‘getting it right’ for the Password Protection plugin, the plugin will now work on Windows, Apache, IIS, Lighthttpd, and will even work running on a blackberry web server. So now everyone using wordpress can at least get some security()

Many of the the other improvements focus on using the fsockopen class and .htaccess tricks to basically enumerate and discover all the different capabilities of your particular server; That way you can learn about all the features and security that are possible for your specific server, and the securty modules wi8ll be geared for that as well. FINALLY this plugin is going to be stable, and I just cant wait to see how people react when they learn all great capability their Apache-based Server has that they didn’t have a clue about. Its amazing in that sense, and hackers will love theh way it works.. but your server admins will love it even more because its entirely 100% focused on helping you to set your site up (if you have Apache) to keep spammers out, to keep virii-serving robots and their log-hogging exploit requests and CPU/Mem robiing 404 errors off of your servers for real. This will have a noticeable affect to whoever is running the server. As you can tell.. I am pumped!

Apache is easy to configure and use, but only when you have root access. Most people on shared and private hosting aren’t even able to view the main config file, let alone execute the Apache binaries to see what features are available and what configuration is being used.

Apache can only be influenced by the main server configs and by .htaccess files. Not by php, not by perl, and the main configs are almost never accessible to the masses. But .htaccess files are. And many hosting providers allow and enable .htaccess files, a configuration file for your web server. The advanced features and capabilities of Apache were out of reach for most of us, it just wasn’t possible to enumerate or access, and most hosting providers are infamous for their lack of .htaccess (customer) support. This plugin goes around those problems to give the power back to the people.

y creating custom .htaccess files containing unpublished .htaccess tricks and techniques and combining that with the use of socket-level networking from WordPress (PHP) using fsockopen, we can effectively enumerate and discover an incredible amount of features and settings you will be able to control and use with this plugin.

Here are a few examples of the capabilities of this plugin, some of which I believe no other software can do.. (Open source free to copy!).

Current Version of Apache (Down to the API Version)
List of ALL Modules currently enabled by Apache (Such as Mod_Rewrite)
List of ALL Directives enabled by EACH enabled Module.
Enumerate .htaccess Overrides, Context Permissions
Test for any builtin Handlers (like the status handler screenshot)
Configure SSI (http://www.askapache.com/htaccess/advanced-htaccess-ssi.html#htaccess-ssi-security)

March 1, 2009
I would focus on the method that WordPress uses. The code they have now (2.8 bleeding-edge) still isn’t where it needs to be, but this is some difficult stuff and they have a brilliant start, it’ll work.. just a question of when.

The main issue with the password protection plugin working for some people and not others is due to file permission configurations. The plugin attempts to write/modify files in your blog’s root directory.

November 05, 2008
To make a long story short, I downloaded each major release of the apache httpd source code starting at version 1.3.0 and finishing with version 2.2.11, I then compiled each version and built a HTTPD from source for all these apache versions.

1.3.0

1.3.1

1.3.11

1.3.12

1.3.14

1.3.17

1.3.19

1.3.2

1.3.20

1.3.22

1.3.23

1.3.24

1.3.27

1.3.28

1.3.29

1.3.3

1.3.31

1.3.32

1.3.33

1.3.34

1.3.35

1.3.36

1.3.37

1.3.39

1.3.4

1.3.41

1.3.6

1.3.9

2.0.35

2.0.36

2.0.39

2.0.40

2.0.42

2.0.43

2.0.44

2.0.45

2.0.46

2.0.47

2.0.48

2.0.49

2.0.50

2.0.51

2.0.52

2.0.53

2.0.54

2.0.55

2.0.58

2.0.59

2.0.61

2.0.63

2.1.3-beta

2.1.6-alpha

2.1.7-beta

2.1.8-beta

2.1.9-beta

2.2.0

2.2.10

2.2.2

2.2.3

2.2.4

2.2.6

2.2.8

2.2.9

2.2.10

2.2.11

Then I went through each version and determined the compatible modules for that version, and I’m pretty confident that I was also able to find each and every directive allowed by the compatible modules for that version (including core directives). See .htaccess directive list. Basically I can now test a server using a variety of methods and determine almost 100% accurately what version of Apache (down to the API) is running, what modules (and versions) are enabled, and each and every directive that is allowed or disallowed for that version. So this is so awesome because now we can enable all sorts of additional security features.

Htaccess enabled Modules
Here are most of the modules that come with Apache. Each one can have new commands that can be used in .htaccess file scopes.

mod_actions, mod_alias, mod_asis, mod_auth_basic, mod_auth_digest, mod_authn_anon, mod_authn_dbd, mod_authn_dbm, mod_authn_default, mod_authn_file, mod_authz_dbm, mod_authz_default, mod_authz_groupfile, mod_authz_host, mod_authz_owner, mod_authz_user, mod_autoindex, mod_cache, mod_cern_meta, mod_cgi, mod_dav, mod_dav_fs, mod_dbd, mod_deflate, mod_dir, mod_disk_cache, mod_dumpio, mod_env, mod_expires, mod_ext_filter, mod_file_cache, mod_filter, mod_headers, mod_ident, mod_imagemap, mod_include, mod_info, mod_log_config, mod_log_forensic, mod_logio, mod_mem_cache, mod_mime, mod_mime_magic, mod_negotiation, mod_proxy, mod_proxy_ajp, mod_proxy_balancer, mod_proxy_connect, mod_proxy_ftp, mod_proxy_http, mod_rewrite, mod_setenvif, mod_speling, mod_ssl, mod_status, mod_substitute, mod_unique_id, mod_userdir, mod_usertrack, mod_version, mod_vhost_alias

Debugging HTTP protocol

Check this out! I’m particularly happy about this feature, which outputs an exact trace of any requests made by the plugin (such as during the testing phase) by saving the actual raw data sent out on the wire using fsockopen, RX and TX. This is useful for a number of reasons, viewing your headers, finding Redirect Loops, testing RewriteRules, and following the request hop-by-hop for debugging. The below example shows 2 requests for 2 URIs. The first URI is protected using Digest Authentication, the 2nd shows Basic.

 ______________
|  RAW TRACE   |
==================================================================================================================================
GET /htaccess/index.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
 
HTTP/1.1 401 Authorization Required
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
WWW-Authenticate: Digest realm="do or die", nonce="03328f3ec7c7b", algorithm=MD5, domain="/", qop="auth"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 882
Connection: close
Content-Type: text/html; charset=UTF-8
 
GET /htaccess/index.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
Authorization: Digest username="test",realm="do or die",nonce="03328f3ec7c7b",uri="/htaccess/index.txt?testing=query",
cnonce="82d057852a9dc497",nc=00000001,algorithm=MD5,response="9d476e9ea3",qop="auth"
 
HTTP/1.1 200 OK
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
Authentication-Info: rspauth="9051b01ee26dd62b3e2b40dada694f45", cnonce="82d057852a9dc497", nc=00000001, qop=auth
Last-Modified: Tue, 21 Jul 2009 23:56:00 GMT
Accept-Ranges: bytes
Cache-Control: max-age=3600
Expires: Wed, 22 Jul 2009 07:29:58 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 27
Connection: close
Content-Type: text/plain; charset=UTF-8
```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````
 
 ______________
|  RAW TRACE   |
==================================================================================================================================
GET /htaccess/po.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
 
HTTP/1.1 401 Authorization Required
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
WWW-Authenticate: Basic realm="Po Pimping"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 878
Connection: close
Content-Type: text/html; charset=UTF-8
 
GET /htaccess/po.txt?testing=query HTTP/1.1
Host: www.askapache.com
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1) AA_PassPro/1.9 (http://www.askapache.com/)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: close
Referer: http://www.askapache.com/
Authorization: Basic adfAGAltcA==
 
HTTP/1.1 200 OK
Date: Wed, 22 Jul 2009 06:29:58 GMT
Server: Apache
Last-Modified: Wed, 22 Jul 2009 05:54:39 GMT
Accept-Ranges: bytes
Cache-Control: max-age=3600
Expires: Wed, 22 Jul 2009 07:29:58 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 27
Connection: close
Content-Type: text/plain; charset=UTF-8
```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````

.htaccess Directives

AcceptFilter, AcceptMutex, AcceptPathInfo, AccessFileName, Action, AddAlt, AddAltByEncoding, AddAltByType, AddCharset, AddDefaultCharset, AddDescription, AddEncoding, AddHandler, AddIcon, AddIconByEncoding, AddIconByType, AddInputFilter, AddLanguage, AddModuleInfo, AddOutputFilter, AddOutputFilterByType, AddType, Alias, AliasMatch, AllowCONNECT, AllowEncodedSlashes, AllowOverride, Anonymous, Anonymous_Authoritative, Anonymous_LogEmail, Anonymous_MustGiveEmail, Anonymous_NoUserID, Anonymous_NoUserId, Anonymous_VerifyEmail, AssignUserId, AuthAuthoritative, AuthBasicAuthoritative, AuthBasicProvider, AuthDBDUserPWQuery, AuthDBDUserRealmQuery, AuthDBM, AuthDBMAuthoritative, AuthDBMGroupFile, AuthDBMType, AuthDBMUserFile, AuthDefaultAuthoritative, AuthDigestAlgorithm, AuthDigestDomain, AuthDigestFile, AuthDigestGroupFile, AuthDigestNcCheck, AuthDigestNonceFormat, AuthDigestNonceLifetime, AuthDigestProvider, AuthDigestQop, AuthDigestShmemSize, AuthGroupFile, AuthLDAPAuthzEnabled, AuthLDAPBindDN, AuthLDAPBindON, AuthLDAPBindPassword, AuthLDAPCharsetConfig, AuthLDAPCompareDNOnServer, AuthLDAPDereferenceAliases, AuthLDAPEnabled, AuthLDAPFrontPageHack, AuthLDAPGroupAttribute, AuthLDAPGroupAttributeIsDN, AuthLDAPRemoteUserAttribute, AuthLDAPRemoteUserIsDN, AuthLDAPStartTLS, AuthLDAPURL, AuthLDAPUrl, AuthName, AuthType, AuthUserFile, AuthzDBMAuthoritative, AuthzDBMType, AuthzDefaultAuthoritative, AuthzGroupFileAuthoritative, AuthzLDAPAuthoritative, AuthzOwnerAuthoritative, AuthzUserAuthoritative, BS2000Account, BalancerMember, BrowserMatch, BrowserMatchNoCase, BufferedLogs, CGIMapExtension, CacheDefaultExpire, CacheDirLength, CacheDirLevels, CacheDisable, CacheEnable, CacheExpiryCheck, CacheFile, CacheForceCompletion, CacheGcClean, CacheGcDaily, CacheGcInterval, CacheGcMemUsage, CacheGcUnused, CacheIgnoreCacheControl, CacheIgnoreHeaders, CacheIgnoreNoLastMod, CacheLastModifiedFactor, CacheMaxExpire, CacheMaxFileSize, CacheMaxStreamingBuffer, CacheMinFileSize, CacheNegotiatedDocs, CacheRoot, CacheSize, CacheStoreNoStore, CacheStorePrivate, CacheTimeMargin, CharsetDefault, CharsetOptions, CharsetSourceEnc, CheckCaseOnly, CheckSpelling, ChildPerUserId, ContentDigest, CookieDomain, CookieExpires, CookieLog, CookieName, CookieStyle, CookieTracking, CoreDumpDirectory, CustomLog, DAV, DAVDepthInfinity, DAVGenericLockDB, DAVMinTimeout, DBDExptime, DBDKeep, DBDMax, DBDMin, DBDParams, DBDPersist, DBDPrepareSQL, DBDriver, Dav, DavDepthInfinity, DavGenericLockDB, DavLockDB, DavMinTimeout, DefaultIcon, DefaultLanguage, DefaultType, DeflateBufferSize, DeflateCompressionLevel, DeflateFilterNote, DeflateMemLevel, DeflateWindowSize, Directory, DirectoryIndex, DirectoryMatch, DirectorySlash, DocumentRoot, DumpIOInput, DumpIOOutput, EnableExceptionHook, EnableMMAP, EnableSendfile, ErrorDocument, ErrorLog, Example, ExpiresActive, ExpiresByType, ExpiresDefault, ExtFilterDefine, ExtFilterOptions, ExtendedStatus, FancyIndexing, FileETag, Files, FilesMatch, FilterChain, FilterDeclare, FilterProtocol, FilterProvider, FilterTrace, ForceLanguagePriority, ForceType, ForensicLog, GprofDir, GracefulShutdownTimeout, Group, Header, HeaderName, HostNameLookups, HostnameLookups, ISAIPFakeAsync, ISAPIAppendLogToErrors, ISAPIAppendLogToQuery, ISAPICacheFile, ISAPIFakeAsync, ISAPILogNotSupported, ISAPIReadAheadBuffer, IdentityCheck, IdentityCheckTimeout, IfDefine, IfModule, IfVersion, ImapBase, ImapDefault, ImapMenu, Include, IndexIgnore, IndexOptions, IndexOrderDefault, IndexStyleSheet, KeepAlive, KeepAliveTimeout, LDAPCacheEntries, LDAPCacheTTL, LDAPCertDBPath, LDAPConnectionTimeout, LDAPOpCacheEntries, LDAPOpCacheTTL, LDAPSharedCacheFile, LDAPSharedCacheSize, LDAPTrustedClientCert, LDAPTrustedGlobalCert, LDAPTrustedMode, LDAPVerifyServerCert, LanguagePriority, Limit, LimitExcept, LimitInternalRecursion, LimitRequestBody, LimitRequestFields, LimitRequestFieldsize, LimitRequestLine, LimitXMLRequestBody, Listen, ListenBacklog, LoadFile, LoadModule, Location, LocationMatch, LockFile, LogFormat, LogLevel, MCacheMaxObjectCount, MCacheMaxObjectSize, MCacheMaxStreamingBuffer, MCacheMinObjectSize, MCacheRemovalAlgorithm, MCacheSize, MMapFile, MaxClients, MaxKeepAliveRequests, MaxMemFree, MaxRequestsPerChild, MaxSpareServers, MaxSpareThreads, MaxSpareThreadsPerChild, MaxThreads, MetaDir, MetaFiles, MetaSuffix, MimeMagicFile, MinSpareServers, MinSpareThreads, ModMimeUsePathInfo, MultiviewsMatch, NWSSLTrustedCerts, NWSSLUpgradeable, NameVirtualHost, NoProxy, NumServers, Options, PassEnv, PerlAccessHandler, PerlAuthenHandler, PerlAuthzHandler, PerlChildExitHandler, PerlChildInitHandler, PerlCleanupHandler, PerlDispatchHandler, PerlFixupHandler, PerlFreshRestart, PerlHandler, PerlHeaderParserHandler, PerlInitHandler, PerlLogHandler, PerlModule, PerlPassEnv, PerlPostReadRequestHandler, PerlRequire, PerlRestartHandler, PerlSendHeader, PerlSetEnv, PerlSetVar, PerlSetupEnv, PerlTaintCheck, PerlTransHandler, PerlTypeHandler, PerlWarn, PidFile, Port, Protocol, ProtocolEcho, Proxy, ProxyBadHeader, ProxyBlock, ProxyDomain, ProxyErrorOverride, ProxyFtpDirCharset, ProxyIOBufferSize, ProxyMatch, ProxyMaxForwards, ProxyPass, ProxyPassInterpolateEnv, ProxyPassMatch, ProxyPassReverse, ProxyPassReverseCookieDomain, ProxyPassReverseCookiePath, ProxyPreserveHost, ProxyReceiveBufferSize, ProxyRemote, ProxyRemoteMatch, ProxyRequests, ProxySet, ProxyStatus, ProxyTimeout, ProxyVia, RLimitCPU, RLimitMEM, RLimitNPROC, ReadmeName, Redirect, RedirectMatch, RedirectPermanent, RedirectTemp, RemoveCharset, RemoveEncoding, RemoveHandler, RemoveInputFilter, RemoveLanguage, RemoveOutputFilter, RemoveType, RequestHeader, Require, RewriteBase, RewriteCond, RewriteEngine, RewriteLock, RewriteLog, RewriteLogLevel, RewriteMap, RewriteOptions, RewriteRule, SSIAccessEnable, SSIEndTag, SSIErrorMsg, SSIStartTag, SSITimeFormat, SSIUndefinedEcho, SSLCACertificateFile, SSLCACertificatePath, SSLCADNRequestFile, SSLCADNRequestPath, SSLCARevocationFile, SSLCARevocationPath, SSLCertificateChainFile, SSLCertificateFile, SSLCertificateKeyFile, SSLCipherSuite, SSLCryptoDevice, SSLEngine, SSLHonorCipherOrder, SSLLog, SSLLogLevel, SSLMutex, SSLOptions, SSLPassPhraseDialog, SSLProtocol, SSLProxyCACertificateFile, SSLProxyCACertificatePath, SSLProxyCARevocationFile, SSLProxyCARevocationPath, SSLProxyCipherSuite, SSLProxyEngine, SSLProxyMachineCertificateFile, SSLProxyMachineCertificatePath, SSLProxyProtocol, SSLProxyVerify, SSLProxyVerifyDepth, SSLRandomSeed, SSLRequire, SSLRequireSSL, SSLSessionCache, SSLSessionCacheTimeout, SSLUserName, SSLVerifyClient, SSLVerifyDepth, Satisfy, ScoreBoardFile, Script, ScriptAlias, ScriptAliasMatch, ScriptInterpreterSource, ScriptLog, ScriptLogBuffer, ScriptLogLength, ScriptStock, SecureListen, SendBufferSize, ServerAdmin, ServerAlias, ServerLimit, ServerName, ServerPath, ServerRoot, ServerSignature, ServerTokens, SetEnv, SetEnvIf, SetEnvIfNoCase, SetHandler, SetInputFilter, SetOutputFilter, StartServers, StartThreads, Substitute, SuexecUserGroup, ThreadLimit, ThreadStackSize, ThreadsPerChild, TimeOut, Timeout, TraceEnable, TransferLog, TypeAuthDBMUserFile, TypesConfig, UnsetEnv, UseCanonicalName, UseCanonicalPhysicalPort, User, UserDir, VirtualDocumentRoot, VirtualDocumentRootIP, VirtualHost, VirtualScriptAlias, VirtualScriptAliasIP, Win32DisableAcceptEx, XBitHack, allow, deny, order, php_admin_flag, php_admin_value, php_flag, php_value

You can view the plugins home page, old, or view it on the wordpress.org site.

An AskApache Plugin Upgrade to Rule them All originally appeared on AskApache.

PHP and AJAX shell console

AskApache — Sun, 14 Jun 2009 01:01:15 +0000

Ever wanted to execute commands on your server through php to mimick a shell login? Now you can. I’m calling this file (see below) shell.php and it allows you to run commands on your web server with the same permissions that your php executable has.

Loading Video

PHP for `shell.php`

Substitue 1.1.1.1 for your IP address.. or see below for password authentication methods.






PHP AJAX Shell


:->

Read this

Note: The history feature works by remembering the last commands that you typed.. Access them by pressing the up or down arrows on your keyboard.

This is not an interactive session, so you cannot cd to a directory and then do stuff in that directory.. You may however be able to do stuff like /bin/bash -c "cd ../../;mv this there;ls -la;" or you could try exporting your current dir or something..

Writing shell scripts and serving them on your web server works by renaming the file.sh to file.cgi and chmodding it to 750 or +x. Also make sure you try dos2unix -dv file.cgi If you can’t get it to work..

Example shell script as cgi

#!/bin/sh
export MYBNAME=`date +%mx%dx%y-%Hx%M.tgz`
tar -czf ${HOME}/backups/${MYBNAME} ${HOME}/site1/
exit 0;

Locking Down Access to your shell.php

Thanks to the comment by Andrew Ramsden, Here are a couple ways to secure your shell.php file so that only you can run this script.

Secure your remote shell by adding this to your shell.php

Add this line to the very top of your shell.php file to make sure that only you can access this script. Everyone else sees a blank screen.

if ($_SERVER['REMOTE_ADDR'] !== '1.1.1.1') die();

Secure your remote shell with htaccess

This only allows access from IP 1.1.1.1 and redirects everyone else. See Using the Allow Directive in Apache htaccess for more info.

Order deny,allow
Deny from all
Allow from 1.1.1.1
ErrorDocument 403 http://www.askapache.com

Secure your remote shell with mod_rewrite and htaccess

Based on the code from htaccess article This only allows access from user with IP of 1.1.1.1 and redirects everyone else.

RewriteEngine On
RewriteBase /
RewriteCond %{REMOTE_HOST} !^1\.1\.1\.1
RewriteRule .* http://www.askapache.com [R=302,L]

PHP and AJAX shell console originally appeared on AskApache.

DNS Round Robin Configuration using Rsync over SSH

AskApache — Tue, 14 Apr 2009 05:46:20 +0000

My traffic is growing, alot, and I need to plan how I’m going to maintain scalability, high availability, and redundancy. Scalability is an application’s ability to support a growing number of users. High availability can be defined as redundancy or speed. I decided to setup Round Robin DNS for z.askapache.com, which is the “static” subdomain of AskApache that serves all the static assets like images, javascript, css, etc.. (BTW, the z stands for ZAP).. All I needed to attempt setting this up was another hosting account on a separate server. I have hosting accounts with around 10 different companies from working with various clients over the years, like Powweb and I don’t use them because they suck in terms of the unix environment. Many of these web hosts are actually very fast bandwidth-wise..

Round Robin Concept

A few months ago I was given a free hosting account on HostGator to evaluate and tempt me away from DreamHost to become a Gator. I get a lot of these types of offers from time to time from the smaller Web companies who read AskApache.. but when I noticed the SSH access was jailed and saw how restrictive the shell was I felt like I was on a windows box.. I want a shell, cpanel sucks. I compile and run everything from the shell so thats was a deal-breaker and I sorta forgot all about it.

The goal is to add the HostGator server to be an exact mirror of the z.askapache.com domain, then to add that server as a 2nd A record to my DNS zone. That way half the visitors to the size will be taking up resources and bandwidth on the HostGator server instead of mine.

Round Robin A records in DNS are intended to evenly distribute queries between each host of the same name. Using some tricks straight out of a hackers toolbox we can verify if the distribution is taking place. (It is.)

DNS – Domain Name System

The Domain Name System is often analogized as a “phone book” for the Internet because it translates human-friendly computer hostnames into IP addresses. For example, www.askapache.com translates to 208.113.134.190. Every request for a human-friendly hostname first needs to be translated to the IP before the server can be queried. Normally each hostname corresponds to exactly 1 IP address. But in a Round Robin setup, the idea is to have the hostname correspond to multiple IP addresses, which are different servers that house the exact same content, resulting in some hosts requesting files from one server, and another host requesting files from the other server, resulting in less CPU resources and bandwidth.

Here is an the zone for z.askapache.com Round Robin records:

QUERY	TTL	CLASS	TYPE	ANSWER
z.askapache.com	13084	IN	A	64.111.114.111
z.askapache.com	13084	IN	A	69.56.174.114

Round Robin DNS

Round robin DNS is a technique of load distribution, load balancing, or fault-tolerance provisioning multiple, redundant Internet Protocol service hosts, e.g., Web servers, FTP servers, by managing the Domain Name System’s (DNS) responses to address requests from client computers according to an appropriate statistical model.

In its simplest implementation Round-robin DNS works by responding to DNS requests not only with a single IP address, but a list of IP addresses of several servers that host identical services. The order in which IP addresses from the list are returned is the basis for the term round robin. With each DNS response, the IP address sequence in the list is permuted. Usually, basic IP clients attempt connections with the first address returned from a DNS query so that on different connection attempts clients would receive service from different providers, thus distributing the overall load among servers.

Round robin DNS is often used for balancing the load of geographically-distributed Web servers. For example, a company has one domain name and three identical web sites residing on three servers with three different IP addresses. When one user accesses the home page it will be sent to the first IP address. The second user who accesses the home page will be sent to the next IP address, and the third user will be sent to the third IP address. In each case, once the IP address is given out, it goes to the end of the list. The fourth user, therefore, will be sent to the first IP address, and so forth.

Although easy to implement, round robin DNS has problematic drawbacks, such as those arising from record caching in the DNS hierarchy itself, as well as client-side address caching and reuse, the combination of which can be difficult to manage. Round robin DNS should not solely be relied upon for service availability. If a service at one of the addresses in the list fails, the DNS will continue to hand out that address and clients will still attempt to reach the inoperable service.

Does Round Robin Work

Definately. I can look at the access logs for both servers and see that they are splitting the requests nicely. It is definately not an exact split however, look at these statistics and you’ll see what I mean.

$ dig @ns1.dreamhost.com +authority +all z.askapache.com
 
;; ANSWER SECTION:
z.askapache.com.        14400   IN      A       69.56.174.114
z.askapache.com.        14400   IN      A       64.111.114.111
 
$ dig @ns1.dreamhost.com +authority +all z.askapache.com
 
;; ANSWER SECTION:
z.askapache.com.        14400   IN      A       64.111.114.111
z.askapache.com.        14400   IN      A       69.56.174.114

The effects of caching will distort the effectiveness of any IP address allocation algorithm unless a 0 TTL is used which has the effect of significantly increasing the load on the DNS (and is not always implemented consistently). In this case the cure may be worse than the disease Good news we have good load balancing on our web servers. Bad news we need 17 more DNS servers!. Intuitively, and without running any experiments to verify, we would suggest that given a normal TTL (12 hours or more) and ANY IP allocation algorithm other than a single static list, loads should be reasonably balanced .

Full root server query

Tracing to z.askapache.com[a] via A.ROOT-SERVERS.NET, maximum of 3 retries
A.ROOT-SERVERS.NET [.] (198.41.0.4)
 |\___ L.GTLD-SERVERS.NET [com] (192.41.162.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ D.GTLD-SERVERS.NET [com] (192.31.80.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ J.GTLD-SERVERS.NET [com] (192.48.79.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ H.GTLD-SERVERS.NET [com] (192.54.112.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ E.GTLD-SERVERS.NET [com] (192.12.94.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ F.GTLD-SERVERS.NET [com] (192.35.51.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ I.GTLD-SERVERS.NET [com] (192.43.172.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ G.GTLD-SERVERS.NET [com] (192.42.93.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ B.GTLD-SERVERS.NET [com] (2001:0503:231d:0000:0000:0000:0002:0030) Not queried
 |\___ B.GTLD-SERVERS.NET [com] (192.33.14.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ A.GTLD-SERVERS.NET [com] (2001:0503:a83e:0000:0000:0000:0002:0030) Not queried
 |\___ A.GTLD-SERVERS.NET [com] (192.5.6.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ C.GTLD-SERVERS.NET [com] (192.26.92.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |\___ M.GTLD-SERVERS.NET [com] (192.55.83.30)
 |     |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
  \___ K.GTLD-SERVERS.NET [com] (192.52.178.30)
       |\___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
       |\___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
        \___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       z.askapache.com -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       z.askapache.com -> 69.56.174.114

Live Online DNS Tools

Mirroring an Entire Site using Rsync over SSH

AskApache — Fri, 10 Apr 2009 11:40:44 +0000

Sometimes there is an urgent need for creating an exact duplicate or “mirror” of a web site on a separate server. This could be needed for creating Round Robin Setups, Load-Balancing, Failovers, or for just plain vanilla backups. In the past I have used a lot of different methods to copy data from one server to another, including creating an archive of the whole directory and then using scp to send the file over, creating an archive and then encrypting it and then sending that file over using ftp, curl, etc., and my persistence at learning new ways to do things has paid off because now I use rsync to keep an exact replica of the entire directory on an external server, without having to use all the CPU and resources of other mirroring methods.

For this article I will show how I setup a web mirror of z.askapache.com from my DreamHost server to my HostGator Server. For the transfer and synchronization of the directories we will be using rsync over SSH. We will also be automating this task using a cronjob with no user-interaction, so creating public and private keys for the ssh will be neccessary. Finally, I provide a simple shell script that prevents anyone from logging into your account with the created keys for any purpose other than to rsync.

Table of Contents

Rsync Synchronization Magic
Securing Rsync with SSH
Generate Keys with No Password
Custom SSH Connection Info
Creating Cronjob for Synchronization
Only Allow rsync
Rsync/SSH Debugging and Stats
Rsync Algorithm

Rsync Synchronization Magic

rsync: rsync is an open source file transfer program for Unix systems that uses the “rsync algorithm” which provides a very fast method for synchronizing files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand.

Securing Rsync with SSH

I NEVER transfer any unencrypted data around unless that data is transported encrypted and to a trusted recipient, (I use HTTPS for WordPress administration) and I haven’t had time to probe the HostGator system for security issues yet, so right away I decided I needed an automated way to securely transfer z.askapache.com files TO hostgator, while not allowing my hostgator account access back on my main server. So if the hostgator account were to get hacked somehow, the cracker/spammer wouldn’t have access back to my main server.

Generate Keys with No Password

First I created a private key, specifically a passwordless host key, meaning to gain access with ssh you only need to supply the key, not a password+key like normal.

[local@dreamhost] $ mkdir -p ~/.ssh && chmod 700 ~/.ssh
 
# Create the public and private keys
# public key at: z.askapache-hostgator.id_rsa.pub
# private key at: z.askapache-hostgator.id_rsa
[local@dreamhost] $ ssh-keygen -t rsa -b 2048 -f ~/.ssh/z.askapache-hostgator.id_rsa
 
# add the public key to remote hosts ~/.ssh/authorized_keys file
[local@dreamhost] $ ssh-copy-id -i ~/.ssh/z.askapache-hostgator.id_rsa.pub remoteuser@remotehost
 
# or use scp + ssh to add the public key
[local@dreamhost] $ scp ~/.ssh/z.askapache-hostgator.id_rsa.pub remoteuser@remotehost:/home/remoteuser/
[local@dreamhost] $ ssh remoteuser@remotehost
[gatoraskapache@gator] $ mkdir -p ~/.ssh && chmod 700 ~/.ssh
[gatoraskapache@gator] $ cat ~/z.askapache-hostgator.id_rsa.pub >> ~/.ssh/authorized_keys
[gatoraskapache@gator] $ chmod 600 ~/.ssh/authorized_keys

Custom SSH Connection Info

This helps alot, by adding this to the very top of my ~/.ssh/config file I don’t have to add all this stuff to the rsync command-line. Basically when I reference connecting to the host ‘gator’ it uses all these options. Very helpful and you can add as many entries as you want.

Host gator
   IdentityFile ~/.ssh/z.askapache-hostgator.id_rsa
   Port 2222
   Protocol 2
   User gatoraskapache
   HostName gator555.hostgator.com
   PasswordAuthentication no

Creating Cronjob for Synchronization

This cronjob runs every 30 minutes, copying all modified files from my local directory ~/z.askapache.com/ to the remote directory ~/public_html/z/. First I backup the current crontab, then I edit the crontab and add this.

crontab -l > .crontab-`date +%F.backup`; crontab -e

*/30 * * * * /usr/bin/rsync -e 'ssh' -rt --delete ~/z.askapache.com/ gator:'~/public_html/z/' 1>/dev/null
@midnight /usr/bin/find ~/z.askapache.com/ -type d ! -perm 755 -exec chmod 755 {} \; 1>/dev/null
@midnight /usr/bin/find ~/z.askapache.com/ -type f ! -perm 644 -exec chmod 644 {} \; 1>/dev/null

Those 2 find commands scheduled to run at midnight simply fix and permissions on files and directories in my static folder. They are all static files so there is no reason they need to have any other permission.

Only Allow rsync

This is SWEET! If you like candy that is.. It’s called each time anything logs into your machine using the password less key we created above, and it simple checks what command the login process is attempting to issue. To set this up you need to edit the ~/.ssh/authorized_keys file on the remote host and prefix your public key that you added with a command directive to execute a script:

command="/home/remoteuser/validate-rsync.sh" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwBhj6UCS7JbJ08C8pWJqCh2iXZMN7tXpYZh47f4gZZBwrNHZQ== localuser@dreamhost

validate-rsync.sh

For rsync requests it will always be rsync --server at the start of the command, so if the command is anything else then this script:

Sends you an email notifying you somethings up..
Moves the ~/.ssh folder to ~/.locked-ssh
Adds cronjob to move the folder back in about an hour.

#!/bin/bash
# Author: http://www.askapache.com
# Version: 1.2
# Date: 04-08-2009
 
# If the command used to login to ssh correctly starts with 'rsync --server'
# then exit this script and dont process the rest of the script
case "$SSH_ORIGINAL_COMMAND" in 'rsync --server'*) exit 0; ;; esac;
 
# the home directory where the .ssh folder is located
H=/home/remoteuser
 
# if there is a locked ssh folder, kill the rsync and die
[[ -d $H/.lssh ]] && echo "REJECTED" && exit 1
 
EMAIL=webmaster@askapache.com # notified about locked status
OC=$H/old-crontab.txt # the original crontab
NC=$H/new-crontab.txt #the new crontab
 
# When to unlock the rsync
UNLOCK_AT=$(( date +%M\ %k --date='30 minute 1 hour' ))
 
# move the .ssh to .lssh which locks all key-logins
mv $H/.ssh $H/.lssh
 
# mail a notice to the boss
echo $SSH_CONNECTION | mail -s 'RSYNC LOCKED!!!' "$EMAIL"
 
# subshell backs-up crontab then deletes active cron
(

 crontab -l > $OC &>/dev/null || echo -n #backup current crontab
 crontab -r >/dev/null 2>&1 || echo -n # delete current crontab
)
 
# subshell creates new crontab combined with old crontab
(
 # create new crontab
 echo -en "MAILTO='${EMAIL}'\n${UNLOCK_AT} * * * mv $H/.lssh $H/.ssh" >> $NC
 echo -n " && date|mail -s 'UNLOCKED!!!' '${EMAIL}' && crontab $OC || rm $OC && rm $OC" >> $NC

 # add old crontab to new crontab minus any MAILTO lines
 cat $OC | sed '/^MAILTO/d' >> $NC

 # load the new crontab and if it doesnt work notify boss
 crontab $NC || echo "manually mv .lssh to .ssh" mail -s 'CRONTAB PROBLEM!!!' "$EMAIL"

 # remove new crontab
 rm $NC
)
 
exit $?

Here is the cronjob entry it creates… This will enable the rsync again by moving the folder back, then it mails you to alert you that its back up, and finally the original crontab is restored.

MAILTO="askapache@gmail.com"
03 9 * * * mv ~/.locked-ssh ~/.ssh && date|mail -s 'RUNLOCKED!!!' "webmaster@askapache.com" && crontab ~/old-crontab.txt

Rsync/SSH Debugging and Stats

Adding the option -v to the ssh command, ie rsync -e 'ssh -vv' will give you a lot of debugging info.

By adding –stats to your rsync command you can get a detailed look at its efficacy.

Number of files: 14900
Number of files transferred: 0
Total file size: 1456832331 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 320551
File list generation time: 17.393 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 320571
Total bytes received: 20
 
sent 320571 bytes  received 20 bytes  17329.24 bytes/sec
total size is 1456832331  speedup is 4544.21

Rsync Algorithm

The rsync utility uses an algorithm (invented by the Australian computer programmer Andrew Tridgell) for efficiently transmitting a structure (such as a file) across a communications link when the receiving computer already has a different version of the same structure.

The recipient splits its copy of the file into fixed-size non-overlapping chunks, of size S, and computes two checksums for each chunk: the MD4 hash, and a weaker ‘rolling checksum’. It sends these checksums to the sender. Version 30 of the protocol (released with rsync version 3.0.0) now uses MD5 hashes rather than MD4.

The sender computes the rolling checksum for every chunk of size S in its own version of the file, even overlapping chunks. This can be calculated efficiently because of a special property of the rolling checksum: if the rolling checksum of bytes n through n + S − 1 is R, the rolling checksum of bytes n + 1 through n + S can be computed from R, byte n, and byte n + S without having to examine the intervening bytes. Thus, if one had already calculated the rolling checksum of bytes 1–25, one could calculate the rolling checksum of bytes 2–26 solely from the previous checksum, and from bytes 1 and 26.

The rolling checksum used in rsync is based on Mark Adler’s adler-32 checksum, which is used in zlib, and which itself is based on Fletcher’s checksum. The sender then compares its rolling checksums with the set sent by the recipient to determine if any matches exist. If they do, it verifies the match by computing the MD4 checksum for the matching block and by comparing it with the MD4 checksum sent by the recipient.

The sender then sends the recipient those parts of its file that did not match any of the recipient’s blocks, along with assembly instructions on how to merge these blocks into the recipient’s version. In practice, this creates a file identical to the sender’s copy. However, it is in principle possible that the recipient’s copy differs at this point from the sender’s: this can happen when the two files have different chunks that nonetheless possess the same MD4 hash and rolling checksum; the chances for this to happen are in practice extremely remote.

If the sender’s and recipient’s versions of the file have many sections in common, the utility needs to transfer relatively little data to synchronize the files.

While the rsync algorithm forms the heart of the rsync application that essentially optimizes transfers between two computers over TCP/IP, the rsync application supports other key features that aid significantly in data transfers or backup. They include compression and decompression of data block by block using zlib at sending and receiving ends, respectively, and support for protocols such as ssh that enables encrypted transmission of compressed and efficient differential data using rsync algorithm. Instead of ssh, stunnel can also be used to create an encrypted tunnel to secure the data transmitted.

Finally, rsync is capable of limiting the bandwidth consumed during a transfer, a useful feature that few other standard file transfer protocol offer.

This page contains content by Author of Article from Wikipedia and is licensed under the GNU FDL.

Mirroring an Entire Site using Rsync over SSH originally appeared on AskApache.

Password Protection Plugin Status

AskApache — Sun, 01 Mar 2009 17:39:57 +0000

I wanted to address why the update to the AskApache Password Protection plugin didn’t happen pre-2009 as I had hoped.. Mostly due to my job but I thought I could at least fill you in. Oh and this is going to get very boring very fast, unless you’re ready to rumble in the zone.

File Permissions!

The main issue with the password protection plugin working for some people and not others is due to file permission configurations. WordPress is simply a group of .php files saved on your server. The actual program that is in fact running WordPress is the PHP interpreter, which is in turn controlled by the Apache Server. Almost all computers are running at least 2 servers, the Web Server which serves and displays your files, and a FTP server.

Here’s a detailed look at the Apache Security Model, from ApacheSecurity.net, a blog maintained by Ivan Ristic, the author of ModSecurity.

The problem is happening because when you login to your FTP server with your username and password, the files that you upload are then owned by that username and password, which is almost always an actual user account on the server system. But the Apache Server is an executable file itself, and it is not owned by your FTP username, for security reasons. Apache controls the PHP Interpreter, which parses and executes the WordPress and plugin files as a separate user. ( SuEXEC, Apache Security Tips )

So what happens is the askapache-password-protect.php file saved on your server and is owned by the user that created it (if you downloaded it to your computer then used ftp to transfer, your ftp user owns it.. if you used a php downloader script, then the php process owner owns it) So when you click on the Run Tests button from the WordPress administration website what you are doing is sending a request via HTTP to your Apache Server process, which sees the requested file is .php so it then runs the php interpreter to execute the askapache-password-protect.php file, then that file uses programming to attempt and write/modify a file in your blog’s root directory.

Process Owner vs. File Owner

So who owns your blog’s root directory? Your ftp user account/ you do.. but who owns the process that is trying to write/modify a file that is owned by your ftp user? The PHP Process that is actually executing the file access/write requests. This is the core way that 99% of all web sites get cracked into.. All these malicious robots and exploit bots do is attempt to write a file onto your server so that it can then be used to take over your site. If they can save a file on your blog’s directory (uploads, insecure plugin code, not filtering user input, etc..) it inherits the permissions of the process that actually wrote the data bits onto the hard-drive.

So some server-admins/web hosts configure the php interpreter to not have write access anywhere except for a couple neccessary locations like /tmp. They have auto-installation’s available through their online web panels, meaning instead of executing .php scripts in your user directory as the php process they force you to use, they can bypass all that because the installation scripts they use are all on their systems, not on your “locked-down” cluster.

This is the fundamental security battle that network server security is all based on.. Apache is owned by a powerful user because it owns the server process, so apache is often run as the user dhapache or nobody.. If a cracker is able to find a way to get a file saved on your server with the dhapache user as the owner then they’ve basically just gotten control of the whole thing. When you upload a file to your server using the add attachment form in wordpress, the file first goes through the dhapache user which passes the file to the php process owner which has much less permissions. Apache has been in open-source development for many many years now, its the safest most secure server in the world, windows servers are hackable, apache servers are hacked usually only when the sysadmin configures it wrong or accidentally.

Believe it or not, as confusing as my feeble explanation was, this is only like .1% of whats going on.. I’ve basically spent the last several months developing the new version specifically to be able to work no matter what configuration you have. What I ended up doing was finding ways to bypass this security on a couple hosting providers that are setup in this way, but even though I got it to work in most instances it basically was hacking their systems, and if I published that code to automatically bypass web-hosts security setups I think I’d be in big trouble and they would just close those specific holes and the plugin would not work again. So I decided instead of exploiting host-specifics hacks to get the plugin to work that I would focus on the method that WordPress sorta uses. The code they have now (2.8 bleeding-edge) still isn’t where it needs to be, but this is some difficult stuff and they have a brilliant start, it’ll work.. just a question of when.

`wp-admin/includes/file.php`

Ok so this function get_filesystem_method is a brilliant bit of code that would’ve been beyond my current PHP skills to come up with. It determines which if any of the following methods can be used to modify files on your server from within WordPress, which is exactly what the new version of the passpro plugin needs to use. The first test simply creates a file from within php using wp_tempnam, a function that attempts to locate and write to a temporary location on your server that has the best chance of having write access. If it is successfully created (this code assumes that it will be, something they need to fix) then the fileowner (uses stat internally) of the temp file just created is compared to the owner of the php script… Normally this works and then the plugin woks too, but on some hosts the script is running as a separate user than that of the file which means you can’t directly access the local file system. That is what is occurring for most of you who experience permission problems while testing the plugin. There are thousands of caveats for each little part depending on your php version, php setup, server setup, server version, which Server API you are using, the type of SAPI being used, and on and on..

function get_filesystem_method($args = array()) {
  $method = false;
  if( function_exists('getmyuid') && function_exists('fileowner') ){
    $temp_file = wp_tempnam();
    if ( getmyuid() == fileowner($temp_file) ) $method = 'direct';
    unlink($temp_file);
  }
  if ( ! $method && isset($args['connection_type']) && 'ssh' == $args['connection_type'] && extension_loaded('ssh2') )
          $method = 'ssh2';
  if ( ! $method && extension_loaded('ftp') )
          $method = 'ftpext';
  if ( ! $method && ( extension_loaded('sockets') || function_exists('fsockopen') ) )
          $method = 'ftpsockets'; //Sockets: Socket extension; PHP Mode: FSockopen / fwrite / fread
  return apply_filters('filesystem_method', $method);
}

Enumerating Permissions can be Annoying

This was part of some tests I did to see what kind of access I had with the very helpful posix functions which are very accurate as well since they were designed for a system with file permissions, ie. not Win. Don’t ask me how because I won’t tell you, but on one of the hosts I was testing on that did not allow direct access I was able to get the Apache server running as dhapache to erroneously write a file into my users blog directory. This is a big security no-no and I now have my .htaccess file written into the blog directory where it should go, but instead of my php script’s user having write access to the file so I can modify it, its owned by dhapache! Because the file is owned by dhapache I shouldn’t even be allowed to know it exists, but there it is. So the next step was to try and take ownership of the .htaccess file so that I could modify it. I tried and tried but was unsuccessful, I couldn’t modify it so that was another dead end. Actually it took me awhile to figure out how to remove the file from my directory. Being that it was owned by dhapache I couldn’t delete or modify it using my php process or even through ftp/ssh! Sysadmins regularly run find commands that search the servers for any files owned by dhapache that should not be there as this is a big red flag that someone has found a way to manipulate dhapache which could potentially lead to modifying dhapche-owned server config files.. Luckily I was able to delete it by basically running the hack again to overwrite the file.

  if ((posix_setgid(getmygid())) !== false) $this->to_log('', 1,
      "Success Changing SETGID of {$file}  to " . getmygid(), 3);
  elseif ((posix_setgid(filegroup(__file__))) !== false) $this->to_log('', 1,
      "Success Changing SETUID of {$file} to " . filegroup(__file__), 3);
  if ((posix_setegid(getmygid())) !== false) $this->to_log('', 1,
      "Success Changing SETEGID of {$file} to " . getmygid(), 3);
  elseif ((posix_setegid(filegroup(__file__))) !== false) $this->to_log('', 1,
      "Success Changing SETEGID of {$file} to " . filegroup(__file__), 3);
  if ((posix_setuid(getmyuid())) !== false) $this->to_log('', 1,
      "Success Changing SETUID of {$file}  to " . getmyuid(), 3);
  elseif ((posix_setuid(get_current_user())) !== false) $this->to_log('', 1,
      "Success Changing SETUID of {$file} to " . get_current_user(), 3);
  if ((posix_seteuid(getmyuid())) !== false) $this->to_log('', 1,
      "Success Changing SETEUID of {$file} to " . getmyuid(), 3);
  elseif ((posix_seteuid(get_current_user())) !== false) $this->to_log('', 1,
      "Success Changing SETEUID of {$file} to " . get_current_user(), 3);
  if ((chmod($file, FS_CHMOD_DIR) || chmod($file, 0776) || chmod($file, 0766) || chmod($file,
    FS_CHMOD_FILE)) !== false) $this->to_log('', 1, "Success Changing Mode of {$file}", 3);
  if ((chown($file, getmyuid())) !== false) $this->to_log('', 1,
      "Success Changing Ownership of {$file} to " . getmyuid(), 3);
  elseif ((chown($file, get_current_user())) !== false) $this->to_log('', 1,
      "Success Changing Ownership of {$file} to " . get_current_user(), 3);
  if ((chgrp($file, getmygid())) !== false) $this->to_log('', 1,
      "Success Changing Group of {$file} to " . getmygid(), 3);
  elseif ((chgrp($file, filegroup(__file__))) !== false) $this->to_log('', 1,
      "Success Changing Group of {$file} to " . filegroup(__file__), 3);
  if ((chmod($file, FS_CHMOD_DIR) || chmod($file, 0776) || chmod($file, 0766) || chmod($file,
    FS_CHMOD_FILE)) !== false) $this->to_log('', 1, "Success Changing Mode of {$file}", 3);
  if ((chown($file, getmyuid())) !== false) $this->to_log('', 1,
      "Success Changing Ownership of {$file} to " . getmyuid(), 3);
  elseif ((chown($file, get_current_user())) !== false) $this->to_log('', 1,
      "Success Changing Ownership of {$file} to " . get_current_user(), 3);
  if ((chgrp($file, getmygid())) !== false) $this->to_log('', 1,
      "Success Changing Group of {$file} to " . getmygid(), 3);
  elseif ((chgrp($file, filegroup(__file__))) !== false) $this->to_log('', 1,
      "Success Changing Group of {$file} to " . filegroup(__file__), 3);
  return (!$this->_fclose($fh)) ? $this->to_log(__function__ . ':' . __line__ .
    " Error closing {$mode} handle for {$file}", 0) : $total;

If php process isn’t allowed to write to your web directory but you have an ftp account that is, then we request your ftp username/password in wordpress and if the php process running the askapache-password-protect.php plugin script is allowed access to raw networking sockets using fsockopen then we can basically access and write to your blog’s .htaccess file by using php to mimick an ftp client session. There are also other protocols and options available using php if ftp/fsockopen isn’t allowed, but you run out of alternatives quick. Using the curl extension is one option.

So I wrote my own ftp library for a fsockopen class I had already developed for specific test requirements in unreleased versions, so the release of the new askapache password protect plugin will work for 75% or so of the people who have trouble now.. not to mention the insane logging and debugging I’ve added while looking for the reasons some web-hosts still don’t work. Some use custom php security modules, wrappers, and custom virtual servers that are akin to a vmware server. So for maybe 10% of those running apache who have had problems they would still have them. I’m still playing with some ssh capability from within the plugin similar to the ftp technique.. I really hope WordPress just adds this functionality by updating their current filesystem classes..

Fsockopen Payload Class

Here’s what I had several versions ago.. Just sticking it up here in case anyone is curious, one cool thing this version starts to incorporate is being able to send direct data payloads across the socket so it can be used like the metasploit framework to send payloads of exploits, but of course we’re using it to mimick other protocols like ftp, which can be setup by feeding hex into the socket direct from a real ftp client, and piping the output. Keep in mind that this is my first time using php classes, so the learning curve has been incredible…

 '1.0', 'method' => 'GET', 'referer' =>
   'http://www.askapache.com/', 'port' => '80', 'ua' =>
   'Mozilla/5.0 (compatible; AskApache_Net/1.6; http://www.askapache.com/)', 'scheme' =>
   'http', 'transport' => '', 'host' => '', 'user' => '', 'pass' => '', 'path' => '/',
   'query' => '', 'fragment' => '');
  var $authtype = 'Basic';
  var $timeout = 15;
  var $_dh = '';
  var $_digest = array('realm' => '', 'nonce' => '', 'uri' => '', 'algorithm' => 'MD5',
   'qop' => 'auth', 'opaque' => '', 'domain' => '', 'nc' => '00000001', 'cnonce' =>
   '82d057852a9dc497', 'A1' => '', 'A2' => '', 'response' => '');
  var $_ACLF = "\r\n";
  var $_request_body = '';
  var $_request_headers = array();
  var $_response_headers = array();
  var $my_headers;
  var $_response_header = '';
  var $_response_protocol = '';
  var $_response_version = '';
  var $_response_code = '';
  var $_response_message = '';
  var $_response_body = '';
  var $_errs = array(3 => 'Socket creation failed', 4 => 'DNS lookup failure', 5 =>
   'Connection refused or timed out', 111 => 'Connection refused', 113 =>
   'No route to host', 110 => 'Connection timed out', 104 => 'Connection reset by client');

  /**
   * AskApache_Net::AskApache_Net()
   */
  function AskApache_Net()
  {
   return $this->__construct();
  }

  /**
   * AskApache_Net::__destruct()
   */
  function __destruct()
  {
   $this->_timer('class');
   return true;
  }

  /**
   * AskApache_Net::__construct()
   */
  function __construct()
  {
   $this->_timer('class');
   $this->_ACLF = chr(13) . chr(10);
   @set_time_limit(60);
   return true;
  }

  /**
   * AskApache_Net::hsockit()
   */
  function hsockit($URI)
  {
   $this->msg(__function__ . ':' . __line__, 3);
   $this->_socket['method'] = 'HEAD';
   return $this->sockit($URI);
  }

  /**
   * AskApache_Net::sockit()
   */
  function sockit($URI = '')
  {
   $this->msg(__function__ . ':' . __line__, 3);
   if (!$this->_build_sock($URI)) return $this->msg(__function__ . ':' . __line__,
     "Failed!", 0);
   if (!$this->_connect()) return $this->msg(__function__ . ':' . __line__, "Failed!", 0);
   $this->_build_request();
   if (!$this->_build_request()) return $this->msg(__function__ . ':' . __line__,
     "Failed!", 0);
   if (!$this->_tx()) return $this->msg(__function__ . ':' . __line__, "tx Failed!", 0);
   if (!$this->_rx()) return $this->msg(__function__ . ':' . __line__, "rx Failed!", 0);
   if (!$this->_disconnect()) return $this->msg(__function__ . ':' . __line__,
     "disconnect Failed!", 0);
   if ((bool)$this->net_debug === true) {
    foreach (array('out_payload', '_request_body', '_response_header', '_response_body') as
     $nam) {
     if (is_array($this->$nam)) {
      if (sizeof($this->$nam) > 1) {
       echo "\n\n{$nam}\n";
       print_r($this->$nam);
      }
     } else {
 
      if (!empty($this->$nam)) {
       echo "\n\n{$nam}\n";
       echo $this->$nam;
      }
     }
    }
    $this->tcp_trace(1);
   }
   return (int)$this->_response_code;
  }

  /**
   * AskApache_Net::_build_sock()
   */
  function _build_sock($url)
  {
   $this->msg(__function__ . ':' . __line__, 3);
   $socket_info = &$this->_socket;
   if (!$u_bits = parse_url($url)) return false;
   if (empty($u_bits['method'])) $u_bits['method'] = 'GET';
   if (empty($u_bits['protocol'])) $u_bits['protocol'] = '1.0';
   if (empty($u_bits['host'])) $u_bits['host'] = $_SERVER['HTTP_HOST'];
   if (empty($u_bits['scheme'])) $u_bits['scheme'] = 'http';
   if (empty($u_bits['port'])) $u_bits['port'] = $_SERVER['SERVER_PORT'];
   $u_bits['path'] = (empty($u_bits['path']) ? '/' : $u_bits['path']) . (!empty($u_bits['query']) ?
    '?' . $u_bits['query'] : '');
   if (empty($u_bits['ua'])) $u_bits['ua'] =
     'Mozilla/5.0 (compatible; AskApache_Net/1.0; http://www.askapache.com)';
   if (empty($u_bits['referer'])) $u_bits['referer'] = 'http://www.askapache.com';
   if (empty($u_bits['fragment'])) unset($u_bits['fragment']);
   if (empty($u_bits['user'])) unset($u_bits['user']);
   if (empty($u_bits['pass'])) unset($u_bits['pass']);
   if ($u_bits['scheme'] == 'https' || $this->_socket['scheme'] == 'https') $u_bits['transport'] =
     'ssl://';
   if ($u_bits['scheme'] == 'https' || $this->_socket['scheme'] == 'https') $u_bits['port'] =
     '443';
   $socket_info = $this->_parse_args($u_bits, $socket_info);
   extract($socket_info, EXTR_SKIP);
   return true;
  }

  /**
   * AskApache_Net::_build_auth_header()
   */
  function _build_auth_header()
  {
   $this->msg(__function__ . ':' . __line__, 3);
   if ($this->authtype == 'Basic') $this->_request_headers[] = 'Authorization: Basic ' .
     base64_encode($this->_socket['user'] . ":" . $this->_socket['pass']);
   elseif ($this->authtype == 'Digest') {
    $this->msg(__function__ . ':' . __line__, 3);
    $this->_socket['protocol'] = '1.1';
    $hdr = $mtx = array();
    preg_match_all('/(\w+)=(?:"([^"]+)"|([^\s,]+))/', $this->_dh, $mtx, PREG_SET_ORDER);
    foreach ($mtx as $m) $hdr[$m[1]] = $m[2] ? $m[2] : $m[3];
    foreach ($hdr as $key => $val)
     if (array_key_exists($key, $this->_digest) && !empty($val)) $this->_digest[$key] = $val;
    $this->_digest['uri'] = $this->_socket['path'];
    $this->_digest['A1'] = md5($this->_socket['user'] . ':' . $this->_digest['realm'] .
     ':' . $this->_socket['pass']);
    $this->_digest['A2'] = md5($this->_socket['method'] . ':' . $this->_socket['path']);
    $this->_digest['response'] = md5($this->_digest['A1'] . ':' . $this->_digest['nonce'] .
     ':' . $this->_digest['nc'] . ':' . $this->_digest['cnonce'] . ':' . $this->_digest['qop'] .
     ':' . $this->_digest['A2']);
    $this->_request_headers[] = sprintf('Authorization: Digest username="%1$s", realm="%2$s", nonce="%3$s",'.
     'uri="%4$s", algorithm=%5$s, response="%6$s", qop="%7$s", nc="%8$s"%9$s%10$s',
     $this->_socket['user'], $this->_digest['realm'], $this->_digest['nonce'], $this->
     _digest['uri'], $this->_digest['algorithm'], $this->_digest['response'], $this->
     _digest['qop'], $this->_digest['nc'], !empty($this->_digest['cnonce']) ? ', cnonce="' .
     $this->_digest['cnonce'] . '"' : '', !empty($this->_digest['opaque']) ? ', opaque="' .
     $this->_digest['opaque'] . '"' : '');
   }
   return true;
  }

  /**
   * AskApache_Net::_build_request()
   */
  function _build_request()
  {
   $this->msg(__function__ . ':' . __line__, 3);
   $this->_request_headers[] = $this->_socket['method'] . " " . $this->_socket['path'] .
    " HTTP/" . $this->_socket['protocol'];
   if (is_array($this->my_headers) && sizeof($this->my_headers) > 0) $this->
     _request_headers = array_merge($this->_request_headers, $this->my_headers);
   else {
    $this->_request_headers[] = "Host: " . $this->_socket['host'];
    $this->_request_headers[] = "User-Agent: " . $this->_socket['ua'];
    $this->_request_headers[] = 'Accept: application/xhtml+xml,text/html;q=0.9,*/*;q=0.5';
    $this->_request_headers[] = 'Accept-Language: en-us,en;q=0.5';
    $this->_request_headers[] = 'Accept-Encoding: none';
    $this->_request_headers[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';
    $this->_request_headers[] = 'Referer: ' . $this->_socket['referer'];
   }
   if (!empty($this->_socket['user']) && !empty($this->_socket['pass'])) $this->
     _build_auth_header();
   if ($this->out_payload !== false) $this->_request_body = $this->out_payload;
   else  $this->_request_body = join($this->_ACLF, $this->_request_headers) . $this->
     _ACLF . $this->_ACLF;
   return true;
  }

  /**
   * AskApache_Net::_tx()
   */
  function _tx()
  {
   $this->msg(__function__ . ':' . __line__, 3);
   return (bool)(is_resource($this->_fp) && $this->_netwrite($this->_fp, $this->
    _request_body));
  }

  /**
   * AskApache_Net::_rx()
   */
  function _rx()
  {
   $this->msg(__function__ . ':' . __line__, 3);
   if (!is_resource($this->_fp)) return false;
   $this->_response = $this->_netread($this->_fp, 500000);
   $parts = explode($this->_ACLF . $this->_ACLF, ltrim($this->_response), 2);
   $this->_response_header = trim($parts[0]);
   $this->_response_body = trim($parts[1]);
   if (preg_match('#([^/]*)/([\d\.]+) ([\d]*?) (.*)#', $this->_response_header, $htx)) {
    $this->_response_protocol = trim($htx[1]);
    $this->_response_version = trim($htx[2]);
    $this->_response_code = trim($htx[3]);
    $this->_response_message = trim($htx[4]);
   }
   if (preg_match_all('#([^:]+)\:?(.*)#', str_replace($htx, '', $this->_response_header),
    $mtx, PREG_SET_ORDER)) {
    foreach ($mtx as $m) {
     $this->_headers[strtolower(trim($m[1]))] = trim($m[2]);
     if (preg_match('/(WWW|Proxy)-Authenticate:.*Digest/i', trim($m[1]))) $this->_dh =
       trim($m[1]);
    }
   }
   return true;
  }

  /**
   * AskApache_Net::tcp_trace()
   */
  function tcp_trace($p = false)
  {
   $this->_timer(__function__ );
   $ret = join("\n", array_merge((array )$this->_request_headers, array(''), (array )$this->
    _response_headers));
   if ($p !== false) {
    echo $ret;
    $ret = true;
   }
   $this->_timer(__function__ );
   return $ret;
  }

  /**
   * AskApache_Net::_get_ip()
   */
  function _get_ip($host)
  {
   $this->msg(__function__ . ':' . __line__, 3);
 
   if (!preg_match('/^[\t ]*[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+[\t ]*$/', $host)) $hostip =
     gethostbyname($host);
   $ip = ($hostip == $host) ? $host : long2ip(ip2long($hostip));
   return $ip;
  }

  /**
   * AskApache_Net::_connect()
   */
  function _connect()
  {
   $this->msg(__function__ . ':' . __line__, 3);
   if (false === ($this->_fp = fsockopen($this->_get_ip($this->_socket['host']), $this->
    _socket['port'], $errno, $errstr, $this->timeout)) || !is_resource($this->_fp)) {
    $err = (array_key_exists($errno, $this->_errs)) ? $this->_errs[$errno] :
     'Connection failed';
    return $this->msg(__function__ . ':' . __line__ . " Fsockopen failed! [{$errno}] {$err} ({$errstr})",
     0);
   }
   if (function_exists("socket_set_timeout")) socket_set_timeout($this->_fp, $this->
     timeout);
   elseif (function_exists("stream_set_timeout")) stream_set_timeout($this->_fp, $this->
     timeout);
   usleep(10000);
   return true;
  }

  /**
   * AskApache_Net::_disconnect()
   */
  function _disconnect()
  {
   $this->msg(__function__ . ':' . __line__, 3);
   if (is_resource($this->_fp)) return $this->_fclose($this->_fp);
   else  $this->_fp = null;
   return true;
  }

  /**
   * AskApache_Net::get_response_headers()
   */
  function get_response_headers($header = false)
  {
   $this->msg(__function__ . ':' . __line__, 3);
   if ($header !== false && array_key_exists($header, $this->_response_headers)) return $this->
     _response_headers[$header];
   return $this->_response_headers;
  }

  /**
   * AskApache_Net::get_response_body()
   */
  function get_response_body()
  {
   $this->msg(__function__ . ':' . __line__, 3);
   return $this->_response_body;
  }

  /**
   * AskApache_Net::_netread()
   */
  function _netread(&$fh, $ts = 50000000, $bs = 124)
  {
   $this->_timer(__function__ );
   for ($d = $b = '', $rt = $at = $r = 0; ($fh !== false && !feof($fh) && $b !== false &&
    $at < 50000000 && $rt < $ts); $r = $ts - $rt, $bs = (($bs > $r) ? $r : $bs), $this->
    _timer("R: {$rt}"), $b = fread($fh, $bs), $br = strlen($b), $d .= $b, $this->_timer("R: {$rt}"),
    $rt += $br, $at++, $this->msg("[RT: {$rt}]\t[BR: {$br}" . (($ts != 50000000) ? "]\t\t [{$r} / {$ts}]" :
    " : {$bs}]\t[{$at}]"))) ;
   $this->_timer(__function__ );
   return ((strlen($d) != 0)) ? $d : false;
  }

  /**
   * AskApache_Net::_netwrite()
   */
  function _netwrite(&$fh, $d = '', $bs = 512)
  {
   $this->_timer(__function__ );
 
   for ($bw = $wt = $at = 0, $dat = '', $ts = strlen($d); ($fh !== false && $bw !== false &&
    $at < 50000000 && $wt < $ts); $r = $ts - $wt, $bs = (($bs > $r) ? $r : $bs), $dat =
    substr($d, $wt, $bs), $bw = fwrite($fh, $dat), $wt += $bw, $this->msg("[WT: {$wt}]\t[BW: {$bw}]\t\t[I: {$r} / {$ts}:{$bs}] - {$at}"),
    $at++) ;
   $this->msg("[WT: {$wt}]\t[BW: {$bw}]\t\t[I: {$r} / {$ts}:{$bs}] - {$at}");
   $this->_timer(__function__ );
   return ($wt == $ts) ? true : false;
  }
 }
endif;
?>

So I decided to finally give in to what I’ve been avoiding all along and added a php-software-based method that will work on everycomputer, windows, blackberrys, etc.. That took me about 15minutes as its just a few lines of code.. The problem I have with it is that php is what is actually controlling the sending, receiving, and verifying of the authentication headers instead of using the builtin super-secure apache method.

Here’s how you would block someone using the apache/askapache way:

[Exploit Request] => ([BLOCKED]-AskApache)

This prevents the exploit from even reaching PHP, saving your computer a lot of CPU/memory and bandwdith, and obviously can’t exploit wordpress if php isn’t even loading.

Here’s how the php-software-based method blocks the same request:

[Exploit Request] => (AskApache) => (PHP) => (WordPress) => ([BLOCKED]-askapache-password-protect.php)

So the last bit of programming and research I’m doing at the moment is how to cause the askapache-password-protect plugin to execute as soon as possible, ideally it would execute before WordPress starts.. And I am still crazy swamped at work, this was the longest non-posting period of the blog to date!

Password Protection Plugin Status originally appeared on AskApache.

Ultimate Htaccess Tutorial for .htaccess files

AskApache — Sat, 10 Jan 2009 13:05:32 +0000

This is not an introduction to .htaccess… This is the evolution of .htaccess… The best of the best .htaccess info I can find.

Originally known as the Ultimate .htaccess Guide, its changed over the years with more and better .htaccess tricks using real .htaccess example code. I add my favorite htaccess-related links and sites, results from my best .htaccess experiments, and frequently update this article.

You’ve come to the right place if you are looking to acquire mad skills for using .htaccess files.

Evolution of Htaccess
Server Config (httpd.conf) Examples
Example .htaccess Files
Example .htaccess Code Snippets
NEW: Advanced Mod_Rewrite Tricks
Best AskApache .htaccess Articles
Favorite .htaccess Links
Master List of .htaccess Directives
Htaccess Modules List

Htaccess – Evolved

The Hyper Text Transfer Protocol (HTTP) was initiated at the CERN in Geneve (Switzerland), where it emerged (together with the HTML presentation language) from the need to exchange scientific information on a computer network in a simple manner. The first public HTTP implementation only allowed for plain text information, and almost instantaneously became a replacement of the GOPHER service. One of the first text-based browsers was LYNX which still exists today; a graphical HTTP client appeared very quickly with the name NCSA Mosaic. Mosaic was a popular browser back in 1994. Soon the need for a more rich multimedia experience was born, and the markup language provided support for a growing multitude of media types.

The following .htaccess tricks, .htaccess tips, examples, samples, tutorials, tools, and plethora of “How To” htaccess articles on creating and using htaccess files will do several things for you, just like its done for me.

Make your website noticeably faster.
Allow you to debug your server with ease.
Make your life easier and more rewarding.
Allow you to work faster and more productively.

What Is .htaccess

.htaccess is a very ancient configuration file that controls the Web Server running your website, and is one of the most powerful configuration files you will ever come across. Htaccess is sometimes called: “HyperText Access” because of its ability to control access of the WWW’s HyperText Transfer Protocol (HTTP) using Password Protection, Allow and Deny capabilities, and more.

Specifically, .htaccess is the default file name of a special configuration file that provides a number of directives (commands) for controlling and configuring the Apache Web Server, and also to control and configure modules that can be built into the Apache installation, or included at run-time like mod_rewrite (for htaccess rewrite), mod_alias (for htaccess redirects), and mod_ssl (for controlling SSL connections).

Htaccess allows for decentralized management of Web Server configurations which makes life very easy for web hosting companies and especially their savvy consumers. They set up and run “server farms” where many hundreds and thousands of web hosting customers are all put on the same Apache Server. This type of hosting is called “virtual hosting” and without .htaccess files would mean that every customer must use the same exact settings as everyone else on their segment. So that is why any half-decent web host allows/enables (DreamHost, Powweb, MediaTemple, GoDaddy) .htaccess files, though few people are aware of it. Let’s just say that if I was a customer on your server-farm, and .htaccess files were enabled, my websites would be a LOT faster than yours, as these configuration files allow you to fully take advantage of and utilize the resources allotted to you by your host. If even 1/10 of the sites on a server-farm took advantage of what they are paying for, the providers would go out of business.

Configuration, commands and the like

One of the design goals for this server was to maintain external compatibility with the NCSA 1.3 server — that is, to read the same configuration files, to process all the directives therein correctly, and in general to be a drop-in replacement for NCSA. On the other hand, another design goal was to move as much of the server’s functionality into modules which have as little as possible to do with the monolithic server core. The only way to reconcile these goals is to move the handling of most commands from the central server into the modules.

However, just giving the modules command tables is not enough to divorce them completely from the server core. The server has to remember the commands in order to act on them later. That involves maintaining data which is private to the modules, and which can be either per-server, or per-directory. Most things are per-directory, including in particular access control and authorization information, but also information on how to determine file types from suffixes, which can be modified by AddType and DefaultType directives, and so forth. In general, the governing philosophy is that anything which can be made configurable by directory should be; per-server information is generally used in the standard set of modules for information like Aliases and Redirects which come into play before the request is tied to a particular place in the underlying file system.

Another requirement for emulating the NCSA server is being able to handle the per-directory configuration files, generally called .htaccess files, though even in the NCSA server they can contain directives which have nothing at all to do with access control. Accordingly, after URI -> filename translation, but before performing any other phase, the server walks down the directory hierarchy of the underlying filesystem, following the translated pathname, to read any .htaccess files which might be present. The information which is read in then has to be merged with the applicable information from the server’s own config files (either from the sections in access.conf, or from defaults in srm.conf, which actually behaves for most purposes almost exactly like ).

Finally, after having served a request which involved reading .htaccess files, we need to discard the storage allocated for handling them. That is solved the same way it is solved wherever else similar problems come up, by tying those structures to the per-transaction resource pool.

Creating Htaccess Files

Htaccess files use the default filename “.htaccess” but any unix-style file name can be specified from the main server config using the AccessFileName directive. The file isn’t .htaccess.txt, its literally just named .htaccess.

In a Windows Environment like the one I use for work, you can change how Windows opens and views .htaccess files by modifying the Folder Options in explorer. As you can see, on my computer files ending in .htaccess are recognized as having the HTACCESS extension and are handled/opened by Adobe Dreamweaver CS4.

Htaccess Scope

Unlike the main server configuration files like httpd.conf, Htaccess files are read on every request therefore changes in these files take immediate effect. Apache searches all directories and subdirectories that are htaccess-enabled for an .htaccess file which results in performance loss due to file accesses. I’ve never noticed a performance loss but OTOH, I know how to use them. If you do have access to your main server configuration file, you should of course use that instead, and lucky for you ALL the .htaccess tricks and examples can be used there as well (just not vice versa).

Htaccess File Syntax

Htaccess files follow the same syntax as the main Apache configuration files, for powerusers here’s an apache.vim for VI. The one main difference is the context of the directive, which means whether or not that directive is ALLOWED to be used inside of an .htaccess file. Htaccess files are incredibly powerful, and can also be very dangerous as some directives allowed in the main configuration files would allow users/customers to completely bypass security/bandwidth-limits/resource-limits/file-permissions, etc.. About 1/4 of all Apache directives cannot be used inside an .htaccess file (also known as a per-directory context config). The Apache Developers are well-regarded throughout the world as being among some of the best programmers, ever. To enable a disallowed directive inside a .htaccess file would require modifying the source code and re-compiling the server (which they allow and encourage if you are the owner/admin). Here’s a taste of that famous Apache source code that builds the directives allowed in .htaccess file context, the key that tells whether its enabled in .htaccess context is the DIR_CMD_PERMS and then the OR_FILEINFO, which means a directive is enabled dependent on the AllowOverride directive that is only allowed in the main config. First Apache 1.3.0, then Apache 2.2.10

//mod_autoindex
AddIcon, add_icon, BY_PATH, DIR_CMD_PERMS, an icon URL followed by one or more filenames
AddIconByType, add_icon, BY_TYPE, DIR_CMD_PERMS, an icon URL followed by one or more MIME types
AddIconByEncoding, add_icon, BY_ENCODING, DIR_CMD_PERMS, an icon URL followed by one or more content encodings
AddAlt, add_alt, BY_PATH, DIR_CMD_PERMS, alternate descriptive text followed by one or more filenames
AddAltByType, add_alt, BY_TYPE, DIR_CMD_PERMS, alternate descriptive text followed by one or more MIME types
AddAltByEncoding, add_alt, BY_ENCODING, DIR_CMD_PERMS, alternate descriptive text followed by one or more content encodings
IndexOptions, add_opts, DIR_CMD_PERMS, RAW_ARGS, one or more index options
IndexIgnore, add_ignore, DIR_CMD_PERMS, ITERATE, one or more file extensions
AddDescription, add_desc, BY_PATH, DIR_CMD_PERMS, Descriptive text followed by one or more filenames
HeaderName, add_header, DIR_CMD_PERMS, TAKE1, a filename
ReadmeName, add_readme, DIR_CMD_PERMS, TAKE1, a filename
FancyIndexing, fancy_indexing, DIR_CMD_PERMS, FLAG, Limited to 'on' or 'off' (superseded by IndexOptions FancyIndexing)
DefaultIcon, ap_set_string_slot, (void *) XtOffsetOf(autoindex_config_rec, default_icon), DIR_CMD_PERMS, TAKE1, an icon URL
 
// mod_rewrite
RewriteEngine, cmd_rewriteengine, OR_FILEINFO, On or Off to enable or disable (default)
RewriteOptions, cmd_rewriteoptions, OR_FILEINFO, List of option strings to set
RewriteBase, cmd_rewritebase, OR_FILEINFO, the base URL of the per-directory context
RewriteCond, cmd_rewritecond, OR_FILEINFO, an input string and a to be applied regexp-pattern
RewriteRule, cmd_rewriterule, OR_FILEINFO, an URL-applied regexp-pattern and a substitution URL
RewriteMap, cmd_rewritemap, RSRC_CONF, a mapname and a filename
RewriteLock, cmd_rewritelock, RSRC_CONF, the filename of a lockfile used for inter-process synchronization
RewriteLog, cmd_rewritelog, RSRC_CONF, the filename of the rewriting logfile
RewriteLogLevel, cmd_rewriteloglevel, RSRC_CONF, the level of the rewriting logfile verbosity (0=none, 1=std, .., 9=max)
RewriteLog, fake_rewritelog, RSRC_CONF, [DISABLED] the filename of the rewriting logfile
RewriteLogLevel, fake_rewritelog, RSRC_CONF, [DISABLED] the level of the rewriting logfile verbosity

Htaccess Directives

Don’t ask why, but I personally downloaded each major/beta release of the Apache HTTPD source code from version 1.3.0 to version 2.2.10 (all 63 Apache versions!), then I configured and compiled each version for a custom HTTPD installation built from source. This allowed me to find every directive allowed in .htaccess files for each particular version, which has never been done before, or since. YES! I think that is so cool..

An .htaccess directive is basically a command that is specific to a module or builtin to the core that performs a specific task or sets a specific setting for how Apache serves your WebSite. Directives placed in Htaccess files apply to the directory they are in, and all sub-directories. Here’s the 3 top links (official Apache Docs) you will repeatedly use, bookmark/print/save them.

Litespeed .htaccess support

Unlike other lightweight web servers, Apache compatible per-directory configuration overridden is fully supported by LiteSpeed Web Server. With .htacess you can change configurations for any directory under document root on-the-fly, which in most cases is a mandatory feature in shared hosting environment. It is worth noting that enabling .htaccess support in LiteSpeed Web Server will not degrade server’s performance, comparing to Apache’s 40% drop in performance.

Main Server Config Examples

Now lets take a look at some htaccess examples to get a feel for the syntax and some general ideas at the capabilities. Some of the best examples for .htaccess files are included with Apache for main server config files, so lets take a quick look at a couple of them on our way down to the actual .htaccess examples further down the page (this site has thousands, take your time). As you can see, the basic syntax is a line starting with # is a comment, everything else are directives followed by the directive argument.

httpd-multilang-errordoc.conf: The configuration below implements multi-language error documents through content-negotiation

Options IncludesNoExec
AddOutputFilter Includes html
AddHandler type-map var
LanguagePriority en cs de es fr it ja ko nl pl pt-br ro sv tr
ForceLanguagePriority Prefer Fallback
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
ErrorDocument 403 /error/HTTP_FORBIDDEN.html.var
ErrorDocument 404 /error/HTTP_NOT_FOUND.html.var

httpd-manual.conf: Provide local access to the server documentation on your server

SetEnvIf Request_URI ^/manual/(de|en|es|fr|ja|ko|pt-br|ru|tr)/ prefer-language=$1
RedirectMatch 301 ^/manual(?:/(de|en|es|fr|ja|ko|pt-br|ru|tr)){2, }(/.*)?$ /manual/$1$2
LanguagePriority en de es fr ja ko pt-br ru tr
ForceLanguagePriority Prefer Fallback

httpd-languages.conf: Settings for hosting different languages.

DefaultLanguage en
AddLanguage ca .ca
# Just list the languages in decreasing order of preference.
LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv tr zh-CN zh-TW
# Commonly used filename extensions to character sets.
AddCharset us-ascii.ascii .us-ascii
AddCharset ISO-8859-1  .iso8859-1  .latin1

httpd-autoindex.conf: Directives controlling the display of server-generated directory listings.

# IndexOptions: Controls the appearance of server-generated directory listings.
IndexOptions FancyIndexing HTMLTable VersionSort
# AddIcon* directives tell the server which icon to show for different files or filename extensions.
AddIconByEncoding (CMP, /icons/compressed.gif) x-compress x-gzip
AddIconByType (TXT, /icons/text.gif) text/*
AddIcon /icons/folder.gif ^^DIRECTORY^^
# DefaultIcon is which icon to show for files which do not have an icon explicitly set.
DefaultIcon /icons/unknown.gif
# AddDescription allows you to place a short description after a file in server-generated indexes.
AddDescription "GZIP compressed document" .gz
# ReadmeName is the name of the README file the server will look for by default, and append to directory listings.
ReadmeName README.html
# HeaderName is the name of a file which should be prepended to directory indexes.
HeaderName HEADER.html

Here are the rest of them if you wanna take a look. (httpd-mpm.conf, httpd-default.conf, httpd-ssl.conf, httpd-info.conf, httpd-vhosts.conf, httpd-dav.conf)

Example .htaccess Files

Here are some samples and examples taken from different .htaccess files I’ve used over the years. Specific solutions are farther down on this page and throughout the site.

# Set the Time Zone of your Server
SetEnv TZ America/Indianapolis
# ServerAdmin:  This address appears on some server-generated pages, such as error documents.
SetEnv SERVER_ADMIN webmaster@askapache.com
# Possible values for the Options directive are "None", "All", or any combination of:
#  Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
Options -ExecCGI -MultiViews -Includes -Indexes FollowSymLinks
# DirectoryIndex: sets the file that Apache will serve if a directory is requested.
DirectoryIndex index.html index.php /index.php
#
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#
Action php5-cgi /bin/php.cgi
#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
AddHandler php-cgi .php .inc
# Commonly used filename extensions to character sets.
AddDefaultCharset UTF-8
# AddType allows you to add to or override the MIME configuration
AddType 'application/rdf+xml; charset=UTF-8' .rdf
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml
AddType 'application/xhtml+xml; charset=UTF-8' .xhtml.gz
AddType 'text/html; charset=UTF-8' .html
AddType 'text/html; charset=UTF-8' .html.gz
AddType application/octet-stream .rar .chm .bz2 .tgz .msi .pdf .exe
AddType application/vnd.ms-excel .csv
AddType application/x-httpd-php-source .phps
AddType application/x-pilot .prc .pdb
AddType application/x-shockwave-flash .swf
AddType application/xrds+xml .xrdf
AddType text/plain .ini .sh .bsh .bash .awk .nawk .gawk .csh .var .c .in .h .asc .md5 .sha .sha1
AddType video/x-flv .flv
# AddEncoding allows you to have certain browsers uncompress information on the fly. Note: Not all browsers support this.
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
# DefaultType: the default MIME type the server will use for a document.
DefaultType text/html
#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature Off

## MAIN DEFAULTS
Options +ExecCGI -Indexes
DirectoryIndex index.html index.htm index.php
DefaultLanguage en-US
AddDefaultCharset UTF-8
ServerSignature Off
## ENVIRONMENT VARIABLES
SetEnv PHPRC /webroot/includes
SetEnv TZ America/Indianapolis
SetEnv SERVER_ADMIN webmaster@askapache.com
## MIME TYPES
AddType video/x-flv .flv
AddType application/x-shockwave-flash .swf
AddType image/x-icon .ico
## FORCE FILE TO DOWNLOAD INSTEAD OF APPEAR IN BROWSER
# http://www.htaccesselite.com/addtype-addhandler-action-vf6.html
AddType application/octet-stream .mov .mp3 .zip
## ERRORDOCUMENTS
# http://askapache.com/htaccess/apache-status-code-headers-errordocument.html
ErrorDocument 400 /e400/
ErrorDocument 401 /e401/
ErrorDocument 402 /e402/
ErrorDocument 403 /e403/
ErrorDocument 404 /e404/
#
# Handlers be builtin, included in a module, or added with Action directive
# default-handler: default, handles static content (core)
#   send-as-is: Send file with HTTP headers (mod_asis)
#   cgi-script: treat file as CGI script (mod_cgi)
#    imap-file: Parse as an imagemap rule file (mod_imap)
#   server-info: Get server config info (mod_info)
#  server-status: Get server status report (mod_status)
#    type-map: type map file for content negotiation (mod_negotiation)
#  fastcgi-script: treat file as fastcgi script (mod_fastcgi)
#
# http://www.askapache.com/php/custom-phpini-tips-and-tricks.html
## PARSE AS CGI
AddHandler cgi-script .cgi .pl .spl
## RUN PHP AS APACHE MODULE
AddHandler application/x-httpd-php .php .htm
## RUN PHP AS CGI
AddHandler php-cgi .php .htm
## CGI PHP WRAPPER FOR CUSTOM PHP.INI
AddHandler phpini-cgi .php .htm
Action phpini-cgi /cgi-bin/php5-custom-ini.cgi
## FAST-CGI SETUP WITH PHP-CGI WRAPPER FOR CUSTOM PHP.INI
AddHandler fastcgi-script .fcgi
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5-wrapper.fcgi
## CUSTOM PHP CGI BINARY SETUP
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php.cgi
## PROCESS SPECIFIC FILETYPES WITH CGI-SCRIPT
Action image/gif /cgi-bin/img-create.cgi
## CREATE CUSTOM HANDLER FOR SPECIFIC FILE EXTENSIONS
AddHandler custom-processor .ssp
Action custom-processor /cgi-bin/myprocessor.cgi
### HEADER CACHING
# http://www.askapache.com/htaccess/speed-up-sites-with-htaccess-caching.html

Header set Cache-Control "max-age=2592000"


Header set Cache-Control "max-age=604800"


Header set Cache-Control "max-age=600"


Header unset Cache-Control

## ALTERNATE EXPIRES CACHING
# htaccesselite.com/d/use-htaccess-to-speed-up-your-site-discussion-vt67.html
ExpiresActive On
ExpiresDefault A604800
ExpiresByType image/x-icon A2592000
ExpiresByType application/x-javascript A2592000
ExpiresByType text/css A2592000
ExpiresByType text/html A300

ExpiresActive Off

## META HTTP-EQUIV REPLACEMENTS

Header set imagetoolbar "no"

Here are some default MOD_REWRITE code examples.

## REWRITE DEFAULTS
RewriteEngine On
RewriteBase /
## REQUIRE SUBDOMAIN
RewriteCond %{HTTP_HOST} !^$
RewriteCond %{HTTP_HOST} !^subdomain\.askapache\.com$ [NC]
RewriteRule ^/(.*)$ http://subdomain.askapache.com/$1 [L,R=301]
## SEO REWRITES
RewriteRule ^(.*)/ve/(.*)$ $1/voluntary-employee/$2 [L,R=301]
RewriteRule ^(.*)/hsa/(.*)$ $1/health-saving-account/$2 [L,R=301]
## WORDPRESS
RewriteCond %{REQUEST_FILENAME} !-f  # Existing File
RewriteCond %{REQUEST_FILENAME} !-d  # Existing Directory
RewriteRule . /index.php [L]
## ALTERNATIVE ANTI-HOTLINKING
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(subdomain\.)?askapache.com/.*$ [NC]
RewriteRule ^.*\.(bmp|tif|gif|jpg|jpeg|jpe|png)$ - [F]
## REDIRECT HOTLINKERS
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(subdomain\.)?askapache.com/.*$ [NC]
RewriteRule ^.*\.(bmp|tif|gif|jpg|jpeg|jpe|png)$ http://google.com [R]
## DENY REQUEST BASED ON REQUEST METHOD
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS|HEAD)$ [NC]
RewriteRule ^.*$ - [F]
## REDIRECT UPLOADS
RewriteCond %{REQUEST_METHOD} ^(PUT|POST)$ [NC]
RewriteRule ^(.*)$ /cgi-bin/form-upload-processor.cgi?p=$1 [L,QSA]
## REQUIRE SSL EVEN WHEN MOD_SSL IS NOT LOADED
RewriteCond %{HTTPS} !=on [NC]
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [R,L]
### ALTERNATATIVE TO USING ERRORDOCUMENT
# http://www.htaccesselite.com/d/htaccess-errordocument-examples-vt11.html
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^.*$ /error.php [L]
## SEO REDIRECTS
Redirect 301 /2006/oldfile.html http://subdomain.askapache.com/newfile.html
RedirectMatch 301 /o/(.*)$ http://subdomain.askapache.com/s/dl/$1

Examples of protecting your files and securing with password protection.

#
# Require (user|group|valid-user) (username|groupname)
#
## BASIC PASSWORD PROTECTION
AuthType basic
AuthName "prompt"
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user
## ALLOW FROM IP OR VALID PASSWORD
Require valid-user
Allow from 192.168.1.23
Satisfy Any
## PROTECT FILES

Order Allow,Deny
Deny from all

## PREVENT HOTLINKING
SetEnvIfNoCase Referer "^http://subdomain.askapache.com/" good
SetEnvIfNoCase Referer "^$" good

Order Deny,Allow
Deny from all
Allow from env=good
ErrorDocument 403 http://www.google.com/intl/en_ALL/images/logo.gif
ErrorDocument 403 /images/you_bad_hotlinker.gif

## LIMIT UPLOAD FILE SIZE TO PROTECT AGAINST DOS ATTACK
#bytes, 0-2147483647(2GB)
LimitRequestBody 10240000
## MOST SECURE WAY TO REQUIRE SSL
# http://www.askapache.com/htaccess/apache-ssl-in-htaccess-examples.html
SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "askapache.com"
ErrorDocument 403 https://askapache.com
## COMBINED DEVELOPER HTACCESS CODE-USE THIS

Header set Cache-Control "max-age=5"

AuthType basic
AuthName "Ooops! Temporarily Under Construction..."
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user      # password prompt for everyone else
Order Deny,Allow
Deny from all
Allow from 192.168.64.5   # Your, the developers IP address
Allow from w3.org      # css/xhtml check jigsaw.w3.org/css-validator/
Allow from googlebot.com   # Allows google to crawl your pages
Satisfy Any        # no password required if host/ip is Allowed
## DONT HAVE TO EMPTY CACHE OR RELOAD TO SEE CHANGES
ExpiresDefault A5 #If using mod_expires

Header set Cache-Control "max-age=5"

## ALLOW ACCESS WITH PASSWORD OR NO PASSWORD FOR SPECIFIC IP/HOSTS
AuthType basic
AuthName "Ooops! Temporarily Under Construction..."
AuthUserFile /.htpasswd
AuthGroupFile /dev/null
Require valid-user      # password prompt for everyone else
Order Deny,Allow
Deny from all
Allow from 192.168.64.5   # Your, the developers IP address
Allow from w3.org      # css/xhtml check jigsaw.w3.org/css-validator/
Allow from googlebot.com   # Allows google to crawl your pages
Satisfy Any        # no password required if host/ip is Allowed

Example .htaccess Code Snippets

Here are some specific examples, this is the most popular section of this page. Updated frequently.

Redirect Everyone Except IP address to alternate page

ErrorDocument 403 http://www.yahoo.com/
Order deny,allow
Deny from all
Allow from 208.113.134.190

When developing sites

This lets google crawl the page, lets me access without a password, and lets my client access the page WITH a password. It also allows for XHTML and CSS validation! (w3.org)

AuthName "Under Development"
AuthUserFile /home/sitename.com/.htpasswd
AuthType basic
Require valid-user
Order deny,allow
Deny from all
Allow from 208.113.134.190 w3.org htmlhelp.com googlebot.com
Satisfy Any

Fix double-login prompt

Redirect non-https requests to https server and ensure that .htpasswd authorization can only be entered across HTTPS

SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "askapache.com"
ErrorDocument 403 https://askapache.com

Set Timezone of the Server (GMT)

SetEnv TZ America/Indianapolis

Administrator Email for ErrorDocument

SetEnv SERVER_ADMIN webmaster@google.com

`ServerSignature` for `ErrorDocument`

ServerSignature off | on | email

Charset and Language headers

Article: Setting Charset in htaccess, and article by Richard Ishida

AddDefaultCharset UTF-8
DefaultLanguage en-US

Disallow Script Execution

Options -ExecCGI
AddHandler cgi-script .php .pl .py .jsp .asp .htm .shtml .sh .cgi

Deny Request Methods

RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|OPTIONS|POST|PUT)
RewriteRule .* - [F]

Force “File Save As” Prompt

AddType application/octet-stream .avi .mpg .mov .pdf .xls .mp4

Show CGI Source Code

RemoveHandler cgi-script .pl .py .cgi
AddType text/plain .pl .py .cgi

Serve all .pdf files on your site using .htaccess and mod_rewrite with the php script.

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+)\.pdf$  /cgi-bin/pdf.php?file=$1 [L,NC,QSA]

Rewrite to www

RewriteCond %{REQUEST_URI} !^/(robots\.txt|favicon\.ico|sitemap\.xml)$
RewriteCond %{HTTP_HOST} !^www\.askapache\.com$ [NC]
RewriteRule ^(.*)$ http://www.askapache.com/$1 [R=301,L]

Rewrite to www dynamically

RewriteCond %{REQUEST_URI} !^/robots\.txt$ [NC]
RewriteCond %{HTTP_HOST} !^www\.[a-z-]+\.[a-z]{2,6} [NC]
RewriteCond %{HTTP_HOST} ([a-z-]+\.[a-z]{2,6})$   [NC]
RewriteRule ^/(.*)$ http://%1/$1 [R=301,L]

301 Redirect Old File

Redirect 301 /old/file.html http://www.askapache.com/new/file.html

301 Redirect Entire Directory

RedirectMatch 301 /blog(.*) http://www.askapache.com/$1

Protecting your php.cgi


Order Deny,Allow
Deny from All
Allow from env=REDIRECT_STATUS

Set Cookie based on Request

This code sends the Set-Cookie header to create a cookie on the client with the value of a matching item in 2nd parantheses.

RewriteEngine On
RewriteBase /
RewriteRule ^(.*)(de|es|fr|it|ja|ru|en)/$ - [co=lang:$2:.askapache.com:7200:/]

Set Cookie with env variable

Header set Set-Cookie "language=%{lang}e; path=/;" env=lang

Custom ErrorDocuments

ErrorDocument 100 /100_CONTINUE
ErrorDocument 101 /101_SWITCHING_PROTOCOLS
ErrorDocument 102 /102_PROCESSING
ErrorDocument 200 /200_OK
ErrorDocument 201 /201_CREATED
ErrorDocument 202 /202_ACCEPTED
ErrorDocument 203 /203_NON_AUTHORITATIVE
ErrorDocument 204 /204_NO_CONTENT
ErrorDocument 205 /205_RESET_CONTENT
ErrorDocument 206 /206_PARTIAL_CONTENT
ErrorDocument 207 /207_MULTI_STATUS
ErrorDocument 300 /300_MULTIPLE_CHOICES
ErrorDocument 301 /301_MOVED_PERMANENTLY
ErrorDocument 302 /302_MOVED_TEMPORARILY
ErrorDocument 303 /303_SEE_OTHER
ErrorDocument 304 /304_NOT_MODIFIED
ErrorDocument 305 /305_USE_PROXY
ErrorDocument 307 /307_TEMPORARY_REDIRECT
ErrorDocument 400 /400_BAD_REQUEST
ErrorDocument 401 /401_UNAUTHORIZED
ErrorDocument 402 /402_PAYMENT_REQUIRED
ErrorDocument 403 /403_FORBIDDEN
ErrorDocument 404 /404_NOT_FOUND
ErrorDocument 405 /405_METHOD_NOT_ALLOWED
ErrorDocument 406 /406_NOT_ACCEPTABLE
ErrorDocument 407 /407_PROXY_AUTHENTICATION_REQUIRED
ErrorDocument 408 /408_REQUEST_TIME_OUT
ErrorDocument 409 /409_CONFLICT
ErrorDocument 410 /410_GONE
ErrorDocument 411 /411_LENGTH_REQUIRED
ErrorDocument 412 /412_PRECONDITION_FAILED
ErrorDocument 413 /413_REQUEST_ENTITY_TOO_LARGE
ErrorDocument 414 /414_REQUEST_URI_TOO_LARGE
ErrorDocument 415 /415_UNSUPPORTED_MEDIA_TYPE
ErrorDocument 416 /416_RANGE_NOT_SATISFIABLE
ErrorDocument 417 /417_EXPECTATION_FAILED
ErrorDocument 422 /422_UNPROCESSABLE_ENTITY
ErrorDocument 423 /423_LOCKED
ErrorDocument 424 /424_FAILED_DEPENDENCY
ErrorDocument 426 /426_UPGRADE_REQUIRED
ErrorDocument 500 /500_INTERNAL_SERVER_ERROR
ErrorDocument 501 /501_NOT_IMPLEMENTED
ErrorDocument 502 /502_BAD_GATEWAY
ErrorDocument 503 /503_SERVICE_UNAVAILABLE
ErrorDocument 504 /504_GATEWAY_TIME_OUT
ErrorDocument 505 /505_VERSION_NOT_SUPPORTED
ErrorDocument 506 /506_VARIANT_ALSO_VARIES
ErrorDocument 507 /507_INSUFFICIENT_STORAGE
ErrorDocument 510 /510_NOT_EXTENDED

Implementing a Caching Scheme with .htaccess

# year

Header set Cache-Control "public"
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"
Header unset Last-Modified

#2 hours

Header set Cache-Control "max-age=7200, must-revalidate"


SetOutputFilter DEFLATE
Header set Expires "Thu, 15 Apr 2010 20:00:00 GMT"

Password Protect single file


AuthName "Prompt"
AuthType Basic
AuthUserFile /home/askapache.com/.htpasswd
Require valid-user

Password Protect multiple files


AuthName "Development"
AuthUserFile /.htpasswd
AuthType basic
Require valid-user

Send Custom Headers

Header set P3P "policyref=\"http://www.askapache.com/w3c/p3p.xml\""
Header set X-Pingback "http://www.askapache.com/xmlrpc.php"
Header set Content-Language "en-US"
Header set Vary "Accept-Encoding"

Blocking based on User-Agent Header

SetEnvIfNoCase ^User-Agent$ .*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures) HTTP_SAFE_BADBOT
SetEnvIfNoCase ^User-Agent$ .*(libwww-perl|aesop_com_spiderman) HTTP_SAFE_BADBOT
Deny from env=HTTP_SAFE_BADBOT

Blocking with RewriteCond

RewriteCond %{HTTP_USER_AGENT} ^.*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]
RewriteRule . - [F,L]

.htaccess for mod_php

SetEnv PHPRC /location/todir/containing/phpinifile

.htaccess for php as cgi

AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5.cgi

Shell wrapper for custom php.ini

#!/bin/sh
export PHP_FCGI_CHILDREN=3
exec php5.cgi -c /abs/php5/php.ini

Add values from HTTP Headers

SetEnvIfNoCase ^If-Modified-Since$ "(.+)" HTTP_IF_MODIFIED_SINCE=$1
SetEnvIfNoCase ^If-None-Match$ "(.+)" HTTP_IF_NONE_MATCH=$1
SetEnvIfNoCase ^Cache-Control$ "(.+)" HTTP_CACHE_CONTROL=$1
SetEnvIfNoCase ^Connection$ "(.+)" HTTP_CONNECTION=$1
SetEnvIfNoCase ^Keep-Alive$ "(.+)" HTTP_KEEP_ALIVE=$1
SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
SetEnvIfNoCase ^Cookie$ "(.+)" HTTP_MY_COOKIE=$1

Stop hotlinking

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?askapache.com/.*$ [NC]
RewriteRule \.(gif|jpg|swf|flv|png)$ http://www.askapache.com/feed.gif [R=302,L]

Turn logging off for IP

SecFilterSelective REMOTE_ADDR "208\.113\.183\.103" "nolog,noauditlog,pass"

Turn logging on for IP

SecFilterSelective REMOTE_ADDR "!^208\.113\.183\.103" "nolog,noauditlog,pass"
SecFilterSelective REMOTE_ADDR "208\.113\.183\.103" "log,auditlog,pass"

Advanced Mod_Rewrite Examples

Here are some specific examples taken mostly from my WordPress Password Protection plugin, that also does alot more than password protection as you can see from the following mod_rewrite examples. These are a few of the mod_rewrite uses that BlogSecurity declared pushed the boundaries of Mod_Rewrite! Some of these snippets are quite exotic and unlike anything you may have seen before, also only for those who understand them as they can kill a website pretty quick.

Directory Protection

Enable the DirectoryIndex Protection, preventing directory index listings and defaulting. [Disable]

Options -Indexes
DirectoryIndex index.html index.php /index.php

Password Protect wp-login.php

Requires a valid user/pass to access the login page[401]


Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user

Password Protect wp-admin

Requires a valid user/pass to access any non-static (css, js, images) file in this directory.[401]

Options -ExecCGI -Indexes +FollowSymLinks -Includes
DirectoryIndex index.php /index.php
Order Deny,Allow
Deny from All
Satisfy Any
AuthName "Protected By AskApache"
AuthUserFile /home/askapache.com/.htpasswda1
AuthType Basic
Require valid-user

Allow from All



SecFilterEngine Off

Allow from All

Protect wp-content

Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [401]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-content/.*$ [NC]
RewriteCond %{REQUEST_FILENAME} !^.+flexible-upload-wp25js.php$
RewriteCond %{REQUEST_FILENAME} ^.+\.(php|html|htm|txt)$
RewriteRule .* - [F,NS,L]

Protect wp-includes

Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /wp-includes/.*$ [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ /wp-includes/js/.+/.+\ HTTP/ [NC]
RewriteCond %{REQUEST_FILENAME} ^.+\.php$
RewriteRule .* - [F,NS,L]

Common Exploits

Block common exploit requests with 403 Forbidden. These can help alot, may break some plugins. [403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ ///.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\=?(http|ftp|ssl|https):/.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\?\?.*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(asp|ini|dll).*\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*\.(htpasswd|htaccess|aahtpasswd).*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

Stop Hotlinking

Denies any request for static files (images, css, etc) if referrer is not local site or empty. [403]

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_REFERER} !^http://www.askapache.com.*$ [NC]
RewriteRule \.(ico|pdf|flv|jpg|jpeg|mp3|mpg|mp4|mov|wav|wmv|png|gif|swf|css|js)$ - [F,NS,L]

Safe Request Methods

Denies any request not using GET,PROPFIND,POST,OPTIONS,PUT,HEAD[403]

RewriteCond %{REQUEST_METHOD} !^(GET|HEAD|POST|PROPFIND|OPTIONS|PUT)$ [NC]
RewriteRule .* - [F,NS,L]

Forbid Proxies

Denies any POST Request using a Proxy Server. Can still access site, but not comment. See Perishable Press [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:VIA}%{HTTP:FORWARDED}%{HTTP:USERAGENT_VIA}%{HTTP:X_FORWARDED_FOR}%{HTTP:PROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION}%{HTTP:HTTP_PC_REMOTE_ADDR}%{HTTP:HTTP_CLIENT_IP} !^$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

Real wp-comments-post.php

Denies any POST attempt made to a non-existing wp-comments-post.php[403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

HTTP PROTOCOL

Denies any badly formed HTTP PROTOCOL in the request, 0.9, 1.0, and 1.1 only[403]

RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ .+\ HTTP/(0\.9|1\.0|1\.1) [NC]
RewriteRule .* - [F,NS,L]

SPECIFY CHARACTERS

Denies any request for a url containing characters other than “a-zA-Z0-9.+/-?=&” – REALLY helps but may break your site depending on your links. [403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]{3,9}\ [a-zA-Z0-9\.\+_/\-\?\=\&]+\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

BAD Content Length

Denies any POST request that doesnt have a Content-Length Header[403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Length} ^$
RewriteCond %{REQUEST_URI} !^/(wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

BAD Content Type

Denies any POST request with a content type other than application/x-www-form-urlencoded|multipart/form-data[403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP:Content-Type} !^(application/x-www-form-urlencoded|multipart/form-data.*(boundary.*)?)$ [NC]
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

Missing HTTP_HOST

Denies requests that dont contain a HTTP HOST Header.[403]

RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteCond %{HTTP_HOST} ^$
RewriteRule .* - [F,NS,L]

Bogus Graphics Exploit

Denies obvious exploit using bogus graphics[403]

RewriteCond %{HTTP:Content-Disposition} \.php [NC]
RewriteCond %{HTTP:Content-Type} image/.+ [NC]
RewriteRule .* - [F,NS,L]

No UserAgent, Not POST

Denies POST requests by blank user-agents. May prevent a small number of visitors from POSTING. [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteCond %{REQUEST_URI} !^/(wp-login.php|wp-admin/|wp-content/plugins/|wp-includes/).* [NC]
RewriteRule .* - [F,NS,L]

No Referer, No Comment

Denies any comment attempt with a blank HTTP_REFERER field, highly indicative of spam. May prevent some visitors from POSTING. [403]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*/wp-comments-post\.php.*\ HTTP/ [NC]
RewriteCond %{HTTP_REFERER} ^-?$
RewriteRule .* - [F,NS,L]

Trackback Spam

Denies obvious trackback spam. See Holy Shmoly! [403]

RewriteCond %{REQUEST_METHOD} =POST
RewriteCond %{HTTP_USER_AGENT} ^.*(opera|mozilla|firefox|msie|safari).*$ [NC]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.+/trackback/?\ HTTP/ [NC]
RewriteRule .* - [F,NS,L]

Map all URIs except those corresponding to existing files to a handler

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteRule . /script.php

Map any request to a handler

In the case where all URIs should be sent to the same place (including potentially requests for static content) the method to use depends on the type of the handler. For php scripts, use:
For other handlers such as php scripts, use:

RewriteEngine On
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php

And for CGI scripts:

ScriptAliasMatch .* /var/www/script.cgi

Map URIs corresponding to existing files to a handler instead

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f
RewriteCond %{REQUEST_URI} !=/script.php
RewriteRule .* /script.php

If the existing files you wish to have handled by your script have a common set of file extensions distinct from that of the hander, you can bypass mod_rewrite and use instead mod_actions. Let’s say you want all .html and .tpl files to be dealt with by your script:

Action foo-action /script.php
AddHandler foo-action html tpl

Deny access if var=val contains the string foo.

RewriteCond %{QUERY_STRING} foo
RewriteRule ^/url - [F]

Removing the Query String

RewriteRule ^/url /url?

Adding to the Query String

Keep the existing query string using the Query String Append flag, but add var=val to the end.

RewriteRule ^/url /url?var=val [QSA]

Rewriting For Certain Query Strings

Rewrite URLs like http://askapache.com/url1?var=val to http://askapache.com/url2?var=val but don’t rewrite if val isn’t present.

RewriteCond %{QUERY_STRING} val
RewriteRule ^/url1 /url2

Modifying the Query String

Change any single instance of val in the query string to other_val when accessing /path. Note that %1 and %2 are back-references to the matched part of the regular expression in the previous RewriteCond.

RewriteCond %{QUERY_STRING} ^(.*)val(.*)$
RewriteRule /path /path?%1other_val%2

Best .htaccess Articles

.htaccess for Webmasters

Mod_Rewrite URL Rewriting

Undocumented techniques and methods will allow you to utilize mod_rewrite at an “expert level” by showing you how to unlock its secrets.

301 Redirects without mod_rewrite

Secure PHP with .htaccess

If you have a php.cgi or php.ini file in your /cgi-bin/ directory or other pub directory, try requesting them from your web browser. If your php.ini shows up or worse you are able to execute your php cgi, you’ll need to secure it ASAP. This shows several ways to secure these files, and other interpreters like perl, fastCGI, bash, csh, etc.

.htaccess Cookie Manipulation

Fresh .htaccess code for you! Check out the Cookie Manipulation and environment variable usage with mod_rewrite! I also included a couple Mod_Security .htaccess examples. Enjoy!

.htaccess Caching

Password Protection and Authentication

Control HTTP Headers

Blocking Spam and bad Bots

Want to block a bad robot or web scraper using .htaccess files? Here are 2 methods that illustrate blocking 436 various user-agents. You can block them using either SetEnvIf methods, or by using Rewrite Blocks.

PHP htaccess tips

By using some cool .htaccess tricks we can control PHP to be run as a cgi or a module. If php is run as a cgi then we need to compile it ourselves or use .htaccess to force php to use a local php.ini file. If it is running as a module then we can use various directives supplied by that modules in .htaccess

HTTP to HTTPS Redirects with mod_rewrite

This is freaking sweet if you use SSL I promise you! Basically instead of having to check for HTTPS using a RewriteCond %{HTTPS} =on for every redirect that can be either HTTP or HTTPS, I set an environment variable once with the value “http” or “https” if HTTP or HTTPS is being used for that request, and use that env variable in the RewriteRule.

SSL in .htaccess

SetEnvIf and SetEnvIfNoCase in .htaccess

Site Security with .htaccess

chmod .htpasswd files 640, chmod .htaccess 644, php files 600, and chmod files that you really dont want people to see as 400. (NEVER chmod 777, try 766)

.htaccess Security with MOD_SECURITY

Merging Notes

The order of merging is:

(except regular expressions) and .htaccess done simultaneously (with .htaccess, if allowed, overriding )
(and )
and done simultaneously
and done simultaneously

Below is an artificial example to show the order of merging. Assuming they all apply to the request, the directives in this example will be applied in the order:

A > B > C > D > E


E


D



B



C


A

My Favorite .htaccess Links

These are just some of my favorite .htaccess resources. I’m really into doing your own hacking to get knowledge and these links are all great resources in that respect. I’m really interested in new or unusual htaccess solutions or htaccess hacks using .htaccess files, so let me know if you find one.

NCSA HTTPd Tutorials

Robert Hansen
Here’s a great Hardening HTAccess part 1, part 2, part 3 article that goes into detail about some of the rarer security applications for .htaccess files.

SAMAXES
Some very detailed and helpful .htaccess articles, such as the “.htaccess – gzip and cache your site for faster loading and bandwidth saving.”

PerishablePress
Stupid .htaccess tricks is probably the best explanation online for many of the best .htaccess solutions, including many from this page. Unlike me they are fantastic writers, even for technical stuff they are very readable, so its a good blog to kick back on and read. They also have a fantastic article detailing how to block/deny specific requests using mod_rewrite.

BlogSecurity
Mostly a site for… blog security (which is really any web-app security) this blog has a few really impressive articles full of solid information for Hardening WordPress with .htaccess among more advanced topics that can be challenging but effective. This is a good site to subscribe to their feed, they publish plugin exploits and wordpress core vulnerabilities quite a bit.

Check-These
Oldschool security/unix dude with some incredibly detailed mod_rewrite tutorials, helped me the most when I first got into this, and a great guy too. See: Basic Mod_Rewrite Guide, and Advanced Mod_Rewrite Tutorial

Reaper-X
Alot of .htaccess tutorials and code. See: Hardening Wordpress with Mod Rewrite and htaccess

jdMorgan
jdMorgan is the Moderator of the Apache Forum at WebmasterWorld, a great place for answers. In my experience he can answer any tough question pertaining to advanced .htaccess usage, haven’t seen him stumped yet.

The W3C
Setting Charset in .htaccess is very informative.

Holy Shmoly!
A great blogger with analysis of attacks and spam. See: More ways to stop spammers and unwanted traffic.

Apache Week
A partnership with Red Hat back in the 90’s that produced some excellent documentation.

Corz
Here’s a resource that I consider to have some of the most creative and ingenious ideas for .htaccess files, although the author is somewhat of a character ;) Its a trip trying to navigate around the site, a fun trip. Its like nothing I’ve ever seen. There are only a few articles on the site, but the htaccess articles are very original and well-worth a look. See: htaccess tricks and tips.

Htaccess Directives

This is an AskApache.com exclusive you won’t find this anywhere else.

Directory, DirectoryMatch, Files, FilesMatch, IfDefine, IfVersion, IfModule, Limit, LimitExcept, Location, LocationMatch, Proxy, ProxyMatch, VirtualHost, AcceptMutex, AcceptPathInfo, AccessFileName, Action, AddCharset, AddDefaultCharset, AddDescription, AddEncoding, AddHandler, AddInputFilter, AddLanguage, AddOutputFilter, AddOutputFilterByType, AddType, Alias, AliasMatch, AllowCONNECT, AllowOverride, Anonymous, Anonymous_Authoritative, Anonymous_LogEmail, Anonymous_MustGiveEmail, Anonymous_NoUserId, Anonymous_VerifyEmail, AuthAuthoritative, AuthDBMAuthoritative, AuthDBMGroupFile, AuthDBMType, AuthDBMUserFile, AuthDigestAlgorithm, AuthDigestDomain, AuthDigestFile, AuthDigestGroupFile, AuthDigestNcCheck, AuthDigestNonceFormat, AuthDigestNonceLifetime, AuthDigestQop, AuthDigestShmemSize, AuthGroupFile, AuthName, AuthType, AuthUserFile, BS2000Account, BrowserMatch, BrowserMatchNoCase, CacheNegotiatedDocs, CharsetDefault, CharsetOptions, CharsetSourceEnc, CheckSpelling, ContentDigest, CookieDomain, CookieExpires, CookieName, CookieStyle, CookieTracking, CoreDumpDirectory, DAV, DAVDepthInfinity, DAVMinTimeout, DefaultIcon, DefaultLanguage, DefaultType, DocumentRoot, ErrorDocument, ErrorLog, ExtFilterDefine, ExtFilterOptions, FancyIndexing, FileETag, ForceLanguagePriority, ForceType, GprofDir, Header, HeaderName, HostnameLookups, IdentityCheck, ImapBase, ImapDefault, ImapMenu, Include, IndexIgnore, LanguagePriority, LimitRequestBody, LimitRequestFields, LimitRequestFieldsize, LimitRequestLine, LimitXMLRequestBody, LockFile, LogLevel, MaxRequestsPerChild, MultiviewsMatch, NameVirtualHost, NoProxy, Options, PassEnv, PidFile, Port, ProxyBlock, ProxyDomain, ProxyErrorOverride, ProxyIOBufferSize, ProxyMaxForwards, ProxyPass, ProxyPassReverse, ProxyPreserveHost, ProxyReceiveBufferSize, ProxyRemote, ProxyRemoteMatch, ProxyRequests, ProxyTimeout, ProxyVia, RLimitCPU, RLimitMEM, RLimitNPROC, ReadmeName, Redirect, RedirectMatch, RedirectPermanent, RedirectTemp, RemoveCharset, RemoveEncoding, RemoveHandler, RemoveInputFilter, RemoveLanguage, RemoveOutputFilter, RemoveType, RequestHeader, Require, RewriteCond, RewriteRule, SSIEndTag, SSIErrorMsg, SSIStartTag, SSITimeFormat, SSIUndefinedEcho, Satisfy, ScoreBoardFile, Script, ScriptAlias, ScriptAliasMatch, ScriptInterpreterSource, ServerAdmin, ServerAlias, ServerName, ServerPath, ServerRoot, ServerSignature, ServerTokens, SetEnv, SetEnvIf, SetEnvIfNoCase, SetHandler, SetInputFilter, SetOutputFilter, Timeout, TypesConfig, UnsetEnv, UseCanonicalName, XBitHack, allow, deny, order, CGIMapExtension, EnableMMAP, ISAPIAppendLogToErrors, ISAPIAppendLogToQuery, ISAPICacheFile, ISAPIFakeAsync, ISAPILogNotSupported, ISAPIReadAheadBuffer, SSLLog, SSLLogLevel, MaxMemFree, ModMimeUsePathInfo, EnableSendfile, ProxyBadHeader, AllowEncodedSlashes, LimitInternalRecursion, EnableExceptionHook, TraceEnable, ProxyFtpDirCharset, AuthBasicAuthoritative, AuthBasicProvider, AuthDefaultAuthoritative, AuthDigestProvider, AuthLDAPAuthzEnabled, AuthLDAPBindDN, AuthLDAPBindPassword, AuthLDAPCharsetConfig, AuthLDAPCompareDNOnServer, AuthLDAPDereferenceAliases, AuthLDAPGroupAttribute, AuthLDAPGroupAttributeIsDN, AuthLDAPRemoteUserIsDN, AuthLDAPURL, AuthzDBMAuthoritative, AuthzDBMType, AuthzDefaultAuthoritative, AuthzGroupFileAuthoritative, AuthzLDAPAuthoritative, AuthzOwnerAuthoritative, AuthzUserAuthoritative, BalancerMember, DAVGenericLockDB, FilterChain, FilterDeclare, FilterProtocol, FilterProvider, FilterTrace, IdentityCheckTimeout, IndexStyleSheet, ProxyPassReverseCookieDomain, ProxyPassReverseCookiePath, ProxySet, ProxyStatus, ThreadStackSize, AcceptFilter, Protocol, AuthDBDUserPWQuery, AuthDBDUserRealmQuery, UseCanonicalPhysicalPort, CheckCaseOnly, AuthLDAPRemoteUserAttribute, ProxyPassMatch, SSIAccessEnable, Substitute, ProxyPassInterpolateEnv

Htaccess Modules

Here are most of the modules that come with Apache. Each one can have new commands that can be used in .htaccess file scopes.

mod_actions, mod_alias, mod_asis, mod_auth_basic, mod_auth_digest, mod_authn_anon, mod_authn_dbd, mod_authn_dbm, mod_authn_default, mod_authn_file, mod_authz_dbm, mod_authz_default, mod_authz_groupfile, mod_authz_host, mod_authz_owner, mod_authz_user, mod_autoindex, mod_cache, mod_cern_meta, mod_cgi, mod_dav, mod_dav_fs, mod_dbd, mod_deflate, mod_dir, mod_disk_cache, mod_dumpio, mod_env, mod_expires, mod_ext_filter, mod_file_cache, mod_filter, mod_headers, mod_ident, mod_imagemap, mod_include, mod_info, mod_log_config, mod_log_forensic, mod_logio, mod_mem_cache, mod_mime, mod_mime_magic, mod_negotiation, mod_proxy, mod_proxy_ajp, mod_proxy_balancer, mod_proxy_connect, mod_proxy_ftp, mod_proxy_http, mod_rewrite, mod_setenvif, mod_speling, mod_ssl, mod_status, mod_substitute, mod_unique_id, mod_userdir, mod_usertrack, mod_version, mod_vhost_alias

Ultimate Htaccess Tutorial for .htaccess files originally appeared on AskApache.

AskApache » Web Hosting

Optimizing Servers and Processes for Speed with ionice, nice, ulimit

CPU and Disk I/O

RAM

SWAP

CPU

The Tools

Part 1: Process Processes Faster

Too much Processing

Breaking Bottles

nice

nice usage

Examples of nice

renice

renice usage

Examples of renice

Part 2: Optimizing Disk I/O

Linux Scheduling Policies

Scheduling Classes

ionice

ionice usage

ionice Examples

iostat

iostat Usage

iostat Examples

Schedule Utils

taskset

taskset Usage

taskset-examples

chrt

chrt Usage

chrt Examples

ulimit – get and set user limits

ulimit Usage

ulimit Examples

More Reading

Man Pages

Kernel Documentation

Future Discussions:

Advanced WordPress wp-config.php Tweaks

wp-config.php

ASKAPACHE_ROOT

ASKAPACHE_LOCK

Debugging WordPress

Ultimate Security Tweaks

Security KEYS

Using SSL for Admin and Login

Mod_Rewrite to Force SSL

File System Permissions

Cookies!

WPMU Stuff

WordPress Database

$table_prefix

Setup PHP Ini Settings

Sessions are slow

Include Custom Files

Some Useful PHP

Debugging Note

mod_rewrite Fix for Caching Updated Files

Raw Speed Benefit

Renaming links vs. Renaming files

XHTML

mod_rewrite code for htaccess or httpd.conf

Alternate mod_rewrite code

Ideal Caching Scheme

Bad Cache information for a file with the ?v=foo hack

Custom bash_profile for Advanced Shell Users

askapache-bash-profile.txt

An AskApache Plugin Upgrade to Rule them All

The Strategy

AskApache Google 404 Upgrade

AskApache Password Protection

The Upgrades Begin

Debugging HTTP protocol

.htaccess Directives

PHP and AJAX shell console

PHP for shell.php

Read this

Example shell script as cgi

Locking Down Access to your shell.php

Bad Cache information for a file with the `?v=foo` hack

PHP for `shell.php`

`wp-admin/includes/file.php`

`ServerSignature` for `ErrorDocument`