This article shows how I continuously scroll the logs for a server/site I am working on, thus saving me a lot of time by providing real-time debugging on a separate screen. Not only does this scroll the latest log entries as they are created, it displays them in color using syntax highlighting to make your logs incredibly easy to understand and parse.
The thing to realize is that this output is continuously scrolling on your monitor, and using some cool linux shell tricks you can make it output at a certain speed and show a certain number of lines. Another cool feature is you can display multiple files at the same time, and the filename will be output for each file above the log output.
I've explained in various articles on this site how to create and use various custom log files using .htaccess files and other tricks for non-root users. I use shared hosting all the time too you know... Here are some log files you can generally use if you can use .htaccess files (if you can control .htaccess you can control a binary php-cgi, which could be a shell script thats execs the php binary of your choice with your own environment variables, which is a workaround to getting a custom php.ini, which is how you specify a php error log).
- php error log
- mod_security audit log
- mod_security debug log
- apache error_log
- apache access_log
Any log file can be used with this method, actually ANY file containing text can be used with this method, even fifo pipes as we will see in a bit ;)
Now you need CCZE, which colorizes and outputs (emulating tail) the log files.
CCZE is a robust and modular log colorizer with plugins for apm, exim, fetchmail, httpd, postfix, procmail, squid, syslog, ulogd, vsftpd, xferlog, and more.
Ok so once you've installed ccze you can start using it right away, or you can continue reading to see how I've set it up for readable scrolling. If the following setup isn't to your taste, you can always just use wtail, a colorless multi-file scroller that uses separate scrolling windows in your terminal similar to the screen program.
First I created a folder called
/logs/ for my site, then for all subsequent commands chdir to it.
$ mkdir /web/site.com/logs $ cd /web/site.com/logs
In the new logs folder I created soft symlinks to all the various log files, not neccessary but makes everything much easier and organized.
$ ln -s /actual/logs/site.com/http/access.log access.log $ ln -s /actual/logs/site.com/http/error.log error.log $ ln -s /web/site.com/php_error.log php_error.log $ ln -s /web/site.com/modsec_audit.log modsec_audit.log $ ln -s /web/site.com/modsec_debug.log modsec_debug.log
FIFO stands for first-in-first-out and is a somewhat complex feature of linux/bsd/unix shells that let you send data into it and read the data that comes out of it, just like a pipe.
$ mkfifo pipe
First cd to the log directory you created.
$ cd /web/site.com/logs
Now we will use the tail program to output 120 lines of each of these log files every half of a second, only displaying changes/new log entries. We send that output into the fifo pipe we created using shell redirection and then instruct the command to run in the background using the ampersand to access built-in (bash/sh) job control.
$ tail -s .5 -n 120 -f access.log php_error.log error.log modsec_audit.log >pipe &
So every .5 seconds the tail command outputs any new log entries from those files into the fifo pipe, so now we need to use the ccze program to use the fifo pipe as its input log file. Normally you run ccze like the cat command and it just outputs the input colorized, by using tail and a fifo pipe we are able to make this awesome trick work.
$ ccze < pipe >/dev/pts/2 &
This appends the colorized output to the file in the CWD, super.log$ ccze -A
Immediately kills any processes used by tail and ccze.$ pkill -9 ccze|tail
Here are some other examples of using ccze with fifo pipes, more as an excercise than anything practical as most lines don't work, I just grabbed them from my shell history file.tail -f access.log >pipe & ccze
/dev/pts/2 tail -f access.log >pipe & ccze /dev/pts/2 & exec 3<> pipe; while ccze >/dev/pts/2 <&3; do tail -f access.log >pipe; done; exec 3>&- exec 3<> pipe; ccze <&3 >/dev/pts/2 & ; tail -f access.log >pipe; exec 3>&- exec 3<> pipe; ccze <&3 >/dev/pts/2 & ; tail -f access.log >pipe exec 3<> pipe; ccze <&3 >/dev/pts/2 & tail -f access.log >pipe ccze <&3 >/dev/pts/2 & ccze 0<&3 |tee -a super.log > /dev/pts/2 & ccze <&3 |tee -a super.log > /dev/pts/2 & ccze pipe & ccze | tee -a /dev/pts/2 /dev/pts/2 ccze <|tee -a /dev/pts/2 pipe >/dev/pts/2 ccze > tee -a super.log >/dev/pts/2 cat >/dev/pts/2 & tail -fq access.log | ccze &>/dev/pts/1 tail -qf access.log |ccze &>/dev/pts/1 & tail -f access.log | ccze > $SSH_TTY ( ccze > /dev/pts/2 ) /dev/pts/2 ) /dev/pts/2 ) /dev/pts/2 ) /dev/pts/2 & tail -f access.log >pipe ccze /dev/pts/2 & tail -f access.log >pipe ccze < pipe > /dev/pts/2 & tail -f access.log >pipe ccze < pipe >/dev/pts/2 & tail -f access.log >pipe ccze < pipe >/dev/pts/2 & tail -f access.log >pipe & ccze < pipe >/dev/pts/2 & tail -n 40 -s 2 -f error.log modsec_audit.log php_error.log >pipe & ccze < pipe >| tee -a super.log | >/dev/pts/2 & tail -n 80 -s 1 -f error.log modsec_audit.log php_error.log >pipe & ccze < pipe >/dev/pts/2 & tail -n 100 -s 1 -f error.log modsec_audit.log php_error.log >pipe & ccze < pipe >/dev/pts/2 & tail -n 100 -s 1 -f error.log modsec_audit.log php_error.log >pipe & tail -f access.log | ccze > pipe & /dev/pts/2/ pipe & /dev/pts/2
I also hacked together a little shell script that you may wish to hack to your needs. Its actually pretty sweet if you can figure it out.#!/bin/bash -l set -o xtrace renice -p $$ 15 pkill -9 tee|ccze &>/dev/null || echo -n disown -a &>/dev/null || echo [[ -r "~/pipe" ]] && rm -rf ~/pipe mkfifo ~/pipe cd /web/askapache.com/logs [[ -r "superlog.log" ]] && echo "Found old logfile, moving." && mv superlog.log `command ls|wc -l`-superlog.log ccze <$HOME/pipe > $SSH_TTY & tail -n 150 -s .5 -f php_error.log error.log modsec_audit.log >$HOME/pipe & disown %2; && disown %3; disown -a sleep 60 & disown $! || disown %1 for i in `seq 1 4`;do echo -n $'a'; sleep 1;done kill -9 $J1 kill -9 $J2 logout exit 0
August 10th, 2008