AskApache Search Results:

.htaccess Plugin Blocks Spam, Hackers, and Password Protects Blog

AskApache — Sun, 06 Jul 2008 19:18:12 +0000

I finally got around to updating this plugin, its now half-way decent code-wise. I’ve added a lot of new security modules and improved the old ones. The original plugin page and description can be found here. You can thank this guy for this update, his article about blocking spam with .htaccess inspired me to do it.

.htaccess Security Modules

Directory Protection
Enable the DirectoryIndex Protection, preventing directory index listings and defaulting. [Disable]
Password Protect wp-login.php
Requires a valid user/pass to access the login page - *** Safe, Use. [401]
Password Protect wp-admin
Requires a valid user/pass to access any non-static (css, js, images) file in this directory. - *** Safe, Use. [401]
Protect wp-content
Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [401]
Protect wp-includes
Denies any Direct request for files ending in .php with a 403 Forbidden.. May break plugins/themes [403]
Common Exploits
Block common exploit requests with 403 Forbidden. These can help alot, may break some plugins. [403]
Stop Hotlinking
Denies any request for static files (images, css, etc) if referrer is not local site or empty. [403]
Safe Request Methods
Denies any request not using GET,PROPFIND,POST,OPTIONS,PUT,HEAD - *** Safe, Use. [403]
Forbid Proxies
Denies any POST Request using a Proxy Server. Can still access site, but not comment. See Perishable Press [403]
Real wp-comments-post.php
Denies any POST attempt made to a non-existing wp-comments-post.php - *** Safe, Use. [403]
HTTP PROTOCOL
Denies any badly formed HTTP PROTOCOL in the request, 0.9, 1.0, and 1.1 only - *** Safe, Use. [403]
SPECIFY CHARACTERS
Denies any request for a url containing characters other than “a-zA-Z0-9.+/-?=&” - REALLY helps but may break your site depending on your links. [403]
BAD Content Length
Denies any POST request that doesnt have a Content-Length Header - *** Safe, Use. [403]
BAD Content Type
Denies any POST request with a content type other than application/x-www-form-urlencoded|multipart/form-data - *** Safe, Use. [403]
Directory Traversal
Denies Requests containing ../ or ./. which is a directory traversal exploit attempt - *** Safe, Use. [403]
PHPSESSID Cookie
Only blocks when a PHPSESSID cookie is sent by the user and it contains characters other than 0-9a-z - *** Safe, Use. [403]
NO HOST:
Denies requests that dont contain a HTTP HOST Header. - *** Safe, Use. [403]
Bogus Graphics Exploit
Denies obvious exploit using bogus graphics - *** Safe, Use. [403]
No UserAgent, No Post
Denies POST requests by blank user-agents. May prevent a small number of visitors from POSTING. [403]
No Referer, No Comment
Denies any comment attempt with a blank HTTP_REFERER field, highly indicative of spam. May prevent some visitors from POSTING. [403]
Trackback Spam
Denies obvious trackback spam. See Holy Shmoly! [403]
SSL-Only Site
Redirects all non-SSL (https) requests to your https-enabled url [301]
Anti-Spam, Anti-Exploits
Denies Obvious Spam and uses advanced mod_security protection [Read More]

.htaccess Security Module Screenshot

Encrypted WordPress Site Backups

AskApache — Fri, 04 Jul 2008 16:28:00 +0000

Because backups contain all your sensitive information, its smart to encrypt any sql backups.. and while we’re at it, also encrypt any site backups.

This simple shell-script is a useful and easy way to securely backup your wordpress site files and database without confusing you. Just generate a GPG key once, enter in 3 settings once, and from then on it runs without any user-input whenever you want.

What it Does

When run, this script asks you for the location of your websites document root and the location of your wp-config.php file. It also asks you for your encryption UID. Then this script saves those settings in a file called .sbackup so that the next time you run the script it will run without having to re-enter that information, making it nice for cronjobs or quick and easy on-demand backups. Another cool feature that I added is this script automatically parses your wp-config.php file for the mysql database name, user, host, and password, meaning you don’t have to compromise your security or take the time to type those settings in manually.

What is Backed Up

This script creates a tarred and gzipped archive of your entire document root in the folder ~/backups/domain.com/domain.com-date.tgz and also creates a backup of your WordPress database in a format that is ideal for restoring from. Both of these files are then encrypted using your GPG key and can then be safely downloaded as a password and key is required to decrypt them.

Generating a GPG Key

If you don’t already have one setup for your shell account run this command remembering the uid which you will enter in the shell script.

gpg --gen-key

Decrypting Files

gpg -r UID --output FILENAME.tgz --decrypt FILENAME.tgz.asc

The Shell Script

site-backup

#!/bin/bash
# SiteBack Version 3.1, 2008-07-04
# GNU Free Documentation License 1.2
# 07-04-08 - AskApache (www.askapache.com)
umask 022
 
### SHELL OPTIONS
set +o noclobber # allowed to clobber files
set +o noglob # globbing on
set +o xtrace # change to - to enable tracing
set +o verbose # change to - to enable verbose debugging
set -e # abort on first error
 
shopt -s extglob
 
###########################################################################--=--=--=--=--=--=--=--=--=--=--#
###
### SETTINGS
###
###########################################################################==-==-==-==-==-==-==-==-==-==-==#
 
DT=$(date +%x); DT=${DT//\/}
DTX=$(date +%x-%H%M); DTX=${DTX//\/}
BDIR=${HOME}/backups
RUN_FILE=${BDIR}/$$.bk.log
MY_CONFIG=".sbackup"
DOMAIN=;DB_NAME=;DB_USER=;DB_PASSWORD=;DB_HOST=;WP_CONFIG=;SQL_DEST=;ARC_DEST=;ENCRYPT_USER=
E_SUCCESS=0;E_YN=0;E_YES=251;E_NO=250;E_RETURN=65;C0=;C1=;C2=;C3=;C4=;C5=;C6=;C7=;C8=;C9=
 
###########################################################################--=--=--=--=--=--=--=--=--=--=--#
###
### FUNCTIONS
###
###########################################################################==-==-==-==-==-==-==-==-==-==-==#
 
#--=--=--=--=--=--=--=--=--=--=--#
# script_title
#==-==-==-==-==-==-==-==-==-==-==#
function script_title(){
 # SET WINDOW TITLE AND COLORS IF CLIENT CAPABLE
 case ${TERM:-dummy} in
  xterm*|vt*|ansi|rxvt|gnome*)
  C0="\033[0m";C1="\033[1;30m";C2="\033[1;32m";C3="\033[0;32m";C4="\033[1;37m"
  C5="\033[0;36m";C6="\033[1;35m";C7="\033[0;37m";C8="\033[30;42m";C9="\033[1;36m"
 esac  
 echo -e "${C1} __________________________________________________________________________ "
 echo -e "| ${C2}             ___       __    ___                 __             ${C1}         |"
 echo -e "| ${C2}            / _ | ___ / /__ / _ | ___  ___ _____/ /  ___        ${C1}         |"
 echo -e "| ${C2}           / __ |(_->> ${C4}${1} ${C0} \n\n"; ;;
   "info") echo -e "${C6}=> ${C4}${1} ${C0}"; ;;
   "item") echo -e "${C4}-- ${C0}${1} "; ;;
 esac
}
 
#--=--=--=--=--=--=--=--=--=--=--#
# yes_no
#==-==-==-==-==-==-==-==-==-==-==#
function yes_no(){
 local ans
 echo -en "${1} [y/n] " ; read -n 1 ans
 case "$ans" in
  n|N) E_YN=$E_NO ;;
  y|Y) E_YN=$E_YES ;;
 esac
}
 
#--=--=--=--=--=--=--=--=--=--=--#
# do_sleep
#==-==-==-==-==-==-==-==-==-==-==#
function do_sleep (){ 
 local END DIFF
 echo -en "${C5}${3:-.}"; while [ -r "$RUN_FILE" ]; do sleep ${2:-3}; echo -en "${3:-.}"; done;
 echo -e "${C0}"; sleep 1; END=$(date +%s);DIFF=$(( $END - $START ))
 echo -e "\n${C8} [T: ${SECONDS}] COMPLETED IN ${DIFF} SEC ${C0} \n\n"; sleep 1; 
 return 0; 
}
 
#--=--=--=--=--=--=--=--=--=--=--#
# get_settings
#==-==-==-==-==-==-==-==-==-==-==#
function get_settings(){
 local cha HOSTED_SITES G
 clear; script_title
 if [[ -r "$MY_CONFIG" ]]; then
  OIFS=$IFS; while IFS=: read DOMAIN DOMAINROOT WP_CONFIG ENCRYPT_USER; do
   DOMAIN=${DOMAIN}; DOMAINROOT=${DOMAINROOT}; WP_CONFIG=${WP_CONFIG}; ENCRYPT_USER=${ENCRYPT_USER}; E_YN=$E_YES; break
  done <${MY_CONFIG}; IFS=$OIFS
 else
 echo -en "\n What domain would you like to backup?  "; read -e DOMAIN; echo
 until [ -d "$DOMAINROOT" ]; do echo -en "\n Where is the domain root?  "; read -e DOMAINROOT; echo; done
 [[ -r "$DOMAINROOT/wp-config.php" ]] && WP_CONFIG=$DOMAINROOT/wp-config.php
 until [[ -r "$WP_CONFIG" ]]; do echo -en "\n Where is the wp-config.php file?  "; read -e WP_CONFIG; echo; done
 echo -en "\n What userid to use for encryption?  "; read -e ENCRYPT_USER; echo
 fi
 
 [[ -r "$WP_CONFIG" ]] && G=$(sed -e "/define('DB_\(NAME\|USER\|PASSWORD\|HOST\)/!d" \
 -e "s/[^']*'DB_\(NAME\|USER\|PASSWORD\|HOST\)'[^']*'\([^']*\)'.*$/DB_\1='\2';/g" ${WP_CONFIG}) && eval $G;
 mkdir -p ${BDIR}/${DOMAIN}
 SQL_DEST=${BDIR}/${DOMAIN}/${DOMAIN}-${DT}.sql;  [[ -r "${SQL_DEST}.asc" ]] && SQL_DEST=${BDIR}/${DOMAIN}/${DOMAIN}-${DTX}.sql
 ARC_DEST=${BDIR}/${DOMAIN}/${DOMAIN}-${DT}.tgz; [[ -r "${ARC_DEST}.asc" ]] && ARC_DEST=${BDIR}/${DOMAIN}/${DOMAIN}-${DTX}.tgz
 
 if [[ "$E_YN" != "$E_YES" ]]; then
  for a in "DOMAIN" "DOMAINROOT" "WP_CONFIG" "ENCRYPT_USER" "DB_NAME" "DB_USER" "DB_PASSWORD" "DB_HOST"; do echo -e "${a}: ${!a}"; done
  echo; yes_no "ARE THESE SETTINGS CORRECT"
 fi

 while [[ "$E_YN" != "$E_YES" ]]; do
  for a in "DOMAIN" "DOMAINROOT" "WP_CONFIG" "ENCRYPT_USER" "DB_NAME" "DB_USER" "DB_PASSWORD" "DB_HOST"; do
   echo -en "\n (Enter for Default: ${!a} )\n ${a}:> "
   read -e cha; echo; [[ ${#cha} -gt 2 ]] && eval "$a"=$cha
  done
  yes_no "ARE THESE SETTINGS CORRECT"
 done
   
 echo -e "${DOMAIN}:${DOMAINROOT}:${WP_CONFIG}:${ENCRYPT_USER}" > $MY_CONFIG
}
 
#--=--=--=--=--=--=--=--=--=--=--#
# exit_cleanup
#==-==-==-==-==-==-==-==-==-==-==#
function exit_cleanup(){
 cd $OLDPWD
 [[ -r ${SQL_DEST} ]] && rm ${SQL_DEST}
 [[ -r ${ARC_DEST} ]] && rm ${ARC_DEST}
}
 
############################################################################################################
###
### MAIN CODE
###
############################################################################################################
 
#=# CATCH SCRIPT KILLED BY USER
trap exit_cleanup SIGHUP SIGINT SIGTERM
 
#=# MAKE MAIN SCRIPT NICE  
renice 19 -p $$ &>/dev/null
 
cd `dirname $0`
 
get_settings
 
pm "CREATING SQL BACKUP"
mysqldump --opt -u${DB_USER} -p${DB_PASSWORD} -h ${DB_HOST} -r ${SQL_DEST} \
--add-drop-table ${DB_NAME} 1>&2 &>/dev/null && sleep 2 1>&2 &>/dev/null && rm ${RUN_FILE} 2>&1& 
do_sleep 1 1 ":"
 
pm "ENCRYPTING SQL BACKUP"
gpg --armor --recipient ${ENCRYPT_USER} --output ${SQL_DEST}.asc --encrypt ${SQL_DEST} \
1>&2 &>/dev/null && sleep 2 1>&2 &>/dev/null && rm ${RUN_FILE} 2>&1& 
do_sleep 1 1 ":"; rm ${SQL_DEST}
 
pm "CREATING ARCHIVE BACKUP"
tar -czf ${ARC_DEST} . 1>&2 &>/dev/null && rm ${RUN_FILE} 2>&1& 
do_sleep 1 5 ":"
 
pm "ENCRYPTING ARCHIVE BACKUP"
gpg --armor --recipient ${ENCRYPT_USER} --output ${ARC_DEST}.asc --encrypt ${ARC_DEST} \
1>&2 &>/dev/null && rm ${RUN_FILE} 2>&1& 
do_sleep 1 1 ":"; rm ${ARC_DEST}
 
echo -e "${C1} __________________________________________________________________________ "
echo -e "|                                                                          |"
echo -e "|                 ${C4} COMPLETED SUCCESSFULLY ${C1}                                 |"
echo -e "${C1} __________________________________________________________________________ ${C0} \n\n"
 
cd $OLDPWD
 
exit $?

Fsockopen Power Plays

AskApache — Wed, 02 Jul 2008 11:42:56 +0000

PHP’s function fsockopen lets you open an Internet or Unix domain socket connection for connecting to a resource, and is one of the most powerful functions. fsockopen could be described as creating a direct link to the wire connected to a resource, which means you can send any information (EBCDIC, ASCII, Hex, C arrays, Raw) directly to the target server.

A Socket is like /dev/null

In unix you can send anything to the /dev/null device, for Windows think Recycle Bin, and likewise you can send anything to a socket created with fsockopen. I’ve seen fsockopen code that sends custom exploits to cisco routers, including being used by the metasploit framework. I’ve seen fsockopen telnet emulation, smtp/pop3 login, and a lot of other advanced raw networking that is exciting for me see.

Some Definitions for Fsockopen

client: A program that establishes connections for the purpose of sending requests.
server: An application program that accepts connections in order to service requests by sending back responses.

Simple Socket Explantion

A web server host listens on TCP port 80. When a client host wishes to view a resource on the web server, it establishes a TCP connection with the server host by opening a socket to send the request for the resource. When the connection is established, the client and server exchange requests and responses (respectively) until the connection is closed or aborted.

HTTP and fsockopen

The Snoopy class is bundled with WordPress distributions and uses fsockopen to achieve most of its cool features. WordPress core, plugins, and other included files and classes also use the fsockopen function to communicate via HTTP.

Fsockopen Examples

Note the warning sign, fsockopen is dangerous in the sense that you can crash your server, perform a DOS against your own server or other site, use up all your servers available sockets and fd descriptors, use up your bandwidth, etc.. Shouldn’t be a problem unless you are being malicious or careless.

Here are some BOSS fsockopen functions I hacked together yesterday for use in my AskApache Crazy Cache WordPress Plugin. I’ve used code and ideas from 100’s of authors, projects, and docs to try to make this the very best I can.

Intro

This is a working example employing as many of the best-practices, tips, and tricks for using fsockopen on remote streams that I could find.

100megs */
function askapache_txrx($fp,$request,$chunk=1024){
  $rec=$buf='';
  if(!@fwrite($fp, $request, strlen($request)))die('fwrite error');
  while ( !@feof($fp) && askapache_time_ok(askapache_time_passed())){
    $buf = @fread($fp, $chunk);
    $rec .= $buf;
  }
  if(!@fclose($fp))die('fclose error');
  return $rec;
}
 
/* initiates the socket and download for the passed url.
   automatically handles gzip, chunked, both, and plain downloads.
   uses the long2ip/ip2long for ip validation, uses gethostbyname to 
   get the ipv4 address which saves fsockopen from having to do the lookup
   final data is saved to $rbody but currently only displays headers.*/
function aa_dl($url=NULL){
  global $aa_time;
  $ub = @parse_url($url);
  if(!isset($ub['host'])||empty($ub['host'])) die("bad url $url"); 
  $proto   = ($ub['scheme']=='https')?'ssl://':'';
  $port   = (isset($ub['port'])&&!empty($ub['port'])) ? $ub['port']:($proto!='')?443:80;
  $path   = (isset($ub['path'])&&!empty($ub['path'])) ? $ub['path']:'/';
  $query   = (isset($ub['query'])&&!empty($ub['query'])) ? '?'.$ub['query'] : '';
  $host   = $ub['host'];
  $ipp     = @gethostbyname($host);
  $ip     = ($ipp!=$host) ? long2ip(ip2long($ipp)) : $host;
  
  $headers=array(
   "GET {$path}{$query} HTTP/1.1",
   "Host: {$host}",
   'User-Agent: Mozilla/5.0 (AskApache/; +http://www.askapache.com/)',
   'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,*/*;q=0.5',
   'Accept-Language: en-us,en;q=0.5',
   'Accept-Encoding: gzip,deflate',
   'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7',
   'Connection: close','Referer: http://www.askapache.com'
  );
  $request=join(AA_LF,$headers).AA_LF.AA_LF;
  
  $fp=askapache_get_sock($proto.$ip, $port);
  if($fp){
    $rbody=$rec='';$resp_headers=array();
    $rec=askapache_txrx($fp,$request);
    list($resp_headers, $rbody) = explode(AA_LF.AA_LF, trim($rec), 2);
  echo "\n$request
\n$resp_headers\n";
    $gzip2=(stripos($resp_headers,'Content-Encoding')!==false && 
        stripos($resp_headers,'gzip')!==false)?1:0;
    $chunk=(stripos($resp_headers,'Transfer-Encoding')!==false && 
        stripos($resp_headers,'chunked')!==false)?1:0;
    $rbody=aa_decode_body($rbody,$chunk,$gzip2);
    unset($rbody);
  }
}
 
/* based on http://us.php.net/manual/en/function.fsockopen.php#75175 
   ungzips and/or re-assembles transfer-encoded:chunked responses
   returns the good response on success */
function aa_decode_body ($str, $chunked, $gzipped){
  if($gzipped && !$chunked) return aa_gzdecode($str);
  if(!$gzipped && !$chunked) return $str;
  $tmp = $str; $str = '';
  do {
    $tmp = ltrim($tmp);
    $pos = strpos($tmp,AA_LF);
    $len = hexdec(substr($tmp, 0, $pos));
    if($gzipped) $str .= gzinflate(substr($tmp,($pos+12),$len));
    else $str .=substr($tmp,($pos+2),$len);
    $tmp = substr($tmp,($len+$pos+2));
  $chk=trim($tmp);
  } while (!empty($chk));
  return $str;
}
 
/*  based on http://us2.php.net/manual/en/function.gzencode.php#82520 
  saves the gzipped data to a tempfile, then outputs the decoded
  data to the output buffer using readgzfile, returning the decoded
  buffer and deleting the tempfile on success */
function aa_gzdecode($data){
  $g=tempnam('/tmp','ff');
  @file_put_contents($g,$data);
  ob_start(); readgzfile($g); $d=ob_get_clean(); @unlink($g);
  return $d;
}
 
/*  very cool!  this is run during socket reads and checks whether the script
  execution time limit or the socket read time limit has been met, killing
  the script if so, otherwise returns true.  Run with a cron-like process */
function askapache_time_ok($sock_time=0) {
  global $aa_time;
  if (time()-$aa_time>AA_MAX_TIME) 
    die('killed script.. time exceeded '.AA_MAX_TIME.' Total: '.$total);
  if ($sock_time>AA_RECV_TIME) 
    die('Killed socket.. time exceeded '.AA_RECV_TIME.' Total: '.$sock_time);
  return true;
}
 
/* input for askapache_time_ok to keep track of each socket read time time. */
function askapache_time_passed() {
  global $aa_time_start;
  return (time() - $aa_time_start);
}
 
/*  handles fsockopen errors, printing them out though you may want to die on err */
function askapache_sock_strerror($errno,$errstr){
  switch($errno){
    case -3:  $err="Socket creation failed"; break;
    case -4:  $err="DNS lookup failure"; break;
    case -5:  $err="Connection refused or timed out"; break;
    case 111: $err="Connection refused"; break;
    case 113: $err="No route to host"; break;
    case 110: $err="Connection timed out"; break;
    case 104: $err="Connection reset by client"; break;
    default:  $err="Connection failed"; break;
  }
  echo 'Fsockopen failed!'."\n[".$errno."] ".$err." (".$errstr.")";
  return false;
}
?>

Debugging Fsockopen

If you really want to know more about fsockopen, you can do what I did and read all the relevant php source files, your OS sys, lib, and user files relevant to fsockopen, and of course you can always trace php using the fsockopen function to get an under-the-hood look at what in the world fsockopen is doing. Personally, I was trying to find more error codes and error strings to display when an fsockopen call failed, and I ended up finding over 50..

Tracing fsockopen using Strace

Once you save the above file on your site, you can use the strace tool to debug it. This is a tad overboard but way cool nevertheless!

strace -e trace=connect php -nef fsockopen-test.php

connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("192.87.106.226")}, 16) = -1 EINPROGRESS (Operation now in progress)

strace -e trace=network php -nef fsockopen-test.php

socket(PF_FILE, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
send(3, "\274\221\1\0\0\1\0\0\0\0\0\0\5httpd\6apache\3org\0\0\1"..., 34, 0) = 34
recvfrom(3, "\274\221\201\200\0\1\0\1\0\0\0\0\5httpd\6apache\3org\0"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, [16]) = 50
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = -1 EAFNOSUPPORT (Address family not supported by protocol)
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("192.87.106.226")}, 16) = -1 EINPROGRESS (Operation now in progress)
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
send(3, "GET / HTTP/1.1\r\nHost: httpd.apac"..., 356, MSG_DONTWAIT) = 356
recv(3, "HTTP/1.1 200 OK\r\nDate: Wed, 02 J"..., 8192, MSG_DONTWAIT) = 2609
recv(3, "", 8192, MSG_DONTWAIT)         = 0

strace -q -e trace=all php -nef fsockopen-test.php

mmap2(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb76ba000
munmap(0xb76ba000, 266240)              = 0
socket(PF_FILE, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No such file or directory)
close(3)                                = 0
open("/etc/hosts", O_RDONLY)            = 3
fcntl64(3, F_GETFD)                     = 0
fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=948, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f6e000
read(3, "# /etc/hosts - dh2 generated\n127"..., 4096) = 948
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0xb7f6e000, 4096)                = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
send(3, "X~\1\0\0\1\0\0\0\0\0\0\2en\twikipedia\3org\0\0\1"..., 34, 0) = 34
gettimeofday({1214998196, 656179}, NULL) = 0
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
ioctl(3, FIONREAD, [100])               = 0
recvfrom(3, "X~\201\200\0\1\0\3\0\0\0\0\2en\twikipedia\3org\0\0\1"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, [16]) = 100
close(3)                                = 0
time(NULL)                              = 1214998196
gettimeofday({1214998196, 656754}, NULL) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
fcntl64(3, F_GETFL)                     = 0×2 (flags O_RDWR)
fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("208.80.152.2")}, 16) = -1 EINPROGRESS (Operation now in progress)
poll([{fd=3, events=POLLIN|POLLOUT|POLLERR|POLLHUP, revents=POLLOUT}], 1, 10000) = 1
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
fcntl64(3, F_SETFL, O_RDWR)             = 0
send(3, "GET /wiki/Main_Page HTTP/1.1\r\nHo"..., 370, MSG_DONTWAIT) = 370
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "HTTP/1.0 200 OK\r\nDate: Wed, 02 J"..., 8192, MSG_DONTWAIT) = 2896
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "\214!\337i\307\336\23w\253wy\215\26EL\227;\227\253\261"..., 8192, MSG_DONTWAIT) = 5792
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "4\201\273\214\17yI\347\257\371\373\344\332\330\227\245"..., 8192, MSG_DONTWAIT) = 7487
time(NULL)                              = 1214998197
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "", 8192, MSG_DONTWAIT)         = 0
close(3)                                = 0
write(1, "\nGET /wiki/Main_Page HTTP/1"..., 1300


More Fsockopen Info
TCP Multiplexing
RFC 793: To allow for many processes within a single Host to use TCP communication facilities simultaneously, the TCP provides a set of addresses or ports within each host.  Concatenated with the network and host addresses from the internet communication layer, this forms a socket.  A pair of sockets uniquely identifies each connection. That is, a socket may be simultaneously used in multiple connections.
The binding of ports to processes is handled independently by each Host.  However, it proves useful to attach frequently used processes (e.g., a “logger” or timesharing service) to fixed sockets which are made known to the public.  These services can then be accessed through the known addresses.  Establishing and learning the port addresses of other processes may involve more dynamic mechanisms.
TCP Connections
The reliability and flow control mechanisms described above require that TCPs initialize and maintain certain status information for each data stream.  The combination of this information, including sockets, sequence numbers, and window sizes, is called a connection. Each connection is uniquely specified by a pair of sockets identifying its two sides.
When two processes wish to communicate, their TCP’s must first establish a connection (initialize the status information on each side).  When their communication is complete, the connection is terminated or closed to free the resources for other uses.
Since connections must be established between unreliable hosts and over the unreliable internet communication system, a handshake mechanism with clock-based sequence numbers is used to avoid erroneous initialization of connections.
Fsockopen Practical Uses

Download Web Pages, Files, etc.
Upload a file
Send POST data to a form
Emulate cron
Download plugin updates
Scan sites for exploits
Auto Login to Google
Pass wp-nonces via cookie headers, and more

Transfer-Encoding
RFC 2068
19.4.6 Introduction of Transfer-Encoding
 
   HTTP/1.1 introduces the Transfer-Encoding header field (section
   14.40).  Proxies/gateways MUST remove any transfer coding prior to
   forwarding a message via a MIME-compliant protocol.
 
   A process for decoding the "chunked" transfer coding (section 3.6)
   can be represented in pseudo-code as:
 
          length := 0
          read chunk-size, chunk-ext (if any) and CRLF
          while (chunk-size > 0) {
             read chunk-data and CRLF
             append chunk-data to entity-body
             length := length + chunk-size
             read chunk-size and CRLF
          }
          read entity-header
          while (entity-header not empty) {
             append entity-header to existing header fields
             read entity-header
          }
          Content-Length := length
          Remove "chunked" from Transfer-Encoding

Socket-Related Man Pages
DESCRIPTION
This  manual  page  describes the Linux networking socket layer user interface. The BSD compatible sockets are the uniform interface between
the user process and the network protocol stacks in the kernel.  The protocol modules are  grouped  into  protocol  families  like  PF_INET,
PF_IPX, PF_PACKET and socket types like SOCK_STREAM or SOCK_DGRAM.  See socket(2) for more information on families and types.
 
SOCKET LAYER FUNCTIONS
These  functions  are  used by the user process to send or receive packets and to do other socket operations. For more information see their
respective manual pages.
 
socket(2) creates a socket, connect(2) connects a socket to a remote socket address, the bind(2) function binds a socket to a  local  socket
address,  listen(2)  tells  the socket that new connections shall be accepted, and accept(2) is used to get a new socket with a new incoming
connection.  socketpair(2) returns two connected anonymous sockets (only implemented for a few local families like PF_UNIX)
 
send(2), sendto(2), and sendmsg(2) send data over a socket, and recv(2), recvfrom(2), recvmsg(2) receive data from a  socket.   poll(2)  and
select(2)  wait  for  arriving  data  or a readiness to send data.  In addition, the standard I/O operations like write(2), writev(2), send-
file(2), read(2), and readv(2) can be used to read and write data.
 
getsockname(2) returns the local socket address and getpeername(2) returns the remote socket address.  getsockopt(2) and  setsockopt(2)  are
used to set or get socket layer or protocol options.  ioctl(2) can be used to set or read some other options.
 
close(2) is used to close a socket.  shutdown(2) closes parts of a full duplex socket connection.
 
Seeking, or calling pread(2) or pwrite(2) with a non-zero position is not supported on sockets.
 
It  is possible to do non-blocking IO on sockets by setting the O_NONBLOCK flag on a socket file descriptor using fcntl(2).  Then all opera-
tions that would block will (usually) return with EAGAIN (operation should be retried later); connect(2) will return EINPROGRESS error.  The
user can then wait for various events via poll(2) or select(2).

From the FreeBSD man page for socket(2)
Sockets of type SOCK_STREAM are full-duplex byte streams, similar to
pipes.  A stream socket must be in a connected state before any data may
be sent or received on it.  A connection to another socket is created
with a connect(2) system call.  Once connected, data may be transferred
using read(2) and write(2) calls or some variant of the send(2) and
recv(2) functions.  (Some protocol families, such as the Internet family,
support the notion of an ``implied connect'', which permits data to be
sent piggybacked onto a connect operation by using the sendto(2) system
call.)  When a session has been completed a close(2) may be performed.
Out-of-band data may also be transmitted as described in send(2) and
received as described in recv(2).
 
The communications protocols used to implement a SOCK_STREAM insure that
data is not lost or duplicated.  If a piece of data for which the peer
protocol has buffer space cannot be successfully transmitted within a
reasonable length of time, then the connection is considered broken and
calls will indicate an error with -1 returns and with ETIMEDOUT as the
specific code in the global variable errno.  The protocols optionally
keep sockets ``warm'' by forcing transmissions roughly every minute in
the absence of other activity.  An error is then indicated if no response
can be elicited on an otherwise idle connection for an extended period
(e.g. 5 minutes).  A SIGPIPE signal is raised if a process sends on a
broken stream; this causes naive processes, which do not handle the sig-
nal, to exit.

Have Fun   ;)

Adding Print Capability to your Site with CSS

AskApache — Tue, 01 Jul 2008 04:38:56 +0000

Its really nice to be able to print out a webpage you are reading using your browsers built-in print feature. Using CSS you can easily transform your site into a print-friendly site.

Today I received an email from a visitor to my site requesting that I add a way to print site articles on AskApache

Finally, you have so much great stuff that I need to print it take it offline so I can consume it. However, your theme prints just awful with huge empty spaces between paragraphs and especially with some of your example code, i.e. see “Redirect All Feeds to Feedburner’s MyBrand”. Not sure if you care but it would really be great for those of us who print if you could clean it up for nicer printing to fully print your examples and to get rid of the excessive whitespace.

Go ahead and hit print preview to see how effective this simple CSS print method is!

Making AskApache Printer-Friendly

The first thing I did was to create a blank style sheet named apacheprint.css and then I added this XHTML to my

Amazingly, to make my site printer-friendly, all I had to do was edit the apacheprint.css file to control how browsers will print my site.

Resetting the CSS

Next I added the Yahoo Reset.css and Base.css files to my apacheprint.css file. That code looks like this.

html {color:#000; background:#FFF;}
body,div,dl,dt,dd,ul,ol,li,h1,h2,h3,h4,h5,h6,pre,code,form,fieldset,legend,input,textarea,p,blockquote,th,td {margin:0; padding:0;}
table {border-collapse:collapse; border-spacing:0;}
fieldset,img {border:0;}
address,caption,cite,code,dfn,em,strong,th,var {font-style:normal; font-weight:normal;}
li {list-style:none;}
caption,th {text-align:left;}
h1,h2,h3,h4,h5,h6 {font-size:100%; font-weight:normal;}
q:before,q:after {content:'';}
abbr,acronym {border:0; font-variant:normal;}
sup {vertical-align:text-top;}
sub {vertical-align:text-bottom;}
input,textarea,select {font-family:inherit; font-size:inherit; font-weight:inherit;}
input,textarea,select {*font-size:100%;}
legend {color:#000;}
code {display:inline;text-indent:3px;}
h1 {font-size:138.5%;}
h2 {font-size:123.1%;}
h3 {font-size:108%;}
h1,h2,h3 {margin:1em 0;}
h1,h2,h3,h4,h5,h6,strong {font-weight:bold;}
abbr,acronym {border-bottom:1px dotted #000; cursor:help;}
em {font-style:italic;}
blockquote,ul,ol,dl {margin:1em;}
ol,ul,dl {margin-left:2em;}
ol li {list-style:decimal outside;}
ul li {list-style:disc outside;}
dl dd {margin-left:1em;}
th,td {border:1px solid #000; padding:.5em;}
th {font-weight:bold; text-align:center;}
caption {margin-bottom:.5em; text-align:center;}
p,fieldset,table,pre {margin-bottom:1em;}
input[type=text],input[type=password],textarea {width:12.25em; *width:11.9em;}

Hiding Unfriendly Content

Ok so I don’t want to display the sidebar, header, footer, and several other elements on this site, so I added them to apacheprint.css with the instruction to not display them.

#HeadW,#HeadW,
#NavM,#HeadW,
#FootW,#FootW,
#simg,#simg,
#BTNfs,#BTNfs,
#sidebar,#sidebar,
#content #pagebar,#pagebar,
#content #digg,#digg,
#content #bcomme,#bcomme,
#content #comments,#comments,
#content #related1p,#related1p,
#content .rnote,
.flef,.flef,
#content #npl,#npl {display:none !important;}
 
#searchbox_002660089121042511758:kk7rwc2gx0i,#searchbox_002660089121042511758:kk7rwc2gx0i,
form#searchbox_002660089121042511758:kk7rwc2gx0i,form#searchbox_002660089121042511758:kk7rwc2gx0i,
#snaptalent,#snaptalent,
h2.HAC {display:none !important;}

Misc CSS Fixes

Once that was done I tested my site using the “print preview” browser feature, and found some other things I needed to fix.

#GlobalW,
#content,
#htaccess {width:auto !important; height:auto !important; overflow:visible !important; background:transparent !important; background-image:none !important; padding:0 !important; margin:0 !important; float:none !important;}
 
.item {width:auto !important; height:100% !important; overflow:visible !important; float:none !important;}
.post-content {width:auto !important; height:auto !important; float:none !important;}
.item .post-content {width:90% !important;padding:0 6% 0 0 !important;max-width:100% !important;margin:0 auto !important;}
 
.C {clear:both; padding:0; margin:0; line-height:5px; font-size:10px; border-bottom-width:0px;}
#content .post-content h2 {margin-top:1em;}
#content h1#sing {margin:0; width:100%; padding:0;}
.item {padding:0; margin:0;}
#content .commentlist li,#content .commentlist li.alt {margin:0.25em;}

Correct Page Breaks and Width

In order to print correctly, I also added the following.

html,body {background-color:#FFF; color:#000; font-size: 12pt;}
img  {max-width: 100%;}
h1,h2,h3,h4,h5,h6 {page-break-after: avoid;}
ul, ol, li {page-break-inside: avoid;}
table table,tr,td {page-break-before: avoid;page-break-after: avoid;}

Making Comments Appear on Separate Page

I created a class called .pb in my main css file

.pb { page-break-before: always; }

that creates a page break and then added that class to a hr element before my comment div.

Wrapping Code in PRE

I use

 tags to markup code in my posts, but printing doesn’t show you a scrollbar like my site does, so I added this fine CSS pre hack to wrap the lines when printing.
pre {page-break-inside: avoid; font-size: 7pt !important; max-width:95% !important; overflow-x: auto; /* Use horizontal scroller if needed; for Firefox 2, not needed in Firefox 3 */ white-space: pre-wrap; /* css-3 */ white-space: -moz-pre-wrap !important; /* Mozilla, since 1999 */ white-space: -pre-wrap; /* Opera 4-6 */ white-space: -o-pre-wrap; /* Opera 7 */ /* width: 99%; */ word-wrap: break-word; /* Internet Explorer 5.5+ */}

Displaying links for print
I opted not to do this because I have way to many links in my posts, but here is how I can display the links next to the link name in p and li tags.  Note that it will not print relative links or anchor links or javascript links.
.item p a:link:after,
.item p a:visited:after,
.item li a:link:after,
.item li a:visited:after { content: " (" attr(href) ") ";}
.item p a[href^="/"]:after,
.item li a[href^="/"]:after,
.item p a[href^="#"]:after,
.item p a[href^="javascript"]:after,
.item li a[href^="#"]:after,
.item li a[href^="javascript"]:after,
.item p a[href^="http://www.askapache"]:after,
.item p a[href^="http://www.askapache"]:after,
.item li a[href^="http://www.askapache"]:after,
.item li a[href^="http://www.askapache"]:after {content: "";}

Learn More about CSS Printing

Yahoo UI Library for Resetting CSS
A List Aparts: Going To Print
Wrapping Text Inside Pre Tags
A List Aparts: Taking Your Design to the Small Screen
Dev Operas: Making Small Devices Look Great
CSS-Tricks finally gets a Print Stylesheet
Advanced CSS Printing: Using Page Breaks
Opera user stylesheets
W3.org Paged Media

The Right to Read

AskApache — Fri, 20 Jun 2008 20:48:16 +0000

by Richard Stallman

This article appeared in the February 1997 issue of Communications of the ACM (Volume 40, Number 2).
(from "The Road To Tycho", a collection of articles about the antecedents of the Lunarian Revolution, published in Luna City in 2096)

For Dan Halbert, the road to Tycho began in college—when Lissa Lenz asked to borrow his computer. Hers had broken down, and unless she could borrow another, she would fail her midterm project. There was no one she dared ask, except Dan.

This put Dan in a dilemma. He had to help her—but if he lent her his computer, she might read his books. Aside from the fact that you could go to prison for many years for letting someone else read your books, the very idea shocked him at first. Like everyone, he had been taught since elementary school that sharing books was nasty and wrong—something that only pirates would do.

And there wasn’t much chance that the SPA—the Software Protection Authority—would fail to catch him. In his software class, Dan had learned that each book had a copyright monitor that reported when and where it was read, and by whom, to Central Licensing. (They used this information to catch reading pirates, but also to sell personal interest profiles to retailers.) The next time his computer was networked, Central Licensing would find out. He, as computer owner, would receive the harshest punishment—for not taking pains to prevent the crime.

Of course, Lissa did not necessarily intend to read his books. She might want the computer only to write her midterm. But Dan knew she came from a middle-class family and could hardly afford the tuition, let alone her reading fees. Reading his books might be the only way she could graduate. He understood this situation; he himself had had to borrow to pay for all the research papers he read. (10% of those fees went to the researchers who wrote the papers; since Dan aimed for an academic career, he could hope that his own research papers, if frequently referenced, would bring in enough to repay this loan.)

Later on, Dan would learn there was a time when anyone could go to the library and read journal articles, and even books, without having to pay. There were independent scholars who read thousands of pages without government library grants. But in the 1990s, both commercial and nonprofit journal publishers had begun charging fees for access. By 2047, libraries offering free public access to scholarly literature were a dim memory.

Debugging code is Illegal

There were ways, of course, to get around the SPA and Central Licensing. They were themselves illegal. Dan had had a classmate in software, Frank Martucci, who had obtained an illicit debugging tool, and used it to skip over the copyright monitor code when reading books. But he had told too many friends about it, and one of them turned him in to the SPA for a reward (students deep in debt were easily tempted into betrayal). In 2047, Frank was in prison, not for pirate reading, but for possessing a debugger.

Dan would later learn that there was a time when anyone could have debugging tools. There were even free debugging tools available on CD or downloadable over the net. But ordinary users started using them to bypass copyright monitors, and eventually a judge ruled that this had become their principal use in actual practice. This meant they were illegal; the debuggers’ developers were sent to prison.

Programmers still needed debugging tools, of course, but debugger vendors in 2047 distributed numbered copies only, and only to officially licensed and bonded programmers. The debugger Dan used in software class was kept behind a special firewall so that it could be used only for class exercises.

It was also possible to bypass the copyright monitors by installing a modified system kernel. Dan would eventually find out about the free kernels, even entire free operating systems, that had existed around the turn of the century. But not only were they illegal, like debuggers—you could not install one if you had one, without knowing your computer’s root password. And neither the FBI nor Microsoft Support would tell you that.

Dan concluded that he couldn’t simply lend Lissa his computer. But he couldn’t refuse to help her, because he loved her. Every chance to speak with her filled him with delight. And that she chose him to ask for help, that could mean she loved him too.

Dan resolved the dilemma by doing something even more unthinkable—he lent her the computer, and told her his password. This way, if Lissa read his books, Central Licensing would think he was reading them. It was still a crime, but the SPA would not automatically find out about it. They would only find out if Lissa reported him.

Of course, if the school ever found out that he had given Lissa his own password, it would be curtains for both of them as students, regardless of what she had used it for. School policy was that any interference with their means of monitoring students’ computer use was grounds for disciplinary action. It didn’t matter whether you did anything harmful—the offense was making it hard for the administrators to check on you. They assumed this meant you were doing something else forbidden, and they did not need to know what it was.

Banned from School Computer Systems

Students were not usually expelled for this—not directly. Instead they were banned from the school computer systems, and would inevitably fail all their classes.

Later, Dan would learn that this kind of university policy started only in the 1980s, when university students in large numbers began using computers. Previously, universities maintained a different approach to student discipline; they punished activities that were harmful, not those that merely raised suspicion.

Lissa did not report Dan to the SPA. His decision to help her led to their marriage, and also led them to question what they had been taught about piracy as children. The couple began reading about the history of copyright, about the Soviet Union and its restrictions on copying, and even the original United States Constitution. They moved to Luna, where they found others who had likewise gravitated away from the long arm of the SPA. When the Tycho Uprising began in 2062, the universal right to read soon became one of its central aims.

Author’s Note

This note was updated in 2007.

The right to read is a battle being fought today. Although it may take 50 years for our present way of life to fade into obscurity, most of the specific laws and practices described above have already been proposed; many have been enacted into law in the US and elsewhere. In the US, the 1998 Digital Millenium Copyright Act established the legal basis to restrict the reading and lending of computerized books (and other works as well). The European Union imposed similar restrictions in a 2001 copyright directive. In France, under the DADVSI law adopted in 2006, mere possession of a copy of DeCSS, the free program to decrypt video on a DVD, is a crime.

In 2001, Disney-funded Senator Hollings proposed a bill called the SSSCA that would require every new computer to have mandatory copy-restriction facilities that the user cannot bypass. Following the Clipper chip and similar US government key-escrow proposals, this shows a long-term trend: computer systems are increasingly set up to give absentees with clout control over the people actually using the computer system. The SSSCA was later renamed to the unpronounceable CBDTPA, which was glossed as the "Consume But Don’t Try Programming Act".

The Republicans took control of the US senate shortly thereafter. They are less tied to Hollywood than the Democrats, so they did not press these proposals. Now that the Democrats are back in control, the danger is once again higher.

In 2001 the US began attempting to use the proposed Free Trade Area of the Americas treaty to impose the same rules on all the countries in the Western Hemisphere. The FTAA is one of the so-called "free trade" treaties, which are actually designed to give business increased power over democratic governments; imposing laws like the DMCA is typical of this spirit. The FTAA was effectively killed by Lula, President of Brazil, who rejected the DMCA requirement and others.

Since then, the US has imposed similar requirements on countries such as Australia and Mexico through bilateral "free trade" agreements, and on countries such as Costa Rica through CAFTA. Ecuador’s President Correa refused to sign the "free trade" agreement, but Ecuador had adopted something like the DMCA in 2003. Ecuador’s new constitution may provide an opportunity to get rid of it.

One of the ideas in the story was not proposed in reality until 2002. This is the idea that the FBI and Microsoft will keep the root passwords for your personal computers, and not let you have them.

The proponents of this scheme have given it names such as "trusted computing" and "palladium". We call it "treacherous computing", because the effect is to make your computer obey companies instead of you. This was implemented in 2007 as part of Windows Vista; we expect Apple to do something similar. In this scheme, it is the manufacturer that keeps the secret code, but the FBI would have little trouble getting it.

What Microsoft keeps is not exactly a password in the traditional sense; no person ever types it on a terminal. Rather, it is a signature and encryption key that corresponds to a second key stored in your computer. This enables Microsoft, and potentially any web sites that cooperate with Microsoft, the ultimate control over what the user can do on his own computer.

Vista also gives Microsoft additional powers; for instance, Microsoft can forcibly install upgrades, and it can order all machines running Vista to refuse to run a certain device driver. The main purpose of Vista’s many restrictions is to make DRM that users can’t overcome.

The SPA, which actually stands for Software Publisher’s Association, has been replaced in this police-like role by the BSA or Business Software Alliance. It is not, today, an official police force; unofficially, it acts like one. Using methods reminiscent of the erstwhile Soviet Union, it invites people to inform on their coworkers and friends. A BSA terror campaign in Argentina in 2001 made slightly-veiled threats that people sharing software would be raped.

When this story was first written, the SPA was threatening small Internet service providers, demanding they permit the SPA to monitor all users. Most ISPs surrendered when threatened, because they cannot afford to fight back in court. (Atlanta Journal-Constitution, 1 Oct 96, D3.) At least one ISP, Community ConneXion in Oakland CA, refused the demand and was actually sued. The SPA later dropped the suit, but obtained the DMCA which gave them the power they sought.

The university security policies described above are not imaginary. For example, a computer at one Chicago-area university prints this message when you log in (quotation marks are in the original):

This system is for the use of authorized users only. Individuals using this computer system without authority or in the excess of their authority are subject to having all their activities on this system monitored and recorded by system personnel. In the course of monitoring individuals improperly using this system or in the course of system maintenance, the activities of authorized user may also be monitored. Anyone using this system expressly consents to such monitoring and is advised that if such monitoring reveals possible evidence of illegal activity or violation of University regulations system personnel may provide the evidence of such monitoring to University authorities and/or law enforcement officials.

This is an interesting approach to the Fourth Amendment: pressure most everyone to agree, in advance, to waive their rights under it.

This essay is published in Free Software, Free Society: The Selected Essays of Richard M. Stallman.

Automated Folder Backup Shell-Script

AskApache — Thu, 12 Jun 2008 05:22:12 +0000

This simple unix shell script automatically creates backups of a specific folder at regular hourly, nightly, weekly, and monthly intervals. DreamHost has this feature for their main accounts but for those of us running on the new Private Servers they haven’t set it up yet. The old perl script they use on the main accounts does not work with the higher security of the PS kernels and virtual server setup.

I just learned that using your dreamhost account for backups like this shell-script is a violation of the Terms of Service, so please don’t use this method on DreamHost.

Why Automated Backups

I make a lot of mistakes while developing and hacking code for my sites and I’ve come to rely on having access to these backup versions so I can quickly revert. Contacting support to get access to them is a waste of everyones time compared to finding a solution, so I did. The DreamHost wiki has a couple of complicated but workable solutions for backups but not for PS users, and nothing this simple. Just another great feature of DreamHosts debian linux based web hosting.

My Shell Script

Instead of the usual method for copying directory trees using tar with fifo, pipes, or DreamHost’s rsync and NFS method this script uses cpio which is much much faster and has a lot of cool options like saving m/a/c times and symlinks, and also being able to handle weird characters and file names.

mysnapshot.sh shell script

#!/bin/bash
#
# GNU Free Documentation License 1.2
# 06-11-08 - AskApache (www.askapache.com)
#
#
#    .mysnapshot
#    |-- hourly.0    (one hour ago)
#    |-- nightly.0   (one night ago)
#    |-- weekly.0    (one week ago)
#    `-- monthly.0   (one month ago)
#
 
### Source and Destination
source=$HOME/sites
dest=$HOME/.mysnapshot/${1:-hourly}.0/`basename $source`
 
### Make Nice - lower load
renice 19 -p $$ &>/dev/null
 
### Non-Absolute links, check source exists
cd $source || exit 1
 
### Hide errs, copy dirtree
find . -depth -print0 2>/dev/null | cpio -0admp $dest &>/dev/null
 
cd $OLDPWD
 
exit 0

mysnapshot.sh crontab entries

MAILTO=user@domain.com
# MY SNAPSHOTS
@hourly /home/user/scripts/mysnapshot.sh hourly &>/dev/null
@midnight /home/user/scripts/mysnapshot.sh nightly &>/dev/null
@weekly /home/user/scripts/mysnapshot.sh weekly &>/dev/null
@monthly /home/user/scripts/mysnapshot.sh monthly &>/dev/null

Just save the script to your server, chmod u+x, add the crontab entries, and you will have automated backups of any folder you want other than your HOME folder.

Stay tuned, I’m learning some really incredible stuff right now *if you run with open-source* and will be posting more tutorials soon about some really cool stuff.

AskApache.com May 2008 DreamHost Site of The Month

AskApache — Wed, 04 Jun 2008 02:54:06 +0000

It’s official. AskApache.com is the DreamHost Site of The Month for May, 2008, not only that…

This site is the highest rated DHSOTM winner of all time.

In Your Face Haters

It’s been exactly 1 year since this site won the same contest back in May 2007, but this time I did not link to the DHSOTM in either the DreamHost wiki which I have been illegally permanently banned from, or the DreamHost forum, which I have also been permanently banned from ;)

Unlike back in 2007 when I was disqualified by the same people who have now succeeded in banning me from the entire DreamHost community, this time they can’t say anything. Proving that no matter how much they badmouth this site DreamHost customers aren’t fooled by it, proof of which is their giving me a historic win for the 2nd time.

8.58 - Highest Rated Winner Ever!

DreamHost has been running this contest for a long time, and the high rating that you all gave to me has proved to be historic, making this site the highest rated website in the history of the DHSOTM.

Thanks to Everyone who Voted

I really appreciate eveyone voting for me, its been a tough year having to defend myself against the 2 self-important haters badmouthing me all over the net, and it feels good to have unbiased third-parties validate my belief that this blog is here for everyone with the goal of spreading free knowledge on the art of pimping out websites.

If anyone knows anyone at DreamHost, I’d appreciate it if you put in a good word for me to get my rewards account, wiki account, and forum account reactivated. But there is fierce opposition to this happening by 2 individuals, one of whom was just hired as an employee, so I’m not holding my breath :)

Fast and Easy Custom WordPress New User Registration

AskApache — Sun, 01 Jun 2008 02:30:01 +0000

Normally when you register on a WordPress blog you enter in a Username and Email to register.

Then a password is auto-generated and emailed to you to verify your email.

Finally you can login to the blog but if you don’t change your password right then you’ll have to check the email again later when you forget it.

Maybe you have a secured site and you want to make registration a fool-proof process for your peeps? Read on.

Easier WordPress Registration

Instead of the safer and more secure method employed by WP, this article shows you the code that lets you create your own register form like the one below, and registration is as simple as typing in an email address and password.

Here’s how this script works:

Enter Email Addy and Password (Email used as username)
Hitting Submit creates a new user, emails user login info to user, logs in the user, and redirects the user wherever.

Bad Idea to implement, Cool to think about

For one thing this would let web robots and spammers register for your blog without having to validate an email address. That could get very bad very fast in terms of comments and other data in your database. But there are probably a lot of reasons why this would be a very bad idea to actually implement.

Register Form Example XHTML

PHP Script autologin.php

This is pretty rough code but it works for WP 2.5, some things to note are that adequate checking of user-input is missing so a blank password will work. Another bit of roughness is how this script will DIE with an error message upon failure.

It will also send a new user notification email including the plaintext username and plaintext password, then it will login the user and redirect them to /wordpress/.

true,'user_login'=>$user_login,'user_password'=>$user_pass);
do_action_ref_array('wp_authenticate', array(&$credentials['user_login'], &$credentials['user_password']));
$user = wp_authenticate($credentials['user_login'], $credentials['user_password']);
wp_set_auth_cookie($user_id, $credentials['remember']);
do_action('wp_login', $credentials['user_login']);
wp_safe_redirect($redirect_to);
exit();
?>

Implementation

In case you want to try it out, it will work as is above.

This code is danger danger

Upgrading to DreamHost Private Servers

AskApache — Sat, 31 May 2008 21:14:27 +0000

I’ve recently relocated to the DreamHost private server setup, not for any reason other than its such a cool offer that I had to take advantage of it to learn more about it.

DreamHost Private Server

DreamHost PS uses Linux-VServer to give you your own “virtual machine”, thereby protecting your CPU and RAM from all other users on your physical machine.

Linux-VServer

Linux-VServer provides virtualization for GNU/Linux systems. This is accomplished by kernel level isolation. It allows to run multiple virtual units at once. Those units are sufficiently isolated to guarantee the required security, but utilize available resources efficiently, as they run on the same kernel. This particular virtual server model is implemented through a combination of “security contexts”, segmented routing, chroot, extended quotas and some other standard tools.

What Bothers Me about DreamHostPS

They don’t have the skill/desire to let you run only certain sites/usernames on the PS, its all or nothing, which completely sucks!
It’s so darn expensive, I spent more after the first 24 hours than I usually do in a MONTH
Although they say you still have the backups of your site, the .snapshot folder is no longer accessible and you have to manually contact support to ask for a backup! Ahh!
The environment on the shell doesn’t have as much access to good stuff like using locate
Processes can get out of control and use up to much memory if you have a busy site like me, which results in the sending of 503’s to everyone else!
Still uses NFS, the slowest thing in the world and not worth it now that backups aren’t even accessible.. it can also cause some hard-to-detect issues with caching setups

What Thrills me about DreamHostPS

I have more access and control over my server, sites, memory, and files via SSH login
I can scale the CPU and Memory whenever I want, and just like the static IP’s you only pay for the time you use, making experimenting affordable!
I can reboot the whole server at anytime! Sometimes useful if a script I’m experimenting with locks the whole server..
Only my shell accounts can access the server! Security is much stronger!
You can configure other webservers like lighttpd, nginx, and fnord to serve content on sites that are mostly static content!
I’ve only had it for 3 days, so I’m sure I’ll find alot more awesome capabilities

Problems migrating to DreamHostPS

Here are some issues I experienced during migration and the solutions I’ve used.

During this process it is important to note how helpful the DreamHost Support Staff were in putting up with my sometimes overly technical and detailed support requests. Thanks John, Brian, and Robert R!

PHP and HTTPD processes hogging all memory

Unlike on shared hosting accounts, where DH technical people have set up a very robust system, it appears they are missing the expertise of a past employee or something because this new setup is not as robust.. YET!

For instance I started out my account CPU and memory at the MAX (2300 MB / 2300 Mhz) but my sites were all still taking forever to serve content, simply because instead of on the shared servers where user processes and HTTPD instances are more controlled, this account seems to not have very well-thought out limits on it. So if 100 people asked for a page on my site, this server loads up 100 HTTPD processes under dhapache user and loads up 100 processes for the custom-compiled php.cgi I am running. This sounds like a cool thing but in reality it takes up so much of my memory that my bash shell login under SSH runs out of memory and won’t even let me do a simple ps, and it just keeps serving 503’s to anyone else who requests something on my site. DH will have to fix this soon or someone will launch a DDOS attack that will cripple them, unless a googlebot does it first!

Solution
I contacted support and received a very friendly and prompt reply suggesting a bad script and offering to setup a process watcher and killer, which I accepted. Eventually I located the problem to be an ErrorDocument 500 directive in my .htaccess that was pointing to a php file instead of a static .html

No crontabs or cronjobs

None of my crontab files were moved to my new server and in fact I was receiving permission denied just to access my crontab.

Solution
I contacted support and they installed new crontabs for me and offered to copy my old ones.

Static IP Changed for site with non-DreamHost DNS

One site uses DNS from Network Solutions, so when my site was migrated and got a new static IP address, my site went down. It would have been nice and should be expected that in this situation DreamHost would alert you that the change is going to happen so you can update your DNS without your site going offline.

Solution
Logged into my Network Solutions account and updated the DNS for my site to point to the new Static IP.

SSH Hosts, Authorized Keys Broken

Some of my sites and user accounts use passwordless SSH to make some things work, and all of these were made useless when I moved to my new private server.

Solution

Logged into my user accounts with SSH
Deleted the old files in folder .ssh
Created new keys and added them to other accounts
Logged in to new accounts to add to host files

Old Server and Static IP References in Site Files

I have some pretty technical and complex cgi’s, .htaccess files, shell scripts run by cronjobs, php scripts, etc., on some sites and shell accounts, and many of my files contain code to the Static IP and/or dreamhost server, either for access control or for faster connects by connecting straight to an IP instead of having to perform a DNS lookup. So when both the Static IP’s and dreamhost server changed it broke all my files.

Solution
Basically I knew I had to search all of my files and replace the old IP with the new IP. I also had to search files relacing the old server with my new server. To make life simpler, I wrote a simple shell script that I run from my account while logged in using SSH that does this automatically with the added feature of asking me if I would like to make the replacement for each file it finds, which is nice because I don’t want to replace this for old log files and misc stuff.

dreamhostps migration shell script

#!/bin/bash
# Version 1.0 by AskApache 5/29/2008
 
shopt -s extglob
renice 19 $$
 
OLDSERVER=208.113.183.103
NEWSERVER=208.113.134.190
 
FIXFILES=$(grep -R -l -i $OLDSERVER $HOME/!(Maildir|logs|backups|source|tmp|doit|php5|php526|ip_abuse) 2>/dev/null)
 
for thefile in ${FIXFILES[@]}; do
 if [ -f "$thefile" ]; then
  echo -e "\n\n\n\n"
  echo "___________________________________________________________________"
  echo "Name:  ${thefile}"
  echo "Type:  $(command file -b ${thefile})"
  echo "Size:  $(command du -hs ${thefile}|awk '{ print $1}')"
  echo "Matching Lines:"
  grep -i --color=auto $OLDSERVER $thefile
  echo -e "___________________________________________________________________\n"
  echo -en "Replace occurances of $OLDSERVER with $NEWSERVER? [y/N] " ; read -n 1 ans
  case "$ans" in
   n|N) echo -e "\nSKIPPING..."; ;;
   y|Y) echo -e "\nREPLACING..."
        cp $thefile $thefile.b1 &>/dev/null
        cat $thefile.b1 | sed "s/${OLDSERVER}/${NEWSERVER}/g" 1>$thefile
        rm $thefile.b1 &>/dev/null
        echo "DONE"; ;;
  esac
 fi
done
exit 0

Other Webservers Allowed on DreamHostPS

Lightweight web servers are Web servers which have been designed to run with very small resource overhead because of hardware, environment, or simply for the challenge of it.

Many of these systems have been created as a mental exercise to determine if a modern webserver could be written to run on limited resources such as those provided in a graphing calculator, an ancient Commodore 64 machine, or in 64 kB (64 KiB) total of memory. Others have been written as commercial endeavors to create webservers with low overhead for embedded systems (network router configuration pages) or low memory environments.

Apache webserver on DH is configured in process-based mode. This means that each process serves one simultaneous connection and uses significant amount of memory for that, thus limiting concurrent user count. You can download and install some lightweight web-servers like (lighttpd) or (nginx). These servers use async i/o and can handle large count of concurrent connections without consuming much RAM, especially when serving static files. But default port 80 is already occupied by apache, and you cannot change that. So, you can use another port for new lightweight server, but visitors will see ugly port number in address and such solution will not work for some corporate firewalls.

lighttpd - small memory footprint compared to other web-servers, effective management of the cpu-load, and advanced feature set
- FastCGI
- SCGI
- Auth
- Output-Compression
- URL-Rewriting
- many more
nginx - lightweight web server/reverse proxy and e-mail (IMAP/POP3) proxy
- Handling of static files, index files and auto-indexing
- Reverse proxy without caching, load balancing, and fault tolerance
- SSL support
- FastCGI support
- Name- and IP-based virtual servers
- FLV streaming
fnord - lightweight webserver written by Felix von Leitner. It aims to be a small, but extremely fast and secure webserver.
- Small! (13k static Linux-x86 binary without CGI, 18k with CGI)
- Fast! Uses mmap (and on Linux, sendfile)
- connection keep-alive
- el-cheapo virtual domains (similar to thttpd)
- IPv6 support (through tcpserver)
- CGI (through pipes, not temp files like Apache)
- Content-Range (not the full specs, just a-b or a- byte ranges)
- transparent content negotiation
- Directory index generation

DreamHost PS Troubleshooting

Symptoms of an Overloaded DreamHost PS
Checking your Daily & Monthly Resource Usage
Changing your CPU & Memory Allocation
Rebooting your DreamHost PS
Overloaded DreamHost PS
Idle DreamHost PS
Changing web server
See also
External link

Symptoms of an Overloaded DreamHost PS

If you experience any of the following symptoms, most likely you will need to increase the resource allocation (CPU & memory) to your Private Server:

Out of memory
Internal server errors ("500" errors)
Killed scripts
Inability to log in ("ssh_exchange_identification: Connection closed by remote host")

Checking your Daily & Monthly Resource Usage

Check the daily & monthly usage graph in the Control Panel under (PRIVATE SERVERS > CPU/MEMORY) to see if your usage is going beyond your currently guaranteed CPU and memory allocation.

Changing your CPU & Memory Allocation

You can increase or decrease your resource allocation by moving the green slider in the Control Panel under (PRIVATE SERVERS > CPU/MEMORY). As you move it the amount of CPU and memory will update automatically, along with the rate you’ll be charged for that setting. Once you’re happy with the setting click on the "Change {servername}’s CPU / Memory Now!" button to push the change into place. It will take a short period of time for the setting change to be reflected. Typically no reboot is necessary.

Rebooting your DreamHost PS

If you have a problem you can try rebooting your Private Server yourself in the Control Panel under (PRIVATE SERVERS > REBOOT SERVER). No need to contact support for that! Isn’t that cool?! It may temporarily fix things, but if it doesn’t provide a long-term solution, you will need to increase the resources allocated to your Private Server by using the green slider as mentioned in the previous paragraph.

Overloaded DreamHost PS

The image above is what your daily usage graph might look if you’ve got your settings too low. Notice that the actual usage routinely exceeds the guaranteed amount of 150MHz/MB. This is bound to cause problems for your sites. To rectify the situation it’s recommended that you increase your resource allocation. Then keep an eye on the graph for several hours, and test our sites to see if you’ve allocated enough resources for things to run smoothly. It is recommended that you start out by doubling your current resource allocation to see if it’s enough. Once you’ve verified that things are running properly you can reduce your resource allocation to the point where your peaks just barely exceed what you have allocated. Of course you’ll want to routinely monitor your usage and increase the resource allocation as your needs increase. It’s best to over allocate then under allocate! You don’t want to find out that you’ve under allocated by your visitors/customers complaining about your sites not working properly.

Note that you’re only charged for the period of time that you have the slider in a particular position. So it’s safe to experiment. In fact, we recommend it. You can increase or decrease your resource allocation at any time.

You will typically see Apache processes running on your server and appearing to consume all of the memory. This is generally not the case because Apache processes share a significant amount of memory between one another. Additionally, we automatically configure Apache to work well within the memory allocation of your PS server. It is still possible for a busy website to overwhelm a DreamHost PS server, but it is not generally the fault of the apache webserver itself.

Of course you may also want to try to reduce your load on the server as well so you can reduce the resource allocation and save some money.

Idle DreamHost PS

The image above is what your daily usage graph might look if you’ve got nothing running under your DreamHost PS. This just shows the overhead resource usage. You could host a lightly loaded web page with this resource allocation (provided it uses static content), but probably not much more. If you’ve got your resource allocation set to 150MHz/MB it is recommendended that you monitor the usage very often.

Some Content on this page included from this article by Author History from the DreamHost Wiki and is licensed under the GNU FDL.

Adding Akismet Anti-Spam Protection Anywhere

AskApache — Thu, 08 May 2008 03:14:03 +0000

Akismet is well-known as THE anti-spam plugin for WordPress that checks every comment and trackback received by your blog for spam. Wouldn’t you love to have that same kind of anti-spam protection for non-wordpress forms and pages?

Add Akismet to Any Form

This article shows you how to add this same type of anti-spam protection to any php form. It will work for any contact forms, surveys, login forms, etc.

Adding Akismet to PHP Form

First you will need to download the Micro-Akismet PHP class by Gaby Vanhegan, and add that to your server so it can be included by php like this.

include_once("class.microakismet.inc.php");

Setup Akismet Class

After you have included the class in the php file of the form you want to protect you need to activate the akismet class like so. akey is your akismet key, apage is the page the form is on, aver is your site and 1.0

$akey='4a5a26db1c';
$apage='http://www.askapache.com/about/contact/';
$aver='askapache.com/1.0';
$akismet = new MicroAkismet( $akey, $apage, $aver );

Provide relevant data to akismet

Now you need to setup an array called vars that has any information about the form submission and the user who submitted the form.

$vars = array();
foreach(array_keys($_SERVER) as $skey){
  if((substr($skey, 0, 5) == "HTTP_") && !empty($_SERVER[$skey]))
   $vars[str_replace('HTTP_','',$skey)]=$_SERVER[$skey];
}
 
$vars["user_ip"]               = $_SERVER["REMOTE_ADDR"];
$vars["user_agent"]            = $_SERVER["HTTP_USER_AGENT"];
$vars["comment_content"]       = $_POST["Message"];
$vars["comment_author"]      = $_POST["FullName"];
$vars["comment_author_email"]  = $_POST["Email"];
$vars["comment_type"]      = 'comment';
$vars['permalink']        = 'http://www.askapache.com'.$_SERVER['REQUEST_URI'];
$vars['referrer']        = $_SERVER['HTTP_REFERER'];
$vars['phone_number']      = $_POST['CallNumber'];
$vars['organization']      = $_POST['Organization'];

Checking for spam with Akismet

Now add this to your php. If the message is spam it will send mail with [SPAM] in the subject line, otherwise it will send mail normally.

if($akismet->check( $vars ))send_mail("[SPAM] Contact");
else send_mail("Contact");

More about Akismet API

About the Akismet Service

Akismet is basically a big machine that sucks up all the data it possibly can, looks for patterns, and learns from its mistakes. Thus far it has been highly effective at stopping spam and adapting to new techniques and attempts to evade it, and time will tell how it stands up. I’ve tried to keep the API interaction as simple as possible.

A Good Consumer

To interact fully with the Akismet API your program really should be putting data back into the system as well as just taking it out. If it is at all possible within the framework of your application you should have a way for your users to submit missed spam and false positives, otherwise Akismet will never learn from its mistakes.