AskApache » Search Results » shell

Chmod, Umask, Stat, Fileperms, and File Permissions

AskApache — Fri, 17 Feb 2012 11:16:56 +0000

Unix file permissions are one of the more difficult subjects to grasp.. Well, ok maybe "grasp" isn't the word.. Master is the right word.. Unix file permissions is a hard topic to fully master, mainly I think because there aren't many instances when a computer user encounters them. Windows has been trying to figure it out for decades with little progress, so don't feel bad if you don't know much about it. Unless you're with the program and running Mac or any other BSD/Unix based OS you've never had the ability to secure your system in this most basic and fundamental way. Usually the first time someone encounters file permissions it's because their website was cracked..

.htaccess

$ chmod 604 .htaccess

604 -rw----r--  /home/askapache/cgi-bin/.htaccess

php.cgi

$ chmod 711 php.cgi

$ 711 -rwx--x--x  /home/askapache/cgi-bin/php.cgi

.php.ini

$ chmod 600 php.ini

$ 600 -rw-------  /home/askapache/cgi-bin/php.ini

I'm in the process of developing an updated version of the .htaccess security plugin, and one thing I have been working on is file permissions. Some people had problems trying to create files on their server and I realized it was bad programming on my part.. so I began researching permissions in detail. I went deep into the source code of Apache (which is why this site is called AskApache, BTW), PHP, Python, Ocaml, Perl, Ruby, and POSIX operating systems and got a pretty good handle on it now..

Tips before we dig in

Here's a few things I've learned that I didn't know before (using php).

Deleting Files and Directories

Deleting a file may require chmodding the file to 666 or even 777 before you are able to delete it. You also might have to chmod the parent directory of the file as well. Also, you may have to chdir to the directory the file is in. And lastly you may have to change the owner or group of the file. Further than that you can try renaming the file first then deleting it..

Deleting a directory means you need to remove every file in it first. It needs to be empty. And if your file system uses NFS or some other networked FS you might have even more problems deleting files. If the file you are trying to delete is being used by say, Apache or php then you might have to kill that process first.

Creating Files in Restrictive Environments

My research has been geared to try and make my code as robust as possible, I'm throwing everything but the kitchen sink into some of these functions because so many people are on such different types of servers. To create a file in a restrictive environment is a fun excercise to take.. You can write a file using many different functions, but there are some tricks if they all fail. One trick is instead of trying to "write" the data to the file, you can UPLOAD the data to the server and let PHP handle the file as if you used an upload form. I like to use fsockopen to do it, as some installations have been setup to prevent this type of fake upload.

Then there are the various other hacks like using an ftp connection (if you know the user/pass) to send the file from php, using ssh from php, whatever is available on the hosts php installation. In addition to those more involved workarounds you can often get around this problem by doing little hacks discussed at php.net in the comments for various functions. Such as changing the umask, changing directories with chdir first, creating a temporary file using a function like tempfile and then renaming or copying the tempfile to your desired file which sometimes gives you the permissions needed to write to the location.

If the php installation is newer than you can also look into creating your own stream context to pass write the data direct.

Stat Function

I've created a stat function in php that goes farther than the normal stat function... Just give the function a file to stat, and it returns an array of information.

function askapache_stat($filename) {
 clearstatcache();
 $ss=@stat($filename);
 if(!$ss) die("Couldnt stat {$filename}");
 $file_convert=array(0140000=>'ssocket',0120000=>'llink',0100000=>'-file',0060000=>'bblock',0040000=>'ddir',0020000=>'cchar',0010000=>'pfifo');
 $p=$ss['mode'];
 $t=decoct($ss['mode'] & 0170000);
 $str = (array_key_exists(octdec($t),$file_convert)) ? $file_convert[octdec($t)]{0} : 'u';
 $str.=(($p&0x0100)?'r':'-').(($p&0x0080)?'w':'-').(($p&0x0040)?(($p&0x0800)?'s':'x'):(($p&0x0800)?'S':'-'));
 $str.=(($p&0x0020)?'r':'-').(($p&0x0010)?'w':'-').(($p&0x0008)?(($p&0x0400)?'s':'x'):(($p&0x0400)?'S':'-'));
 $str.=(($p&0x0004)?'r':'-').(($p&0x0002)?'w':'-').(($p&0x0001)?(($p&0x0200)?'t':'x'):(($p&0x0200)?'T':'-'));
 
 $s=array(
 'perms'=>array(
  'umask'=>sprintf("%04o",umask()),
  'human'=>$str,
  'octal1'=>sprintf("%o", ($ss['mode'] & 000777)),
  'octal2'=>sprintf("0%o", 0777 & $p),
  'decimal'=>sprintf("%04o", $p),
  'fileperms'=>@fileperms($filename),
  'mode1'=>$p,
  'mode2'=>$ss['mode']),
 
 'filetype'=>array(
  'type'=>substr($file_convert[octdec($t)],1),
  'type_octal'=>sprintf("%07o", octdec($t)),
  'is_file'=>@is_file($filename),
  'is_dir'=>@is_dir($filename),
  'is_link'=>@is_link($filename),
  'is_readable'=> @is_readable($filename),
  'is_writable'=> @is_writable($filename)),
 
 'owner'=>array(
  'fileowner'=>$ss['uid'],
  'filegroup'=>$ss['gid'],
  'owner_name'=>(function_exists('posix_getpwuid')) ? @reset(@posix_getpwuid($ss['uid'])) : '',
  'group_name'=>(function_exists('posix_getgrgid')) ? @reset(@posix_getgrgid($ss['gid'])) : ''),
 
 'file'=>array(
  'filename'=>$filename,
  'realpath'=>(@realpath($filename) != $filename) ? @realpath($filename) : '',
  'dirname'=>@dirname($filename),
  'basename'=>@basename($filename)),
 
 'device'=>array(
  'device'=>$ss['dev'], //Device
  'device_number'=>$ss['rdev'], //Device number, if device.
  'inode'=>$ss['ino'], //File serial number
  'link_count'=>$ss['nlink'], //link count
  'link_to'=>($s['type']=='link') ? @readlink($filename) : ''),
 
 'size'=>array(
  'size'=>$ss['size'], //Size of file, in bytes.
  'blocks'=>$ss['blocks'], //Number 512-byte blocks allocated
  'block_size'=> $ss['blksize']), //Optimal block size for I/O.
 
 'time'=>array(
  'mtime'=>$ss['mtime'], //Time of last modification
  'atime'=>$ss['atime'], //Time of last access.
  'ctime'=>$ss['ctime'], //Time of last status change
  'accessed'=>@date('Y M D H:i:s',$ss['atime']),
  'modified'=>@date('Y M D H:i:s',$ss['mtime']),
  'created'=>@date('Y M D H:i:s',$ss['ctime'])),
 );
 
 clearstatcache();
 return $s;
}

PHP Stat Function Output

Example output, say from print_r(askapache_stat( __FILE__ ) );

Array(
[perms] => Array
  (
  [umask] => 0022
  [human] => -rw-r--r--
  [octal1] => 644
  [octal2] => 0644
  [decimal] => 100644
  [fileperms] => 33188
  [mode1] => 33188
  [mode2] => 33188
  )
 
[filetype] => Array
  (
  [type] => file
  [type_octal] => 0100000
  [is_file] => 1
  [is_dir] =>
  [is_link] =>
  [is_readable] => 1
  [is_writable] => 1
  )
 
[owner] => Array
  (
  [fileowner] => 035483
  [filegroup] => 23472
  [owner_name] => askapache
  [group_name] => grp22558
  )
 
[file] => Array
  (
  [filename] => /home/askapache/askapache-stat/public_html/ok/g.php
  [realpath] =>
  [dirname] => /home/askapache/askapache-stat/public_html/ok
  [basename] => g.php
  )
 
[device] => Array
  (
  [device] => 25
  [device_number] => 0
  [inode] => 92455020
  [link_count] => 1
  [link_to] =>
  )
 
[size] => Array
  (
  [size] => 2652
  [blocks] => 8
  [block_size] => 8192
  )
 
[time] => Array
  (
  [mtime] => 1227685253
  [atime] => 1227685138
  [ctime] => 1227685253
  [accessed] => 2008 Nov Tue 23:38:58
  [modified] => 2008 Nov Tue 23:40:53
  [created] => 2008 Nov Tue 23:40:53
  )
)

Every Permission 0000 to 0777

This shows what each numeric permission does to a REGULAR file. I'll provide the code to do this below so you can do the same thing on your server.

chmod 0: ----------
chmod 1: ---------x
chmod 2: --------w-
chmod 3: --------wx
chmod 4: -------r--
chmod 5: -------r-x
chmod 6: -------rw-
chmod 7: -------rwx
chmod 10: ------x---
chmod 11: ------x--x
chmod 12: ------x-w-
chmod 13: ------x-wx
chmod 14: ------xr--
chmod 15: ------xr-x
chmod 16: ------xrw-
chmod 17: ------xrwx
chmod 20: -----w----
chmod 21: -----w---x
chmod 22: -----w--w-
chmod 23: -----w--wx
chmod 24: -----w-r--
chmod 25: -----w-r-x
chmod 26: -----w-rw-
chmod 27: -----w-rwx
chmod 30: -----wx---
chmod 31: -----wx--x
chmod 32: -----wx-w-
chmod 33: -----wx-wx
chmod 34: -----wxr--
chmod 35: -----wxr-x
chmod 36: -----wxrw-
chmod 37: -----wxrwx
chmod 40: ----r-----
chmod 41: ----r----x
chmod 42: ----r---w-
chmod 43: ----r---wx
chmod 44: ----r--r--
chmod 45: ----r--r-x
chmod 46: ----r--rw-
chmod 47: ----r--rwx
chmod 50: ----r-x---
chmod 51: ----r-x--x
chmod 52: ----r-x-w-
chmod 53: ----r-x-wx
chmod 54: ----r-xr--
chmod 55: ----r-xr-x
chmod 56: ----r-xrw-
chmod 57: ----r-xrwx
chmod 60: ----rw----
chmod 61: ----rw---x
chmod 62: ----rw--w-
chmod 63: ----rw--wx
chmod 64: ----rw-r--
chmod 65: ----rw-r-x
chmod 66: ----rw-rw-
chmod 67: ----rw-rwx
chmod 70: ----rwx---
chmod 71: ----rwx--x
chmod 72: ----rwx-w-
chmod 73: ----rwx-wx
chmod 74: ----rwxr--
chmod 75: ----rwxr-x
chmod 76: ----rwxrw-
chmod 77: ----rwxrwx
chmod 100: ---x------
chmod 101: ---x-----x
chmod 102: ---x----w-
chmod 103: ---x----wx
chmod 104: ---x---r--
chmod 105: ---x---r-x
chmod 106: ---x---rw-
chmod 107: ---x---rwx
chmod 110: ---x--x---
chmod 111: ---x--x--x
chmod 112: ---x--x-w-
chmod 113: ---x--x-wx
chmod 114: ---x--xr--
chmod 115: ---x--xr-x
chmod 116: ---x--xrw-
chmod 117: ---x--xrwx
chmod 120: ---x-w----
chmod 121: ---x-w---x
chmod 122: ---x-w--w-
chmod 123: ---x-w--wx
chmod 124: ---x-w-r--
chmod 125: ---x-w-r-x
chmod 126: ---x-w-rw-
chmod 127: ---x-w-rwx
chmod 130: ---x-wx---
chmod 131: ---x-wx--x
chmod 132: ---x-wx-w-
chmod 133: ---x-wx-wx
chmod 134: ---x-wxr--
chmod 135: ---x-wxr-x
chmod 136: ---x-wxrw-
chmod 137: ---x-wxrwx
chmod 140: ---xr-----
chmod 141: ---xr----x
chmod 142: ---xr---w-
chmod 143: ---xr---wx
chmod 144: ---xr--r--
chmod 145: ---xr--r-x
chmod 146: ---xr--rw-
chmod 147: ---xr--rwx
chmod 150: ---xr-x---
chmod 151: ---xr-x--x
chmod 152: ---xr-x-w-
chmod 153: ---xr-x-wx
chmod 154: ---xr-xr--
chmod 155: ---xr-xr-x
chmod 156: ---xr-xrw-
chmod 157: ---xr-xrwx
chmod 160: ---xrw----
chmod 161: ---xrw---x
chmod 162: ---xrw--w-
chmod 163: ---xrw--wx
chmod 164: ---xrw-r--
chmod 165: ---xrw-r-x
chmod 166: ---xrw-rw-
chmod 167: ---xrw-rwx
chmod 170: ---xrwx---
chmod 171: ---xrwx--x
chmod 172: ---xrwx-w-
chmod 173: ---xrwx-wx
chmod 174: ---xrwxr--
chmod 175: ---xrwxr-x
chmod 176: ---xrwxrw-
chmod 177: ---xrwxrwx
chmod 200: --w-------
chmod 201: --w------x
chmod 202: --w-----w-
chmod 203: --w-----wx
chmod 204: --w----r--
chmod 205: --w----r-x
chmod 206: --w----rw-
chmod 207: --w----rwx
chmod 210: --w---x---
chmod 211: --w---x--x
chmod 212: --w---x-w-
chmod 213: --w---x-wx
chmod 214: --w---xr--
chmod 215: --w---xr-x
chmod 216: --w---xrw-
chmod 217: --w---xrwx
chmod 220: --w--w----
chmod 221: --w--w---x
chmod 222: --w--w--w-
chmod 223: --w--w--wx
chmod 224: --w--w-r--
chmod 225: --w--w-r-x
chmod 226: --w--w-rw-
chmod 227: --w--w-rwx
chmod 230: --w--wx---
chmod 231: --w--wx--x
chmod 232: --w--wx-w-
chmod 233: --w--wx-wx
chmod 234: --w--wxr--
chmod 235: --w--wxr-x
chmod 236: --w--wxrw-
chmod 237: --w--wxrwx
chmod 240: --w-r-----
chmod 241: --w-r----x
chmod 242: --w-r---w-
chmod 243: --w-r---wx
chmod 244: --w-r--r--
chmod 245: --w-r--r-x
chmod 246: --w-r--rw-
chmod 247: --w-r--rwx
chmod 250: --w-r-x---
chmod 251: --w-r-x--x
chmod 252: --w-r-x-w-
chmod 253: --w-r-x-wx
chmod 254: --w-r-xr--
chmod 255: --w-r-xr-x
chmod 256: --w-r-xrw-
chmod 257: --w-r-xrwx
chmod 260: --w-rw----
chmod 261: --w-rw---x
chmod 262: --w-rw--w-
chmod 263: --w-rw--wx
chmod 264: --w-rw-r--
chmod 265: --w-rw-r-x
chmod 266: --w-rw-rw-
chmod 267: --w-rw-rwx
chmod 270: --w-rwx---
chmod 271: --w-rwx--x
chmod 272: --w-rwx-w-
chmod 273: --w-rwx-wx
chmod 274: --w-rwxr--
chmod 275: --w-rwxr-x
chmod 276: --w-rwxrw-
chmod 277: --w-rwxrwx
chmod 300: --wx------
chmod 301: --wx-----x
chmod 302: --wx----w-
chmod 303: --wx----wx
chmod 304: --wx---r--
chmod 305: --wx---r-x
chmod 306: --wx---rw-
chmod 307: --wx---rwx
chmod 310: --wx--x---
chmod 311: --wx--x--x
chmod 312: --wx--x-w-
chmod 313: --wx--x-wx
chmod 314: --wx--xr--
chmod 315: --wx--xr-x
chmod 316: --wx--xrw-
chmod 317: --wx--xrwx
chmod 320: --wx-w----
chmod 321: --wx-w---x
chmod 322: --wx-w--w-
chmod 323: --wx-w--wx
chmod 324: --wx-w-r--
chmod 325: --wx-w-r-x
chmod 326: --wx-w-rw-
chmod 327: --wx-w-rwx
chmod 330: --wx-wx---
chmod 331: --wx-wx--x
chmod 332: --wx-wx-w-
chmod 333: --wx-wx-wx
chmod 334: --wx-wxr--
chmod 335: --wx-wxr-x
chmod 336: --wx-wxrw-
chmod 337: --wx-wxrwx
chmod 340: --wxr-----
chmod 341: --wxr----x
chmod 342: --wxr---w-
chmod 343: --wxr---wx
chmod 344: --wxr--r--
chmod 345: --wxr--r-x
chmod 346: --wxr--rw-
chmod 347: --wxr--rwx
chmod 350: --wxr-x---
chmod 351: --wxr-x--x
chmod 352: --wxr-x-w-
chmod 353: --wxr-x-wx
chmod 354: --wxr-xr--
chmod 355: --wxr-xr-x
chmod 356: --wxr-xrw-
chmod 357: --wxr-xrwx
chmod 360: --wxrw----
chmod 361: --wxrw---x
chmod 362: --wxrw--w-
chmod 363: --wxrw--wx
chmod 364: --wxrw-r--
chmod 365: --wxrw-r-x
chmod 366: --wxrw-rw-
chmod 367: --wxrw-rwx
chmod 370: --wxrwx---
chmod 371: --wxrwx--x
chmod 372: --wxrwx-w-
chmod 373: --wxrwx-wx
chmod 374: --wxrwxr--
chmod 375: --wxrwxr-x
chmod 376: --wxrwxrw-
chmod 377: --wxrwxrwx
chmod 400: -r--------
chmod 401: -r-------x
chmod 402: -r------w-
chmod 403: -r------wx
chmod 404: -r-----r--
chmod 405: -r-----r-x
chmod 406: -r-----rw-
chmod 407: -r-----rwx
chmod 410: -r----x---
chmod 411: -r----x--x
chmod 412: -r----x-w-
chmod 413: -r----x-wx
chmod 414: -r----xr--
chmod 415: -r----xr-x
chmod 416: -r----xrw-
chmod 417: -r----xrwx
chmod 420: -r---w----
chmod 421: -r---w---x
chmod 422: -r---w--w-
chmod 423: -r---w--wx
chmod 424: -r---w-r--
chmod 425: -r---w-r-x
chmod 426: -r---w-rw-
chmod 427: -r---w-rwx
chmod 430: -r---wx---
chmod 431: -r---wx--x
chmod 432: -r---wx-w-
chmod 433: -r---wx-wx
chmod 434: -r---wxr--
chmod 435: -r---wxr-x
chmod 436: -r---wxrw-
chmod 437: -r---wxrwx
chmod 440: -r--r-----
chmod 441: -r--r----x
chmod 442: -r--r---w-
chmod 443: -r--r---wx
chmod 444: -r--r--r--
chmod 445: -r--r--r-x
chmod 446: -r--r--rw-
chmod 447: -r--r--rwx
chmod 450: -r--r-x---
chmod 451: -r--r-x--x
chmod 452: -r--r-x-w-
chmod 453: -r--r-x-wx
chmod 454: -r--r-xr--
chmod 455: -r--r-xr-x
chmod 456: -r--r-xrw-
chmod 457: -r--r-xrwx
chmod 460: -r--rw----
chmod 461: -r--rw---x
chmod 462: -r--rw--w-
chmod 463: -r--rw--wx
chmod 464: -r--rw-r--
chmod 465: -r--rw-r-x
chmod 466: -r--rw-rw-
chmod 467: -r--rw-rwx
chmod 470: -r--rwx---
chmod 471: -r--rwx--x
chmod 472: -r--rwx-w-
chmod 473: -r--rwx-wx
chmod 474: -r--rwxr--
chmod 475: -r--rwxr-x
chmod 476: -r--rwxrw-
chmod 477: -r--rwxrwx
chmod 500: -r-x------
chmod 501: -r-x-----x
chmod 502: -r-x----w-
chmod 503: -r-x----wx
chmod 504: -r-x---r--
chmod 505: -r-x---r-x
chmod 506: -r-x---rw-
chmod 507: -r-x---rwx
chmod 510: -r-x--x---
chmod 511: -r-x--x--x
chmod 512: -r-x--x-w-
chmod 513: -r-x--x-wx
chmod 514: -r-x--xr--
chmod 515: -r-x--xr-x
chmod 516: -r-x--xrw-
chmod 517: -r-x--xrwx
chmod 520: -r-x-w----
chmod 521: -r-x-w---x
chmod 522: -r-x-w--w-
chmod 523: -r-x-w--wx
chmod 524: -r-x-w-r--
chmod 525: -r-x-w-r-x
chmod 526: -r-x-w-rw-
chmod 527: -r-x-w-rwx
chmod 530: -r-x-wx---
chmod 531: -r-x-wx--x
chmod 532: -r-x-wx-w-
chmod 533: -r-x-wx-wx
chmod 534: -r-x-wxr--
chmod 535: -r-x-wxr-x
chmod 536: -r-x-wxrw-
chmod 537: -r-x-wxrwx
chmod 540: -r-xr-----
chmod 541: -r-xr----x
chmod 542: -r-xr---w-
chmod 543: -r-xr---wx
chmod 544: -r-xr--r--
chmod 545: -r-xr--r-x
chmod 546: -r-xr--rw-
chmod 547: -r-xr--rwx
chmod 550: -r-xr-x---
chmod 551: -r-xr-x--x
chmod 552: -r-xr-x-w-
chmod 553: -r-xr-x-wx
chmod 554: -r-xr-xr--
chmod 555: -r-xr-xr-x
chmod 556: -r-xr-xrw-
chmod 557: -r-xr-xrwx
chmod 560: -r-xrw----
chmod 561: -r-xrw---x
chmod 562: -r-xrw--w-
chmod 563: -r-xrw--wx
chmod 564: -r-xrw-r--
chmod 565: -r-xrw-r-x
chmod 566: -r-xrw-rw-
chmod 567: -r-xrw-rwx
chmod 570: -r-xrwx---
chmod 571: -r-xrwx--x
chmod 572: -r-xrwx-w-
chmod 573: -r-xrwx-wx
chmod 574: -r-xrwxr--
chmod 575: -r-xrwxr-x
chmod 576: -r-xrwxrw-
chmod 577: -r-xrwxrwx
chmod 600: -rw-------
chmod 601: -rw------x
chmod 602: -rw-----w-
chmod 603: -rw-----wx
chmod 604: -rw----r--
chmod 605: -rw----r-x
chmod 606: -rw----rw-
chmod 607: -rw----rwx
chmod 610: -rw---x---
chmod 611: -rw---x--x
chmod 612: -rw---x-w-
chmod 613: -rw---x-wx
chmod 614: -rw---xr--
chmod 615: -rw---xr-x
chmod 616: -rw---xrw-
chmod 617: -rw---xrwx
chmod 620: -rw--w----
chmod 621: -rw--w---x
chmod 622: -rw--w--w-
chmod 623: -rw--w--wx
chmod 624: -rw--w-r--
chmod 625: -rw--w-r-x
chmod 626: -rw--w-rw-
chmod 627: -rw--w-rwx
chmod 630: -rw--wx---
chmod 631: -rw--wx--x
chmod 632: -rw--wx-w-
chmod 633: -rw--wx-wx
chmod 634: -rw--wxr--
chmod 635: -rw--wxr-x
chmod 636: -rw--wxrw-
chmod 637: -rw--wxrwx
chmod 640: -rw-r-----
chmod 641: -rw-r----x
chmod 642: -rw-r---w-
chmod 643: -rw-r---wx
chmod 644: -rw-r--r--
chmod 645: -rw-r--r-x
chmod 646: -rw-r--rw-
chmod 647: -rw-r--rwx
chmod 650: -rw-r-x---
chmod 651: -rw-r-x--x
chmod 652: -rw-r-x-w-
chmod 653: -rw-r-x-wx
chmod 654: -rw-r-xr--
chmod 655: -rw-r-xr-x
chmod 656: -rw-r-xrw-
chmod 657: -rw-r-xrwx
chmod 660: -rw-rw----
chmod 661: -rw-rw---x
chmod 662: -rw-rw--w-
chmod 663: -rw-rw--wx
chmod 664: -rw-rw-r--
chmod 665: -rw-rw-r-x
chmod 666: -rw-rw-rw-
chmod 667: -rw-rw-rwx
chmod 670: -rw-rwx---
chmod 671: -rw-rwx--x
chmod 672: -rw-rwx-w-
chmod 673: -rw-rwx-wx
chmod 674: -rw-rwxr--
chmod 675: -rw-rwxr-x
chmod 676: -rw-rwxrw-
chmod 677: -rw-rwxrwx
chmod 700: -rwx------
chmod 701: -rwx-----x
chmod 702: -rwx----w-
chmod 703: -rwx----wx
chmod 704: -rwx---r--
chmod 705: -rwx---r-x
chmod 706: -rwx---rw-
chmod 707: -rwx---rwx
chmod 710: -rwx--x---
chmod 711: -rwx--x--x
chmod 712: -rwx--x-w-
chmod 713: -rwx--x-wx
chmod 714: -rwx--xr--
chmod 715: -rwx--xr-x
chmod 716: -rwx--xrw-
chmod 717: -rwx--xrwx
chmod 720: -rwx-w----
chmod 721: -rwx-w---x
chmod 722: -rwx-w--w-
chmod 723: -rwx-w--wx
chmod 724: -rwx-w-r--
chmod 725: -rwx-w-r-x
chmod 726: -rwx-w-rw-
chmod 727: -rwx-w-rwx
chmod 730: -rwx-wx---
chmod 731: -rwx-wx--x
chmod 732: -rwx-wx-w-
chmod 733: -rwx-wx-wx
chmod 734: -rwx-wxr--
chmod 735: -rwx-wxr-x
chmod 736: -rwx-wxrw-
chmod 737: -rwx-wxrwx
chmod 740: -rwxr-----
chmod 741: -rwxr----x
chmod 742: -rwxr---w-
chmod 743: -rwxr---wx
chmod 744: -rwxr--r--
chmod 745: -rwxr--r-x
chmod 746: -rwxr--rw-
chmod 747: -rwxr--rwx
chmod 750: -rwxr-x---
chmod 751: -rwxr-x--x
chmod 752: -rwxr-x-w-
chmod 753: -rwxr-x-wx
chmod 754: -rwxr-xr--
chmod 755: -rwxr-xr-x
chmod 756: -rwxr-xrw-
chmod 757: -rwxr-xrwx
chmod 760: -rwxrw----
chmod 761: -rwxrw---x
chmod 762: -rwxrw--w-
chmod 763: -rwxrw--wx
chmod 764: -rwxrw-r--
chmod 765: -rwxrw-r-x
chmod 766: -rwxrw-rw-
chmod 767: -rwxrw-rwx
chmod 770: -rwxrwx---
chmod 771: -rwxrwx--x
chmod 772: -rwxrwx-w-
chmod 773: -rwxrwx-wx
chmod 774: -rwxrwxr--
chmod 775: -rwxrwxr-x
chmod 776: -rwxrwxrw-
chmod 777: -rwxrwxrwx

Congratulations!

Here's my custom stat function, which I am definately not finished with, so check back in a couple days and if you find any improvements please hook me up with a comment!

function askapache_stat( $filename ) {
$p=@fileperms($filename);
$s=@stat($filename);
$str='';
$t=decoct($s['mode'] & 0170000);
 
switch (octdec($t)) {
case 0140000: $str = 's'; $stat['type']='socket'; break;
case 0120000: $str = 'l'; $stat['type']='link'; break;
case 0100000: $str = '-'; $stat['type']='file'; break;
case 0060000: $str = 'b'; $stat['type']='block'; break;
case 0040000: $str = 'd'; $stat['type']='dir'; break;
case 0020000: $str = 'c'; $stat['type']='char'; break;
case 0010000: $str = 'p'; $stat['type']='fifo'; break;
default: $str = 'u'; $stat['type']='unknown'; break;
}
 
$stat['type_octal'] = sprintf("%07o", octdec($t));
 
$str .= (($p&0x0100)?'r':'-').(($p&0x0080)?'w':'-').(($p&0x0040)?(($p&0x0800)?'s':'x'):(($p&0x0800)?'S':'-'));
$str .= (($p&0x0020)?'r':'-').(($p&0x0010)?'w':'-').(($p&0x0008)?(($p&0x0400)?'s':'x'):(($p&0x0400)?'S':'-'));
$str .= (($p&0x0004)?'r':'-').(($p&0x0002)?'w':'-').(($p&0x0001)?(($p&0x0200)?'t':'x'):(($p&0x0200)?'T':'-'));
 
$stat['default_umask']=sprintf("%04o",umask());
$stat['perm_human']=$str;
$stat['perm_octal1'] = sprintf( "%o", ( $s['mode'] & 00777 ) );
$stat['perm_octal2'] = sprintf("0%o", 0777 & $p);
$stat['perm_dec'] = sprintf("%04o", $p);
$stat['perm_mode']=$s['mode'];   // File mode.
 
$stat['file'] = @realpath($filename);
$stat['basename'] = basename( $filename );
 
$stat['user_id'] = $s['uid'];
$stat['group_id'] = $s['gid'];
 
$stat['device']=$s['dev'];      // Device
$stat['device_number']=$s['rdev'];    // Device number, if device.
$stat['inode']=$s['ino'];      // File serial number
$stat['link_count']=$s['nlink'];    // link count
if($stat['type']=='link')$stat['link_to']=@readlink( $filename );
 
$stat['size']=$s['size'];    // Size of file, in bytes.
$stat['block_size']=$s['blksize'];  // Optimal block size for I/O.
$stat['blocks']=$s['blocks'];  // Number 512-byte blocks allocated
 
$stat['time_access']=@date( 'Y M D H:i:s',$s['atime']);    // Time of last access.
$stat['time_modified']=@date( 'Y M D H:i:s',$s['mtime']);    // Time of last modification
$stat['time_created']=@date( 'Y M D H:i:s',$s['ctime']);    // Time of last status change
 
clearstatcache();
return $stat;
}
 
header('Content-Type: text/plain');
$stat=askapache_stat(__FILE__);
print_r($stat);

Defining Permission Bits

!defined('S_IFMT') && define('S_IFMT', 0170000); //  mask for all types
!defined('S_IFSOCK') && define('S_IFSOCK', 0140000); // type: socket
!defined('S_IFLNK') && define('S_IFLNK', 0120000); // type:  symbolic link
!defined('S_IFREG') && define('S_IFREG', 0100000); // type:  regular file
!defined('S_IFBLK') && define('S_IFBLK', 0060000); // type:  block device
!defined('S_IFDIR') && define('S_IFDIR', 0040000); // type:  directory
!defined('S_IFCHR') && define('S_IFCHR', 0020000); // type:  character device
!defined('S_IFIFO') && define('S_IFIFO', 0010000); // type:  fifo
 
!defined('S_ISUID') && define('S_ISUID', 0004000); // set-uid bit
!defined('S_ISGID') && define('S_ISGID', 0002000); // set-gid bit
!defined('S_ISVTX') && define('S_ISVTX', 0001000); // sticky bit
!defined('S_IRWXU') && define('S_IRWXU', 00700); //  mask for owner permissions
!defined('S_IRUSR') && define('S_IRUSR', 00400); //  owner: read permission
!defined('S_IWUSR') && define('S_IWUSR', 00200); //  owner: write permission
!defined('S_IXUSR') && define('S_IXUSR', 00100); //  owner: execute permission
!defined('S_IRWXG') && define('S_IRWXG', 00070); //  mask for group permissions
!defined('S_IRGRP') && define('S_IRGRP', 00040); //  group: read permission
!defined('S_IWGRP') && define('S_IWGRP', 00020); //  group: write permission
!defined('S_IXGRP') && define('S_IXGRP', 00010); //  group: execute permission
!defined('S_IRWXO') && define('S_IRWXO', 00007); //  mask for others permissions
!defined('S_IROTH') && define('S_IROTH', 00004); //  others:  read permission
!defined('S_IWOTH') && define('S_IWOTH', 00002); //  others:  write permission
!defined('S_IXOTH') && define('S_IXOTH', 00001); //  others:  execute permission
 
!defined('S_IRWXUGO') && define('S_IRWXUGO', (S_IRWXU | S_IRWXG | S_IRWXO));
!defined('S_IALLUGO') && define('S_IALLUGO', (S_ISUID | S_ISGID | S_ISVTX | S_IRWXUGO));
!defined('S_IRUGO') && define('S_IRUGO', (S_IRUSR | S_IRGRP | S_IROTH));
!defined('S_IWUGO') && define('S_IWUGO', (S_IWUSR | S_IWGRP | S_IWOTH));
!defined('S_IXUGO') && define('S_IXUGO', (S_IXUSR | S_IXGRP | S_IXOTH));
!defined('S_IRWUGO') && define('S_IRWUGO', (S_IRUGO | S_IWUGO));

How File Permissions Work

When PHP is installed on your server by you or whoever runs the server, it uses the file permissions that are used by the Operating System running the server.. If you are smart or just lucky than you are running some type of BSD/Unix/Solaris/Linux/Sun based Operating system and PHP won't have any problems. If you are running on a Locked, proprietary OS like Windows, PHP will still work but it has to use a lot of shortcuts and hacks to basically "Pretend" to act like the OS is BSD/Unix, and some key features just won't be available.

The OS Permission Bits

Here's the file permissions my Linux server uses, and which PHP automatically uses. The code basically just defines the default permissions for files, and defines the file atributes for each file that you can access by using the stat function, which I've improved upon to make things easier.

Download: POSIX Standard: 5.6 File Characteristicssys/stat.h

Protection bits for File Owner

#define S_IRWXU 00700
#define S_IRUSR 00400
#define S_IWUSR 00200
#define S_IXUSR 00100

Protection bits for File Group

#define S_IRWXG 00070
#define S_IRGRP 00040
#define S_IWGRP 00020
#define S_IXGRP 00010

Protection bits for All Others

#define S_IRWXO 00007
#define S_IROTH 00004
#define S_IWOTH 00002
#define S_IXOTH 00001

Some Example Permissions

0477 // owner has read only, other and group has rwx 0677 // owner has rw only, other and group has rwx

0444 // all have read only 0666 // all have rw only

0400 // owner has read only, group and others have no permission 0600 // owner has rw only, group and others have no permission

0470 // owner has read only, group has rwx, others have no permission 0407 // owner has read only, other has rwx, group has no permission

0670 // owner has rw only, group has rwx, others have no permission 0607 // owner has rw only, group has no permission and others have rwx

What's a File

A file is not merely its contents, a name, and a file type. A file also has an owner (a user ID), a group (a group ID), permissions (what the owner can do with the file, what people in the group can do, and what everyone else can do), various timestamps, and other information. Collectively, we call these a file's attributes.

Structure of File Mode Bits

The file mode bits have two parts: the file permission bits, which control ordinary access to the file, and special mode bits, which affect only some files.

There are three kinds of permissions that a user can have for a file:

permission to read the file. For directories, this means permission to list the contents of the directory.
permission to write to (change) the file. For directories, this means permission to create and remove files in the directory.
permission to execute the file (run it as a program). For directories, this means permission to access files in the directory.

There are three categories of users who may have different permissions to perform any of the above operations on a file:

the file's owner.
other users who are in the file's group
everyone else.

Files are given an owner and group when they are created. Usually the owner is the current user and the group is the group of the directory the file is in, but this varies with the operating system, the file system the file is created on, and the way the file is created. You can change the owner and group of a file by using the chown and chgrp commands.

In addition to the three sets of three permissions listed above, the file mode bits have three special components, which affect only executable files (programs) and, on most systems, directories:

Set the process's effective user ID to that of the file upon execution (called the set-user-ID bit, or sometimes the setuid bit). For directories on a few systems, give files created in the directory the same owner as the directory, no matter who creates them, and set the set-user-ID bit of newly-created subdirectories.
Set the process's effective group ID to that of the file upon execution (called the set-group-ID bit, or sometimes the setgid bit). For directories on most systems, give files created in the directory the same group as the directory, no matter what group the user who creates them is in, and set the set-group-ID bit of newly-created subdirectories.
Prevent unprivileged users from removing or renaming a file in a directory unless they own the file or the directory; this is called the restricted deletion flag for the directory, and is commonly found on world-writable directories like /tmp.

For regular files on some older systems, save the program's text image on the swap device so it will load more quickly when run; this is called the sticky bit.

Setting Permissions

The basic symbolic operations on a file's permissions are adding, removing, and setting the permission that certain users have to read, write, and execute or search the file. These operations have the following format:

users operation permissions

The spaces between the three parts above are shown for readability only; symbolic modes cannot contain spaces. The users part tells which users' access to the file is changed. It consists of one or more of the following letters (or it can be empty). When more than one of these letters is given, the order that they are in does not matter.

u - the user who owns the file.
g - other users who are in the file's group.
o - all other users.
a - all users; the same as ugo.

The operation part tells how to change the affected users' access to the file, and is one of the following symbols:

+ - to add the permissions to whatever permissions the users already have for the file.
- - to remove the permissions from whatever permissions the users already have for the file.
= - to make the permissions the only permissions that the users have for the file.

The permissions part tells what kind of access to the file should be changed; it is normally zero or more of the following letters. As with the users part, the order does not matter when more than one letter is given. Omitting the permissions part is useful only with the = operation, where it gives the specified users no access at all to the file.

r - the permission the users have to read the file.
w - the permission the users have to write to the file.
x - the permission the users have to execute the file, or search it if it is a directory.

For example, to give everyone permission to read and write a regular file, but not to execute it, use:

a=rw

To remove write permission for all users other than the file's owner, use:

go-w

The above command does not affect the access that the owner of the file has to it, nor does it affect whether other users can read or execute the file.

To give everyone except a file's owner no permission to do anything with that file, use the mode below. Other users could still remove the file, if they have write permission on the directory it is in.

go=

Another way to specify the same thing is:

og-rwx

Copying Existing Permissions

You can base a file's permissions on its existing permissions. To do this, instead of using a series of r, w, or x letters after the operator, you use the letter u, g, or o. For example, the mode

o+g

adds the permissions for users who are in a file's group to the permissions that other users have for the file. Thus, if the file started out as mode 664 (rw-rw-r--), the above mode would change it to mode 666 (rw-rw-rw-). If the file had started out as mode 741 (rwxr----x), the above mode would change it to mode 745 (rwxr--r-x). The - and = operations work analogously.

Umask and Protection

If the users part of a symbolic mode is omitted, it defaults to a (affect all users), except that any permissions that are set in the system variable umask are not affected. The value of umask can be set using the umask command. Its default value varies from system to system.

Omitting the users part of a symbolic mode is generally not useful with operations other than +. It is useful with + because it allows you to use umask as an easily customizable protection against giving away more permission to files than you intended to. As an example, if umask has the value 2, which removes write permission for users who are not in the file's group, then the mode:

+w

adds permission to write to the file to its owner and to other users who are in the file's group, but not to other users. In contrast, the mode:

a+w

ignores umask, and does give write permission for the file to all users.

Directories, Set-User-ID and Set-Group-ID Bits

On most systems, if a directory's set-group-ID bit is set, newly created subfiles inherit the same group as the directory, and newly created subdirectories inherit the set-group-ID bit of the parent directory. On a few systems, a directory's set-user-ID bit has a similar effect on the ownership of new subfiles and the set-user-ID bits of new subdirectories. These mechanisms let users share files more easily, by lessening the need to use chmod or chown to share new files.

These convenience mechanisms rely on the set-user-ID and set-group-ID bits of directories. If commands like chmod and mkdir routinely cleared these bits on directories, the mechanisms would be less convenient and it would be harder to share files. Therefore, a command like chmod does not affect the set-user-ID or set-group-ID bits of a directory unless the user specifically mentions them in a symbolic mode, or sets them in a numeric mode. For example, on systems that support set-group-ID inheritance:

# These commands leave the set-user-ID and
# set-group-ID bits of the subdirectories alone,
# so that they retain their default values.
mkdir A B C
chmod 755 A
chmod 0755 B
chmod u=rwx,go=rx C
mkdir -m 755 D
mkdir -m 0755 E
mkdir -m u=rwx,go=rx F

If you want to try to set these bits, you must mention them explicitly in the symbolic or numeric modes, e.g.:

# These commands try to set the set-user-ID
# and set-group-ID bits of the subdirectories.
mkdir G H
chmod 6755 G
chmod u=rwx,go=rx,a+s H
mkdir -m 6755 I
mkdir -m u=rwx,go=rx,a+s J

If you want to try to clear these bits, you must mention them explicitly in a symbolic mode, e.g.:

# This command tries to clear the set-user-ID
# and set-group-ID bits of the directory D.
chmod a-s D

Numeric Modes

The permissions granted to the user, to other users in the file's group, and to other users not in the file's group each require three bits, which are represented as one octal digit. The three special mode bits also require one bit each, and they are as a group represented as another octal digit. Here is how the bits are arranged, starting with the lowest valued bit:

Other users not in the file's group:

1 Execute/search
2 Write
4 Read

Other users in the file's group:

10 Execute/search
20 Write
40 Read

The file's owner:

100 Execute/search
200 Write
400 Read

Special mode bits:

1000 Restricted deletion flag or sticky bit
2000 Set group ID on execution
4000 Set user ID on execution

For example, numeric mode 4755 corresponds to symbolic mode u=rwxs,go=rx, and numeric mode 664 corresponds to symbolic mode ug=rw,o=r. Numeric mode 0 corresponds to symbolic mode a=.

Apache's Internal Bits (hex)

#define APR_FPROT_USETID   0x8000 /* Set user id */
#define APR_FPROT_UREAD   0x0400 /* Read by user */
#define APR_FPROT_UWRITE   0x0200 /* Write by user */
#define APR_FPROT_UEXECUTE 0x0100 /* Execute by user */
 
#define APR_FPROT_GSETID   0x4000 /* Set group id */
#define APR_FPROT_GREAD   0x0040 /* Read by group */
#define APR_FPROT_GWRITE   0x0020 /* Write by group */
#define APR_FPROT_GEXECUTE 0x0010 /* Execute by group */
 
#define APR_FPROT_WSTICKY 0x2000 /* Sticky bit */
#define APR_FPROT_WREAD   0x0004 /* Read by others */
#define APR_FPROT_WWRITE 0x0002 /* Write by others */
#define APR_FPROT_WEXECUTE 0x0001 /* Execute by others */
 
#define APR_FPROT_OS_DEFAULT  0x0FFF /* use OS's default permissions */
 
/* additional permission flags for apr_file_copy  and apr_file_append */
#define APR_FPROT_FILE_SOURCE_PERMS 0x1000 /* Copy source file's permissions */

Download: httpd-2.2.10/srclib/apr/file_io/unix/fileacc.c Here's some interesting bitmasking done by Apache that uses the defined bits set earlier by stat.h

apr_unix_perms2mode(perms){
 mode=0;
 if (perms & APR_USETID) mode |= S_ISUID;
 if (perms & APR_UREAD)  mode |= S_IRUSR;
 if (perms & APR_UWRITE) mode |= S_IWUSR;
 if (perms & APR_UEXECUTE) mode |= S_IXUSR;
 
 if (perms & APR_GSETID) mode |= S_ISGID;
 if (perms & APR_GREAD)  mode |= S_IRGRP;
 if (perms & APR_GWRITE) mode |= S_IWGRP;
 if (perms & APR_GEXECUTE) mode |= S_IXGRP;
 
 if (perms & APR_WSTICKY) mode |= S_ISVTX;
 if (perms & APR_WREAD)  mode |= S_IROTH;
 if (perms & APR_WWRITE) mode |= S_IWOTH;
 if (perms & APR_WEXECUTE) mode |= S_IXOTH;
 return mode;
}
 
apr_unix_mode2perms(mode){
 perms = 0;
 if (mode & S_ISUID)perms |= APR_USETID;
 if (mode & S_IRUSR)perms |= APR_UREAD;
 if (mode & S_IWUSR)perms |= APR_UWRITE;
 if (mode & S_IXUSR)perms |= APR_UEXECUTE;
 
 if (mode & S_ISGID)perms |= APR_GSETID;
 if (mode & S_IRGRP)perms |= APR_GREAD;
 if (mode & S_IWGRP)perms |= APR_GWRITE;
 if (mode & S_IXGRP)perms |= APR_GEXECUTE;
 
 if (mode & S_ISVTX)perms |= APR_WSTICKY;
 if (mode & S_IROTH)perms |= APR_WREAD;
 if (mode & S_IWOTH)perms |= APR_WWRITE;
 if (mode & S_IXOTH)perms |= APR_WEXECUTE;
 return perms;
}

umask

umask(int mask){
 arg1;
 int oldumask;
 int arg_count = ZEND_NUM_ARGS();
 oldumask = umask(077);
 
 if (BG(umask) == -1) BG(umask) = oldumask;
 if (arg_count == 0) umask(oldumask);
 
 convert_to_long_ex(arg1);
 umask(Z_LVAL_PP(arg1));
 RETURN_LONG(oldumask);
}

File Attributes

Each file will have attributes based on the type of OS.. Using the stat command you can view them.

Viewing stat results

* %a - Access rights in octal
* %A - Access rights in human readable form
* %b - Number of blocks allocated (see %B)
* %B - The size in bytes of each block reported by %b
* %d - Device number in decimal
* %D - Device number in hex
* %f - Raw mode in hex
* %F - File type
* %g - Group ID of owner
* %G - Group name of owner
* %h - Number of hard links
* %i - Inode number
* %n - File name
* %N - Quoted file name with dereference if symbolic link
* %o - I/O block size
* %s - Total size, in bytes
* %t - Major device type in hex
* %T - Minor device type in hex
* %u - User ID of owner
* %U - User name of owner
* %x - Time of last access
* %X - Time of last access as seconds since Epoch
* %y - Time of last modification
* %Y - Time of last modification as seconds since Epoch
* %z - Time of last change
* %Z - Time of last change as seconds since Epoch

The OS Attribute Bits

These defined values are what allows your operating system to determine the type of file being accessed.

#define S_IFMT   00170000  /* These bits determine file type. */
#define S_IFSOCK 0140000  /* Socket file */
#define S_IFLNK   0120000  /* Symbolic Link */
#define S_IFREG   0100000  /* Regular file */
#define S_IFDIR   0040000  /* Directory */
#define S_IFIFO  0010000   /* FIFO first-in-first-out file */
 
/* Such devices can be read either a character at a time or a "block" (many characters) at a time,
hence we say there are block special files and character special files. */
#define S_IFBLK   0060000  /* Block device */
#define S_IFCHR  0020000  /* Character device */

Special Permission Bits

#define S_ISUID  0004000  /* Set user ID on execution.  */
#define S_ISGID  0002000  /* Set group ID on execution.  */
#define S_ISVTX  0001000 /* Save swapped text after use (sticky).  */

Bitmasking to determine Filetype

#define S_ISLNK(m) (((m) & S_IFMT) == S_IFLNK)
#define S_ISREG(m) (((m) & S_IFMT) == S_IFREG)
#define S_ISDIR(m) (((m) & S_IFMT) == S_IFDIR)
#define S_ISCHR(m) (((m) & S_IFMT) == S_IFCHR)
#define S_ISBLK(m) (((m) & S_IFMT) == S_IFBLK)
#define S_ISFIFO(m) (((m) & S_IFMT) == S_IFIFO)
#define S_ISSOCK(m) (((m) & S_IFMT) == S_IFSOCK)

Default Permission Masks

#define S_IRWXUGO (S_IRWXU|S_IRWXG|S_IRWXO)
#define S_IALLUGO (S_ISUID|S_ISGID|S_ISVTX|S_IRWXUGO)
#define S_IRUGO  (S_IRUSR|S_IRGRP|S_IROTH)
#define S_IWUGO  (S_IWUSR|S_IWGRP|S_IWOTH)
#define S_IXUGO  (S_IXUSR|S_IXGRP|S_IXOTH)

Download: httpd-2.2.10/srclib/apr/file_io/unix/filestat.c, this file shows a simple way to determine the type of file.

filetype_from_mode(mode){
 type;
 switch (mode & S_IFMT) {
  case S_IFREG:  type = APR_REG;  break;
  case S_IFDIR:  type = APR_DIR;  break;
  case S_IFLNK:  type = APR_LNK;  break;
 
  case S_IFCHR:  type = APR_CHR;  break;
  case S_IFBLK:  type = APR_BLK;  break;
  case S_IFFIFO: type = APR_PIPE; break;
  case S_IFSOCK: type = APR_SOCK; break;
  default: type = APR_UNKFILE;
 }
 return type;
}

Apache Stat Bits

#define APR_FINFO_LINK  0x00000001 /* Stat the link not the file itself if it is a link */
#define APR_FINFO_MTIME  0x00000010 /* Modification Time */
#define APR_FINFO_CTIME  0x00000020 /* Creation or inode-changed time */
#define APR_FINFO_ATIME  0x00000040 /* Access Time */
#define APR_FINFO_SIZE  0x00000100 /* Size of the file */
#define APR_FINFO_CSIZE  0x00000200 /* Storage size consumed by the file */
#define APR_FINFO_DEV  0x00001000 /* Device */
#define APR_FINFO_INODE  0x00002000 /* Inode */
#define APR_FINFO_NLINK  0x00004000 /* Number of links */
#define APR_FINFO_TYPE  0x00008000 /* Type */
#define APR_FINFO_USER  0x00010000 /* User */
#define APR_FINFO_GROUP  0x00020000 /* Group */
#define APR_FINFO_UPROT  0x00100000 /* User protection bits */
#define APR_FINFO_GPROT  0x00200000 /* Group protection bits */
#define APR_FINFO_WPROT  0x00400000 /* World protection bits */
#define APR_FINFO_ICASE  0x01000000 /* if dev is case insensitive */
#define APR_FINFO_NAME  0x02000000 /* name in proper case */
#define APR_FINFO_MIN  0x00008170 /* type, mtime, ctime, atime, size */
#define APR_FINFO_IDENT  0x00003000 /* dev and inode */
#define APR_FINFO_OWNER  0x00030000 /* user and group */
#define APR_FINFO_PROT  0x00700000 /* all protections */
#define APR_FINFO_NORM  0x0073b170 /* an atomic unix apr_stat() */
#define APR_FINFO_DIRENT 0x02000000 /* an atomic unix apr_dir_read() */

The Apache file information structure.

apr_uid_t user;  /* The user id that owns the file */
apr_gid_t group;  /* The group id that owns the file */
apr_ino_t inode; /* The inode of the file. */
apr_dev_t device; /* The id of the device the file is on. */
apr_int32_t nlink; /* The number of hard links to the file. */
apr_off_t size;  /* The size of the file */
apr_off_t csize; /* The storage size consumed by the file */
apr_time_t atime; /* The time the file was last accessed */
apr_time_t mtime; /* The time the file was last modified */
apr_time_t ctime; /* The time the file was created, or the inode was last changed */
const char *fname; /* The pathname of the file (possibly unrooted) */
const char *name; /* The file's name (no path) in filesystem case */

File Time Attributes

touch
If changing both the access and modification times to the current time, touch can change the timestamps for files that the user running it does not own but has write permission for. Otherwise, the user must own the files.

Although touch provides options for changing two of the times the times of last access and modification of a file, there is actually a third one as well: the inode change time. This is often referred to as a file's ctime. The inode change time represents the time when the file's meta-information last changed. One common example of this is when the permissions of a file change. Changing the permissions doesn't access the file, so the atime doesn't change, nor does it modify the file, so the mtime doesn't change. Yet, something about the file itself has changed, and this must be noted somewhere. This is the job of the ctime field. This is necessary, so that, for example, a backup program can make a fresh copy of the file, including the new permissions value. Another operation that modifies a file's ctime without affecting the others is renaming. In any case, it is not possible, in normal operations, for a user to change the ctime field to a user-specified value.

WebHost allows you to create multiple users per account. Each user can have domain assigned to its home home directory accessible via FTP or SSH/SCP. The problem with multiple users on the same account is that they share the same default unix group, and default permissions allow their files to be easily modified by the members of this group. Usually this doesn't pose a problem as each user is probably trusted by account owner to not to mess with others files, but if one of the users have their web application hacked then all other users on the same account will be in danger.

By default all files in your account are created with 644 privileges and directories are with 775. That means any user can read your files and any user from the same account can move and add files in your freshly made directories. Your home directory is different, though. By default it carries 751 attribute meaning that only members of your group can see your files, but can't add any new. These group access schemes are possible, because every user in your account has its primary/default group set to "pgxxxxxx", which is assigned to every new file you create by default. The normal way to secure users from web-intrusion is to assign a separate group to the web-server user, removing it from default group. This way, exploited scripts will not be able to traverse into home directories of other users on your account. To allow account users to update centralized web-site they could be added to web-site group explicitly. But this "normal way" doesn't work with DreamHost, because you can't delete web-user from the default group and unless you set access for every new file explicitly, it will be possible for an intruder to read it.

To make managing privileges easier in interactive sessions "umask 007" command can be specified in your .bash_profile - this makes all new files carry xx0 mask. You also need to control your scripts (web based or cron/shell) so that they set mask for critical files explicitly. To secure account users from access by means of hacked user script you would also like to define another group for every user in your account and change group ownership of the user's home directory to that group with "set gid" bit set (and optional umask 007 in .bash_profile).

Therefore, to secure your users from web-intrusion you need to:

Add a separate user and group for every domain where apache will be running
Add a separate group for other user accounts
Change the default group for new files created by your users by changing the group of their home directory and setting "set gid" bit for it (it is impossible to do this with FTP accounts, therefore you will need to login in each account via SSH)
Add users who need access to web-site into the web-user group
Optionally set umask 007 in .bash_profile for every user to tweak default WebHost775/664 permissions to something like 770/660 for directories and files that are not meant to be read by Apache (660 could also be used for all web scripts including .php as they are not read by dhapache CGI, but merely executed)

Apache Security

All your web files that need to be read by Apache should be readable by everyone as Apache itself is run under dhapache user. However, executable scripts like .php are executed under your own user and do not have to be world readable as they are not actually read by Apache, but executed via suEXEC. Quite the opposite - to prevent your code or database settings from being messed by any third-parties you SHOULD set permissions to these files explicitly to something like 640 or even 600 depending on who do you trust.

Multiuser security setup example

For our example, we will create a rainforce_www user and a aapp_www group for serving web files with apache and setup a rainforce user with a 'aapp group to manage mail and keep other files on DH privately. Since these records already exist, you will need to subsitute your own names.

Login to create the users rainforce_www and rainforce with shell access.
Create two groups - aapp_www and aapp. Note that users created in previous step are still members of the same default pgxxxxxx group.
Add rainforce_www to 'the 'aapp_www group and rainforce to both the aapp_www and aapp groups
Move your domain to rainforce_www account (mine is rainforce.org)
Now login to SSH with your rainforce_www user and change the default group for your home directory with "sgid" bit set to make all current and new files/directories created in this directory have the same aapp_www group.

 $ chgrp -R aapp_www .
 $ chmod 2751 .
 $ chmod 2771 rainforce.org

By setting 2771 the directory will be writable by the owner, the group and will be only executable by others. The contents of an executable only directory cannot be listed, but the files inside it can be read (if the permissions of the file allow it). It is important that the directory can be executable in order to allow static content (e.g. .html files) inside it to be read. Remember that directories you don't want anyone to have web access to, should be 0770 (writable by the owner and group, or 0750 writable by the owner and readable by group). Such strict permissions should by applied to password files, php include files or databases files (such as SQLite, BDB, etc).

Do the same for rainforce user, but specify aapp group instead.

 $ chgrp -R aapp .
 $ chmod 2751 .

Optionally modify umask in .bash_profile in user's home to 007 to make all files created by this user have 660 permissions set by default. If you want that newly created files by accessible by the web, you need to manually setup it's permissions to 664.

Now I can login as the user "rainforce" and update the web-site in the ../rainforce_www/rainforce.org directory. There is one more setup needed. Because files copied from other accounts can have 644 permissions set instead of 664, you need a script which will update permissions to 664 or 660 to allow other group members modify such files.

SSH key fingerprints

Just gen your own I guess

External Links

Original Article from DreamHost Wiki

Content is available under GNU Free Documentation License 1.2.

Example File Permission Bits

`/usr/lib/w3m/cgi-bin/dirlist.cgi`

sub utype {
  local($_) = @_;
  local(%T) = (
    0010000, 'PIPE',
    0020000, 'CHR',
    0040000, 'DIR',
    0060000, 'BLK',
    0100000, 'FILE',
    0120000, 'LINK',
    0140000, 'SOCK',
  );
  return $T{($_ & 0170000)} || 'FILE';
}
 
sub umode {
  local($_) = @_;
  local(%T) = (
    0010000, 'p',
    0020000, 'c',
    0040000, 'd',
    0060000, 'b',
    0100000, '-',
    0120000, 'l',
    0140000, 's',
  );
 
  return ($T{($_ & 0170000)} || '-')
     . (($_ & 00400) ? 'r' : '-')
     . (($_ & 00200) ? 'w' : '-')
     . (($_ & 04000) ? 's' :
       (($_ & 00100) ? 'x' : '-'))
     . (($_ & 00040) ? 'r' : '-')
     . (($_ & 00020) ? 'w' : '-')
     . (($_ & 02000) ? 's' :
       (($_ & 00010) ? 'x' : '-'))
     . (($_ & 00004) ? 'r' : '-')
     . (($_ & 00002) ? 'w' : '-')
     . (($_ & 01000) ? 't' :
       (($_ & 00001) ? 'x' : '-'));
}

`/usr/lib/perl/5.8.4/linux/stat.ph`

        eval 'sub S_IFMT () {00170000;}' unless defined(&S_IFMT);
        eval 'sub S_IFSOCK () {0140000;}' unless defined(&S_IFSOCK);
        eval 'sub S_IFLNK () {0120000;}' unless defined(&S_IFLNK);
        eval 'sub S_IFREG () {0100000;}' unless defined(&S_IFREG);
        eval 'sub S_IFBLK () {0060000;}' unless defined(&S_IFBLK);
        eval 'sub S_IFDIR () {0040000;}' unless defined(&S_IFDIR);
        eval 'sub S_IFCHR () {0020000;}' unless defined(&S_IFCHR);
        eval 'sub S_IFIFO () {0010000;}' unless defined(&S_IFIFO);
        eval 'sub S_ISUID () {0004000;}' unless defined(&S_ISUID);
        eval 'sub S_ISGID () {0002000;}' unless defined(&S_ISGID);
        eval 'sub S_ISVTX () {0001000;}' unless defined(&S_ISVTX);
        eval 'sub S_ISLNK {
            local($m) = @_;
            eval q(((($m) &  &S_IFMT) ==  &S_IFLNK));
        }' unless defined(&S_ISLNK);
        eval 'sub S_ISREG {
            local($m) = @_;
            eval q(((($m) &  &S_IFMT) ==  &S_IFREG));
        }' unless defined(&S_ISREG);
        eval 'sub S_ISDIR {
            local($m) = @_;
            eval q(((($m) &  &S_IFMT) ==  &S_IFDIR));
        }' unless defined(&S_ISDIR);
        eval 'sub S_ISCHR {
            local($m) = @_;
            eval q(((($m) &  &S_IFMT) ==  &S_IFCHR));
        }' unless defined(&S_ISCHR);
        eval 'sub S_ISBLK {
            local($m) = @_;
            eval q(((($m) &  &S_IFMT) ==  &S_IFBLK));
        }' unless defined(&S_ISBLK);
        eval 'sub S_ISFIFO {
            local($m) = @_;
            eval q(((($m) &  &S_IFMT) ==  &S_IFIFO));
        }' unless defined(&S_ISFIFO);
        eval 'sub S_ISSOCK {
            local($m) = @_;
            eval q(((($m) &  &S_IFMT) ==  &S_IFSOCK));
        }' unless defined(&S_ISSOCK);
        eval 'sub S_IRWXU () {00700;}' unless defined(&S_IRWXU);
        eval 'sub S_IRUSR () {00400;}' unless defined(&S_IRUSR);
        eval 'sub S_IWUSR () {00200;}' unless defined(&S_IWUSR);
        eval 'sub S_IXUSR () {00100;}' unless defined(&S_IXUSR);
        eval 'sub S_IRWXG () {00070;}' unless defined(&S_IRWXG);
        eval 'sub S_IRGRP () {00040;}' unless defined(&S_IRGRP);
        eval 'sub S_IWGRP () {00020;}' unless defined(&S_IWGRP);
        eval 'sub S_IXGRP () {00010;}' unless defined(&S_IXGRP);
        eval 'sub S_IRWXO () {00007;}' unless defined(&S_IRWXO);
        eval 'sub S_IROTH () {00004;}' unless defined(&S_IROTH);
        eval 'sub S_IWOTH () {00002;}' unless defined(&S_IWOTH);
        eval 'sub S_IXOTH () {00001;}' unless defined(&S_IXOTH);
    }
    if(defined(&__KERNEL__)) {
        eval 'sub S_IRWXUGO () {( &S_IRWXU| &S_IRWXG| &S_IRWXO);}' unless defined(&S_IRWXUGO);
        eval 'sub S_IALLUGO () {( &S_ISUID| &S_ISGID| &S_ISVTX| &S_IRWXUGO);}' unless defined(&S_IALLUGO);
        eval 'sub S_IRUGO () {( &S_IRUSR| &S_IRGRP| &S_IROTH);}' unless defined(&S_IRUGO);
        eval 'sub S_IWUGO () {( &S_IWUSR| &S_IWGRP| &S_IWOTH);}' unless defined(&S_IWUGO);
        eval 'sub S_IXUGO () {( &S_IXUSR| &S_IXGRP| &S_IXOTH);}' unless defined(&S_IXUGO);
        require 'linux/types.ph';
        require 'linux/time.ph';
    }

Mozilla-Source 1.8a2

/* notice that these valuse are octal. */
const PERM_IRWXU = 00700;  /* read, write, execute/search by owner */
const PERM_IRUSR = 00400;  /* read permission, owner */
const PERM_IWUSR = 00200;  /* write permission, owner */
const PERM_IXUSR = 00100;  /* execute/search permission, owner */
const PERM_IRWXG = 00070;  /* read, write, execute/search by group */
const PERM_IRGRP = 00040;  /* read permission, group */
const PERM_IWGRP = 00020;  /* write permission, group */
const PERM_IXGRP = 00010;  /* execute/search permission, group */
const PERM_IRWXO = 00007;  /* read, write, execute/search by others */
const PERM_IROTH = 00004;  /* read permission, others */
const PERM_IWOTH = 00002;  /* write permission, others */
const PERM_IXOTH = 00001;  /* execute/search permission, others */
 
const MODE_RDONLY   = 0x01;
const MODE_WRONLY   = 0x02;
const MODE_RDWR     = 0x04;
const MODE_CREATE   = 0x08;
const MODE_APPEND   = 0x10;
const MODE_TRUNCATE = 0x20;
const MODE_SYNC     = 0x40;
const MODE_EXCL     = 0x80;

`/usr/include/libpng12/png.h`

/* Transform masks for the high-level interface */
#define PNG_TRANSFORM_IDENTITY       0x0000    /* read and write */
#define PNG_TRANSFORM_STRIP_16       0x0001    /* read only */
#define PNG_TRANSFORM_STRIP_ALPHA    0x0002    /* read only */
#define PNG_TRANSFORM_PACKING        0x0004    /* read and write */
#define PNG_TRANSFORM_PACKSWAP       0x0008    /* read and write */
#define PNG_TRANSFORM_EXPAND         0x0010    /* read only */
#define PNG_TRANSFORM_INVERT_MONO    0x0020    /* read and write */
#define PNG_TRANSFORM_SHIFT          0x0040    /* read and write */
#define PNG_TRANSFORM_BGR            0x0080    /* read and write */
#define PNG_TRANSFORM_SWAP_ALPHA     0x0100    /* read and write */
#define PNG_TRANSFORM_SWAP_ENDIAN    0x0200    /* read and write */
#define PNG_TRANSFORM_INVERT_ALPHA   0x0400    /* read and write */
#define PNG_TRANSFORM_STRIP_FILLER   0x0800    /* WRITE only */

`/usr/lib/python2.4/stat.py`

# Extract bits from the mode
 
def S_IMODE(mode):
    return mode & 07777
 
def S_IFMT(mode):
    return mode & 0170000
 
# Constants used as S_IFMT() for various file types
# (not all are implemented on all systems)
 
S_IFDIR  = 0040000
S_IFCHR  = 0020000
S_IFBLK  = 0060000
S_IFREG  = 0100000
S_IFIFO  = 0010000
S_IFLNK  = 0120000
S_IFSOCK = 0140000
 
# Functions to test for each file type
 
def S_ISDIR(mode):
    return S_IFMT(mode) == S_IFDIR
 
def S_ISCHR(mode):
    return S_IFMT(mode) == S_IFCHR
 
def S_ISBLK(mode):
    return S_IFMT(mode) == S_IFBLK
 
def S_ISREG(mode):
    return S_IFMT(mode) == S_IFREG
 
def S_ISFIFO(mode):
    return S_IFMT(mode) == S_IFIFO
 
def S_ISLNK(mode):
    return S_IFMT(mode) == S_IFLNK
 
def S_ISSOCK(mode):
    return S_IFMT(mode) == S_IFSOCK
 
# Names for permission bits
 
S_ISUID = 04000
S_ISGID = 02000
S_ENFMT = S_ISGID
S_ISVTX = 01000
S_IREAD = 00400
S_IWRITE = 00200
S_IEXEC = 00100
S_IRWXU = 00700
S_IRUSR = 00400
S_IWUSR = 00200
S_IXUSR = 00100
S_IRWXG = 00070
S_IRGRP = 00040
S_IWGRP = 00020
S_IXGRP = 00010
S_IRWXO = 00007
S_IROTH = 00004
S_IWOTH = 00002
S_IXOTH = 00001

`/usr/include/bits/stat.h`

/* Encoding of the file mode.  */
 
#define __S_IFMT        0170000 /* These bits determine file type.  */
 
/* File types.  */
#define __S_IFDIR       0040000 /* Directory.  */
#define __S_IFCHR       0020000 /* Character device.  */
#define __S_IFBLK       0060000 /* Block device.  */
#define __S_IFREG       0100000 /* Regular file.  */
#define __S_IFIFO       0010000 /* FIFO.  */
#define __S_IFLNK       0120000 /* Symbolic link.  */
#define __S_IFSOCK      0140000 /* Socket.  */
 
/* POSIX.1b objects.  Note that these macros always evaluate to zero.  But
   they do it by enforcing the correct use of the macros.  */
#define __S_TYPEISMQ(buf)  ((buf)->st_mode - (buf)->st_mode)
#define __S_TYPEISSEM(buf) ((buf)->st_mode - (buf)->st_mode)
#define __S_TYPEISSHM(buf) ((buf)->st_mode - (buf)->st_mode)
 
/* Protection bits.  */
 
#define __S_ISUID       04000   /* Set user ID on execution.  */
#define __S_ISGID       02000   /* Set group ID on execution.  */
#define __S_ISVTX       01000   /* Save swapped text after use (sticky).  */
#define __S_IREAD       0400    /* Read by owner.  */
#define __S_IWRITE      0200    /* Write by owner.  */
#define __S_IEXEC       0100    /* Execute by owner.  */

`/usr/include/linux/nfs.h`

#define NFS_FIFO_DEV    (-1)
#define NFSMODE_FMT     0170000
#define NFSMODE_DIR     0040000
#define NFSMODE_CHR     0020000
#define NFSMODE_BLK     0060000
#define NFSMODE_REG     0100000
#define NFSMODE_LNK     0120000
#define NFSMODE_SOCK    0140000
#define NFSMODE_FIFO    0010000

`/usr/include/linux/nfs3.h`

#define NFS3_FIFO_DEV           (-1)
#define NFS3MODE_FMT            0170000
#define NFS3MODE_DIR            0040000
#define NFS3MODE_CHR            0020000
#define NFS3MODE_BLK            0060000
#define NFS3MODE_REG            0100000
#define NFS3MODE_LNK            0120000
#define NFS3MODE_SOCK           0140000
#define NFS3MODE_FIFO           0010000
 
/* Flags for access() call */
#define NFS3_ACCESS_READ        0x0001
#define NFS3_ACCESS_LOOKUP      0x0002
#define NFS3_ACCESS_MODIFY      0x0004
#define NFS3_ACCESS_EXTEND      0x0008
#define NFS3_ACCESS_DELETE      0x0010
#define NFS3_ACCESS_EXECUTE     0x0020
#define NFS3_ACCESS_FULL        0x003f

`/usr/include/linux/stat.h`

#define S_IFMT  00170000
#define S_IFSOCK 0140000
#define S_IFLNK  0120000
#define S_IFREG  0100000
#define S_IFBLK  0060000
#define S_IFDIR  0040000
#define S_IFCHR  0020000
#define S_IFIFO  0010000
#define S_ISUID  0004000
#define S_ISGID  0002000
#define S_ISVTX  0001000
 
#define S_ISLNK(m)      (((m) & S_IFMT) == S_IFLNK)
#define S_ISREG(m)      (((m) & S_IFMT) == S_IFREG)
#define S_ISDIR(m)      (((m) & S_IFMT) == S_IFDIR)
#define S_ISCHR(m)      (((m) & S_IFMT) == S_IFCHR)
#define S_ISBLK(m)      (((m) & S_IFMT) == S_IFBLK)
#define S_ISFIFO(m)     (((m) & S_IFMT) == S_IFIFO)
#define S_ISSOCK(m)     (((m) & S_IFMT) == S_IFSOCK)
 
#define S_IRWXU 00700
#define S_IRUSR 00400
#define S_IWUSR 00200
#define S_IXUSR 00100
 
#define S_IRWXG 00070
#define S_IRGRP 00040
#define S_IWGRP 00020
#define S_IXGRP 00010
 
#define S_IRWXO 00007
#define S_IROTH 00004
#define S_IWOTH 00002
#define S_IXOTH 00001

Further File Permissions Reading

Related PHP Functions

Special file types

link invocation: Make a hard link via the link syscall
ln invocation: Make links between files
mkdir invocation: Make directories
mkfifo invocation: Make FIFOs (named pipes)
mknod invocation: Make block or character special files
readlink invocation: Print the referent of a symbolic link
rmdir invocation: Remove empty directories
unlink invocation: Remove files via unlink syscall

Changing file attributes

chown invocation: Change file owner and group
chgrp invocation: Change group ownership
chmod invocation: Change access permissions
touch invocation: Change file timestamps

Chmod, Umask, Stat, Fileperms, and File Permissions originally appeared on AskApache.com

HOWTO: Uninstall CPANEL over SSH

AskApache — Mon, 04 Jul 2011 18:55:40 +0000

AskApache.com

WARNING! This is for advanced users of the shell, this is NOT a howto or tutorial. The thing is, I googled how to uninstall cpanel and for once in my life I came up empty.. And certainly the cpanel official sites themselves don't provide any instructions other than to say "Dont uninstall it, reinstall your entire operating system without it.".. they sure don't seem confident that they know exactly what their code is doing. Cpanel is great for most people, perfect for many situations, so don't get me wrong.. this is just for fun.

Why Uninstall?

Why? Because I have always built my servers, php installations, perl installs, ruby, iptables, everything from source. I read the INSTALL/README docs, I read the man pages, and I read the info pages as well. I google for configuration advice, I google for tips, and I don't need a web-based perl script messing my stuff up!

The main problems I had with cpanel, which really is a great bit of software for millions of website developers, is that it was incredibly sneaky! I used it for about 6 months and spent that entire time trying to figure out what the heck it was doing. A couple issues that I really disliked, it takes over your bind install, it takes over your apache install, it takes over your php install. And although it does let you configure some things (very few) for custom configurations and the like, I just don't need any of that. By removing the darn thing I am saving GIGS of space on my server, tons of bandwidth, and most importantly to me I am saving CPU and processing time along with RAM and IO speed.

Anything Else

Please.. I could go on for DAYS! Another reason I have wanted to be rid of cpanel is that I like my servers to be as lean and mean as possible. This means I like as few files and processes as possible. There are many benefits to this, like it's much easier for my integrity checking software and rootkit/antivirus software to run and drastically reduces the false positives. And there is that glaring security issue of constantly having cpanel run it's own software to create the WHM/Cpanel web interface, which is accessible online. I haven't researched cpanel security at all, it's possible that it never has security problems that are published, but for me, why take the chance?

Warning - Caution!

The following is just a quick reference of some of the commands I used to uninstall cpanel. This is for advanced users of the shell. If you aren't that advanced and you run a single one of these commands without fully understanding it, you will probably kill your server, probably lose everything on it permanently, probably not have a website or email for weeks.. So backup all your data FIRST. Also, if you aren't 100% sure you won't run into problems, you should contact your hosts technical support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for hosts, and the last thing tech support wants is to fix a server broken by someone who doesn't know what they are doing.

Last Warning!

Unless you understand what theses commands do and the purpose they serve, do not try any of this. These are not the exact commands I used verbatim, they are also not in order. I only put them up here because I was so amazed that google didn't have any uninstall cpanel intructions. Hopefully it's not a conspiracy that will get my site taken down.. ;)

That said and out of the way, it really only took me about 10 minutes to uninstall cpanel completely. But keep in mind I have been closely monitoring and debugging cpanel for 6 months, so I knew what I was doing. And finally, I do apologize for not having better instructions.. but hey, if you don't get this then you have no business trying to figure out how to uninstall cpanel! It's great software and shouldn't be removed unless you are fully capable of managing email/dns/www/ftp and any/all other servers and services on your machine by hand.

Do This First

I have a few drafts I'm working on at the moment with specifics, but for now you will have to figure it out with google. Basically you want to make sure you don't totally knock your machine offline without being able to reconnect. What I do is compile a static version of openssh and a few other security-type shell tools, and configure this binary sshd to run by using inittab, which is the file run by init (pid 1) and makes sure if it dies it is restarted. Here is my /etc/inittab to run the static sshd binary:

hh:12345:respawn:/failover/os/sbin/aassh -D -q -u0 -f /failover/os/etc/aassh

Another trick is to keep a detached screen logged in to root. That way if you mess up your sudoers or securetty or pam or whatever, you can just reattach and fix it.

Finally, you may want to setup your syslog to start earlier than usual, and set up more than normal verbosity. ( I take it to the max ). Then you should setup a 2nd server or machine somewhere to act as a syslog server. Lastly, configure your web server syslog to copy all messages to the remote syslog you set up. I use a reverse ssh tunnel to encrypt the syslog packets, but when I do something serious like reboot after uninstalling cpanel, I prepare for it by adding additional networking routes on my machine to make sure I will get some logs even if sshd cant start or even if my network addresses aren't brought up correctly.

If that sounds easy to you, please continue. If you are saying: Wha??? Continue in read-only mode.

Also, you can't just uninstall cpanel, I have replaced a lot of cpanel already, like building my own bind, apache, php, syslog and making sure they work and aren't being tampered with by cpanel. Basically cpanel runs everything on your server in most cases, so you should prepare by creating your own static software to replace cpanel, and make sure it works.

Find files Accessing /var/cpanel

More than likely these will need to be killed.

lsof +w -Rg -nP +c15 -x f +D /var/cpanel
lsof +w -Rg -nP +c15 -x f +D /usr/local/cpanel

Killing cpanel

Just an example, your machine may have a lot more than these, I have been slowly taking control of my machine back from cpanel for 6 months, so it was easier for me.

for P in tailwatchd queueprocd cpanellogd exim; do pkill -9 $P; done

Commands and Shortcuts

alias NF='nice find $PWD -mount -depth ! -type d'
alias NFF='nice find $PWD -mount -depth ! -type d | xargs -IF87 file F87'
alias NA='nice find $PWD -mount -depth'
alias NAF='nice find $PWD -mount -depth | xargs -IF87 file F87'

Watch out for crontab

An example of the sneakiness (from my POV, from most it's called builtin robustness) that cpanel does is automagically adding crontab entries that make it behave similarly to a self-propagating virus. If you don't disable the cronjobs and kill the right processes within a short period of time, be prepared for a magic resurrection.

Here's my awesome crontab information function, you will need to check every file, it lists the default crons on my box, and every users crontab, but it can't account for other cron software like at and other crons.

function askapache_crontab()
{
  local GG i;
  for i in `getent passwd|cut -d ":" -f1`;
  do
    GG=$(sudo crontab -u $i -l 2>$N6 | tr -s '\n\000' | sed '/^#/d');
    [[ ${#GG} -gt 3 ]] && sleep 1 && echo -e "$i \n\n${GG}"
  done;
   sleep 4;
   ls -aLls1ch --color=always /etc/cron.{hourly,daily,weekly,monthly,d} | sed '/^total/d; /\ drwxr-xr-x/d';
}

grep -ir /var/spool cpan

#6 3 * * * /scripts/upcp
#0 1 * * * /scripts/cpbackup
#0 2 * * * /scripts/mailman_chown_archives
#35 * * * * /usr/bin/test -x /usr/local/cpanel/bin/tail-check && /usr/local/cpanel/bin/tail-check
#11,26,41,56 * * * * /usr/local/cpanel/whostmgr/bin/dnsqueue > /dev/null 2>&1
#30 */4 * * * /usr/bin/test -x /scripts/update_db_cache && /scripts/update_db_cache
#45 */8 * * * /usr/bin/test -x /usr/local/cpanel/bin/optimizefs && /usr/local/cpanel/bin/optimizefs
#*/5 * * * * /usr/local/cpanel/bin/dcpumon >/dev/null 2>&1
#25 1 * * * /usr/local/cpanel/whostmgr/docroot/cgi/cpaddons_report.pl --notify

Delete Crontabs

sudo crontab -u mailman -r

Find INIT scripts with cpanel

This is the main startup script: /usr/local/cpanel/etc/init/startup

I had no idea ruby-on-rails was being controlled by cpanel.. sneaky bugger. You can tell by all of these advanced unix commands just how difficult it would be to uninstall cpanel, its totally like the Alien!

(1:3744)# find . ! -type d -print0|xargs -0 -I'F87' grep -Hi "cpan\|tailwat\|chkser" F87
./fastmail:# Author:       cPanel, Inc. 
./httpd:        HTTPD=/usr/local/cpanel/bin/chroothttpd
./cpanel:# cpanel8       Start Cpanel Services
./cpanel:# Author:       cPanel, Inc. 
./cpanel:# description: This is the cpanel webserver and chat.
./cpanel:# processname: cpaneld
./cpanel:# pidfile: /var/run/cpanel.pid
./cpanel:[ -f /usr/local/cpanel/etc/init/startup ] || exit 0
./cpanel:       if [ -f "/var/cpanel/smtpgidonlytweak" ]; then
./cpanel:       echo -n "Starting cPanel services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpsrvd
./cpanel:       echo -n "Starting cPanel brute force detector services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcphulkd
./cpanel:    echo -n "Starting cPanel dav services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startcpdavd
./cpanel:               daemon /usr/local/cpanel/etc/init/startcppop
./cpanel:       echo -n "Starting cPanel Chat services: "
./cpanel:               daemon /usr/local/cpanel/entropychat/entropychat
./cpanel:               daemon /usr/local/cpanel/bin/startmelange
./cpanel:                       /usr/local/cpanel/bin/startinterchange
./cpanel:       echo -n "Starting cPanel ssl services: "
./cpanel:       daemon /usr/local/cpanel/startstunnel
./cpanel:    echo -n "Starting cPanel Queue services: "
./cpanel:       daemon /usr/local/cpanel/etc/init/startqueueprocd
./cpanel:    echo -n "Starting tailwatchd: "
./cpanel:    daemon /usr/local/cpanel/libexec/tailwatchd --start
./cpanel:       echo -n "Starting cPanel Log services: "
./cpanel:       daemon /usr/local/cpanel/cpanellogd
./cpanel:    action "Starting mailman services: " /usr/local/cpanel/etc/init/startmailman
./cpanel:    action "Stopping tailwatchd: " /usr/local/cpanel/libexec/tailwatchd --stop
./cpanel:    action "Stopping cPanel services: " /usr/local/cpanel/etc/init/stopcpsrvd
./cpanel:       action "Stopping cPanel dav services: " /usr/local/cpanel/etc/init/stopcpdavd
./cpanel:       action "Stopping cPanel queue services: " /usr/local/cpanel/etc/init/stopqueueprocd
./cpanel:       action "Stopping cPanel brute force detector services: " /usr/local/cpanel/etc/init/stopcphulkd
./cpanel:               action "Stopping pop3 services: " /usr/local/cpanel/etc/init/stopcppop
./cpanel:       echo -n "Stopping cPanel log services: "
./cpanel:       killproc cpanellogd
./cpanel:       echo -n "Stopping cPanel Chat services: "
./cpanel:       action "Stopping cPanel ssl services: " /usr/local/cpanel/etc/init/stopstunnel
./cpanel:       action "Stopping mailman services: " /usr/local/cpanel/etc/init/stopmailman
./cpanel:       if [ -e "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl" ]; then
./exim:if [ -e "/etc/chkserv.d" ]; then
./exim:        for file in `ls /etc/chkserv.d`
./exim:            if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./exim:    if [ -x "/usr/local/cpanel/etc/init/startspamd" ]; then
./exim:        /usr/local/cpanel/etc/init/startspamd
./exim:        if [ ! -e "/usr/local/cpanel/libexec/tailwatchd" ]; then
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --stopall
./ror:  /usr/local/cpanel/bin/rormgr --startboot
./ror:  /usr/local/cpanel/bin/rormgr --statusall
./securetmp:# Author:       cPanel, Inc.

Turn off cpanel services

You should remove the below delete command and start by just disabling the inits by turning them off. Then reboot. Then delete. If your machine won't reboot, I told you so, Cpanel told you so, and likely your host told you so.

for S in cpanel ror securetmp fastmail exim; do R=$(command chkconfig --level 123456 $S off ||echo); R=$(command chkconfig --del $S ||echo); done

HTTPD

If you are running chrooted httpd then you'll need to make sure you don't delete your entire webserver on accident. Here's a relevant part from the /etc/init.d/httpd script.

# the path to your httpd binary, including options if necessary
if [ -e "/etc/chroothttpd" ]; then
        HTTPD=/usr/local/cpanel/bin/chroothttpd
else
        HTTPD=/usr/local/apache/bin/httpd
fi

Finding files owned by cpanel

Some super cool bash commands in this post.. let's start with one to find all the files and folders on your machine owned by cpanel. Check your /etc/passwd file for your machines specific usernames and groups. This command saves all the filenames to ~/cpanel-files-backup.txt, which is used by tar next to create a backup of all of them.

{ find / -mount -depth -maxdepth 150 \( -group cpanel -o   -group cpanel-phpmyadmin -o -group cpanel-phppgadmin   -o -group cpanelphpmyadmin   -o -group cpanelphppgadmin   -o -group cpanelhorde   -o -group cpanelroundcube \) -print; find / -mount -depth -maxdepth 150 \( -user cpanel -o   -user cpanel-phpmyadmin -o -user cpanel-phppgadmin   -o -user cpanelphpmyadmin   -o -user cpanelphppgadmin   -o -user cpanelhorde   -o -user cpanelroundcube \) -print; } > ~/cpanel-files-backup.txt

Here's another way to search directories.

 grep --color=always -Hir cpanel /var

Create the Backup

Note that you must have the latest version of tar for this exact command, also you should backup /var/cpanel and /usr/local/cpanel and /etc and heck the whole machine why dontcha!

tar -T ~/cpanel-files-backup.txt -cvz --checkpoint=1000 --checkpoint-action="ttyout=\rHit %s checkpoint #%u" -f /cpanel-files-backup.tgz --totals

Remove Files

Once you do this your upstream without a paddle, you better make sure you know what you're doing with this. This removes all those files.

cat ~/cpanel-files-backup.txt | xargs -I'F87' rm -vfr F87

Additionally you will want to remove /usr/local/cpanel and /var/cpanel - What I always do when running as root is alias my rm command to instead simply move the files to a .trash folder. That way if something goes bork you have a better chance at fixing it.

Find Group-Owned Files

find / -mount -depth -maxdepth 150 \
\(  -group cpanel -o \
  -group cpanel-phpmyadmin \
  -o -group cpanel-phppgadmin \
  -o -group cpanelphpmyadmin \
  -o -group cpanelphppgadmin \
  -o -group mailman \
  -o -group cpanelhorde \
  -o -group cpanelroundcube \
\) -fprintf /root/cpanel-group-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'

Find User-Owned Files

find / -mount -depth -maxdepth 150 \(
  -user cpanel \
  -o -user cpanel-phpmyadmin \
  -o -user cpanel-phppgadmin \
  -o -user cpanelphpmyadmin \
  -o -user cpanelphppgadmin \
  -o -user mailman \
  -o -user cpanelhorde \
  -o -user cpanelroundcube
\) -fprintf /root/cpanel-users-files.log '%#8k %#5m %11M %#10u:%-10g %-5U:%-5G %p %f %Y %F\n'

       4  0755  drwxr-xr-x     cpanel:cpanel     32002:32004 /var/cpanel/userhomes/cpanel cpanel d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/tmp tmp d reiserfs
       4  0644  -rw-r--r-- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanel-phpmyadmin:cpanel-phpmyadmin 32005:32007 /var/cpanel/userhomes/cpanel-phpmyadmin cpanel-phpmyadmin d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/sessions sessions d reiserfs
       4  0644  -rw-r--r-- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel/caches caches d reiserfs
       4  0700  drwx------ cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/.cpanel .cpanel d reiserfs
       4  0750  drwxr-x--- cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin/mail mail d reiserfs
       4  0711  drwx--x--x cpanelphppgadmin:cpanelphppgadmin 32009:32011 /var/cpanel/userhomes/cpanelphppgadmin cpanelphppgadmin d reiserfs
       4  0750  drwxr-x--- cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/mail mail d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube/sessions sessions d reiserfs
       4  0711  drwx--x--x cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/userhomes/cpanelroundcube cpanelroundcube d reiserfs
       4  0644  -rw-r--r--     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists/default.cache default.cache f reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches/featurelists featurelists d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel/caches caches d reiserfs
       4  0700  drwx------     cpanel:cpanel     32002:32004 /var/cpanel/.cpanel .cpanel d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/tmp tmp d reiserfs
       4  0700  drwx------ cpanelroundcube:cpanelroundcube 514  :514   /var/cpanel/roundcube/log log d reiserfs

Find Permissions

cat ~/cpanel-group-files.log ~/cpanel-users-files.log |tr -s '\000 \t'|cut -d' ' -f3|sort -u

Find files tailwatchd

(1:3732)# $NICE find ${1:-`pwd`} -mount -name '*tailwatch*'
/usr/local/cpanel/libexec/tailwatchd
/usr/local/cpanel/libexec/tailwatch
/usr/local/cpanel/libexec/tailwatch/tailwatchd
/usr/local/cpanel/etc/init/scripts/freebsd/tailwatchd.sh
/usr/local/cpanel/etc/init/scripts/trustix/tailwatchd
/usr/local/cpanel/etc/init/scripts/centos/tailwatchd
/usr/local/cpanel/etc/init/scripts/suse/tailwatchd
/usr/local/cpanel/etc/init/scripts/caos/tailwatchd
/usr/local/cpanel/etc/init/scripts/whitebox/tailwatchd
/usr/local/cpanel/etc/init/scripts/mandrake/tailwatchd
/usr/local/cpanel/etc/init/scripts/debian/tailwatchd
/usr/local/cpanel/etc/init/scripts/redhat/tailwatchd
/usr/local/cpanel/etc/init/scripts/fedora/tailwatchd
/usr/local/cpanel/etc/init/stoptailwatchd
/usr/local/cpanel/etc/init/starttailwatchd
/usr/local/cpanel/bin/tailwatchd
/usr/local/cpanel/logs/tailwatchd_log
/var/log/cpanel/tailwatchd_log
/var/cpanel/log_rotation/cp_tailwatchd_log.cpanellogd
/var/cpanel/tailwatch.positions
/var/run/tailwatchd.pid
/etc/chkserv.d/tailwatchd
/scripts/restartsrv_tailwatchd

Delete cpanel Users/Groups

for U in cpanel-phpmyadmin cpanel-phppgadmin cpanelphpmyadmin cpanelphppgadmin cpanelhorde cpanelroundcube machbuild; do userdel -fr $U; groupdel $U; done

Check for broken symlinks

find / -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'

Especially check /etc

$ find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p'
/etc/ftpd-rsa.pem                   broken symbolic link to `/var/cpanel/ssl/ftp/ftpd-rsa.pem'
/etc/rc.d/rc1.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc1.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc1.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc3.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc3.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc6.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc6.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc5.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc5.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc2.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc2.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc2.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc4.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/rc.d/rc4.d/S80chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc4.d/S80antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K10chkservd         broken symbolic link to `../init.d/chkservd'
/etc/rc.d/rc0.d/K30antirelayd       broken symbolic link to `../init.d/antirelayd'
/etc/rc.d/rc0.d/K80dcc              broken symbolic link to `../init.d/dcc'
/etc/authlib/authProg               broken symbolic link to `/usr/local/cpanel/bin/courier-auth'

And delete if you are sure

find /etc -mount -depth -type l -print0 |xargs -0 -P0 -I'F87' file -s 'F87' | sed -n '/: broken symbolic link to/p' |cut -d' ' -f1|xargs -I'F87' rm -rvf 'F87'

Reinstall CSF

The only thing I actually used that came with cpanel is the CSF/LFD Firewall package, which is a fantastic piece of software. I had to reinstall this, and to get it working without cpanel add the following line to the csf.conf

GENERIC = "1"

Thats It

Now once you've cleaned up everything, you should try everything conceivable to get an error before rebooting. Like you should start and stop every service in /etc/init.d/, you should use telinit to check various runlevels (which keeps your sshd connection still live). Go all out, should take at least a full hour.

Another thing I like to do is rebuild alot of my source-built software again in case anything got messed up. I upgrade perl from cpanels 5.8.8 to 5.10, which is pretty thorough, and you know, reinstall anything else I think I might need. One of the benefits of compiling your own software is all I have to do is cd to the source directory and type make -B && ( { make test || make check || make checks || make tests; } || echo ) && sudo make install and that's it. The tests/checks are optional of course.

If anyone actually ever reads this and does it, please share your advice here.. everybody knows we need it! Good Luck

HOWTO: Uninstall CPANEL over SSH originally appeared on AskApache.com

Enhanced printenv Script for Server Debugging

AskApache — Thu, 14 Apr 2011 19:05:17 +0000

AskApache.com

Every Apache server comes pre-shipped with an old cgi script named printenv. It's useful and used for printing out the environment variables from your servers point of view. Basically you save this file where you can then call it from your browser, like site.com/printenv.cgi -- then it will be executing as the user running the web server, nobody, apache, etc.. So this tells you all sorts of interesting things about your server, especially if you want/need to know about the users, groups, and file permissions.

Enabling CGI

You need to do 1 of 2 things to use this:

Make it executable and in your server directory
Use a trick to cause your server to execute it regardless

Htaccess CGI

If you have htaccess then you most likely can get this to work by giving it an execute bit with chmod or from your ftp client, and then use htaccess like this:

Options +ExecCGI Indexes +FollowSymLinks
AddHandler cgi-script .cgi .pl .sh
 
Order Deny,Allow
Deny from All
Allow from 127.0.0.1 ADDYOURIPHERE env=REDIRECT_STATUS
Satisfy All

Search around on this site for php.cgi tricks if you can't seem to get it. Basically you can try stuff like making it an errordocument, using other directives like forcetype, AddType, and even try stuff like mod_security's upload binary setting.

You do not want anyone other than you ever looking at the output of this. It's so full of details about your machine that it would be a huge security problem.

The AskApache Printenv

#!/bin/sh
echo -e "Content-type: text/plain\n\n"
 
# FUNCTIONS
################################################################################################################
function __A ()
{ local __a __i __z;for __a;do __z=\${!${__a}*};for __i in `eval echo "${__z}"`;do echo -e "$__i: ${!__i}";done;done; }
 
function __S ()
{ local L IFS=';';while read -r L;do builtin printf "${#L}@%s\n" "$L";done|sort -n|sed -u 's/^[^@]*//'; }
 
function __P ()
{ local l=`builtin printf %${2:-$__WIDTH}s` && echo -e "${l// /${1:-=}}"; }
 
function __T ()
{ echo -e "\n\n+`__P -`+\n| $*\n+`__P '='`+"; }
 
function __M ()
{ echo -e " >>> $M" $*; }
 
function __H ()
{ command builtin type $1 &>/dev/null && local a="yes" || return 1; }
 
function LE ()
{
 [[ ! -r /proc/${1:-$$}/limits ]] && return;
 sed -e '1z;s/ *$//;:a;$!N;s/\nM.. [a-z]* [a-z]* [a-z]* \{1,\}\([^ ]*\) *\([^ ]*\) *[a-z]* */\1:\2 /;ta;s/u\w\+d/u/g;s/ *$//;s/ / | /g;s/\([^:]\+\):\1/\1:=/g' /proc/${1:-$$}/limits;
}
function LH ()
{
 [[ ! -r /proc/${1:-$$}/limits ]] && return;
 sed -e '1z;s/ *$//;:a;$!N;s/\nM.. \([a-z]\+ [a-z]* [a-z]*\) \{1,\}\([^ ]*\) *\([^ ]*\) *\([a-z]*\) */:\1/;ta;s/ *:/:/g;s/size/sz/g;s/file/f/g;s/ p\w\+y/ pri/g;s/memory/mem/g;s/p\w\+s/procs/g;s/^://;s/:/ | /g' /proc/${1:-$$}/limits
}
 
# CUSTOM SETTINGS
################################################################################################################
__WIDTH=170
LC_COLLATE=C LC_CTYPE=C LC_ALL=C
 
# RUNTIME SETUP
#################################################################################################################
shopt -s dotglob nocaseglob extglob
 
# -C If set, disallow existing regular files to be overwritte
# -f Disable file name generation (globbing
# -e Exit immediately if a command exits with a non-zero status
# -B enable brace expansion
# +H disable History
set -C +f +H -B
 
# make sure we dont create any files
umask 0177
 
# redirect everything to output (no logs or stderr is used)
exec 2>/dev/null
 
# MAIN EXECUTION
#################################################################################################################
{
 __T "EXPANDING PATH"
 {
  __M "ORIG PATH:$PATH"
  PATH=$PATH:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/libexec:/usr/local/apache/bin
  for t in ${PATH//:/ };
  do
   [[ -d "$t" ]] && sed -n -e "/:${t//\//\\/}:/Q1" <<< ":${p:=}:" && p=$p:$t || continue;
  done
  PATH=${p/:/}:.
  __M "NEW PATH:$PATH"
 }
 
 __T "USER INFO"
 {
  __M "UMASK: `(umask 2>/dev/null)` ( `(umask -S 2>/dev/null)` )"
  __H uname && __M "UNAME: `eval echo $(uname -a 2>/dev/null)`"
  __H whoami && __M "WHOAMI: `(whoami 2>/dev/null)`"
  __H id && __M "ID: `(id 2>/dev/null)`"
  __H logname && __M "LOGNAME: `(logname 2>/dev/null)`"
  __H groups && __M "GROUPS: `(groups 2>/dev/null)`"
  echo -e "\n\n"
 }
 
 if __H who;
 then
  __T "LOGGED ON USERS"
  {
   (who -a 2>/dev/null)
  }
 fi;
 
 if [[ -r /etc/passwd ]];
 then
  __T "/etc/passwd"
  {
   (cat /etc/passwd)
  }
 fi;
 
 if __H ulimit;
 then
  __T "USER LIMITS"
  {
   ulimit -a
  }
 fi
 
 if [[ -d /dev ]] && __H ls;
 then
  __T "/dev Directory"
  {
   ( ls -vlaph /dev 2>/dev/null )
  }
 fi;
 
 if [[ -d /proc ]];
 then
  __T "CURRENT PROCESS LIMITS"
  {
   for p in `echo /proc/[0-9]*/limits`
   do
    pid=${p:6:$((${#p}-13))}
    [[ $pid == $PPID || $pid == $$ ]] && continue;
    echo -e "\n/proc/$pid:"
    sed '1s/\x00/ /g;n;s/\x00/\n/g;/.\{2,\}/!d' /proc/$pid/cmdline $p 2>/dev/null
   done
  }
 
  __T "CURRENT PROCESS CMDLINE"
  {
   for p in `echo /proc/[0-9]*/cmdline`;
   do
    pid=${p:6:$((${#p}-13))}
    [[ $pid == $PPID || $pid == $$ ]] && continue;
    __M "[ /proc/$pid ]";
    sed 's/\x00/ /g;G' $p 2>/dev/null
   done
  }
 fi
 
 __T "IP INFORMATION"
 {
  __H ip && __M "IP:" && (ip -o -f inet addr 2>/dev/null) | sed 's/^.*inet \([0-9.]*\).*$/\1/g';
  __H nmap && __M "NMAP:" && (nmap --iflist 2>/dev/null) | sed 1,4d | sed -n '/ethernet/s/^.*) \([0-9.]*\).*$/\1/gp';
  __H ifconfig && __M "IFCONFIG:" && (ifconfig -a 2>/dev/null) | sed -n '/inet a/s/^.*addr:\([0-9.]*\).*$/\1/gp';
  [[ -f "$HOME/.cpanel/datastore/_sbin_ifconfig_-a" ]] && __M "CPANEL CACHE:" && sed -e '/inet/!d; s/.*addr:\([0-9\.]*\).*/\1/g' "$HOME/.cpanel/datastore/_sbin_ifconfig_-a" | sort -u
 }
 
 __T "ROUTE / INTERFACE INFO"
 {
  __H route && __M "ROUTE" && (route -nv 2>/dev/null)
  __H ip && ( ip rule && ip route && ip address ) 2>/dev/null
  __H ifconfig && (ifconfig -a 2>/dev/null)
 }
 
 __T "CGI/1.0 test script report:"
 {
  __A SERVER REQUEST GET SERVER PATH REMOTE AUTH CONTENT HTTP TZ GATEWAY QUERY MO
  echo -e "\n\n"
 }
 
 __T "HIDDEN VARIABLES"
 {
  __A {a..z} {A..Z} _{0..9} _{A..Z} _{a..z} | cat -Tsv 2>/dev/null
  echo -e "\n\n"
 }
 
 __T "DECLARE INFO"
 {
  for i in "r" "i" "a" "x" "t" "-";
  do
   builtin eval declare -$i && echo;
  done | sed 's/^declare //' | cat -Tsv 2>/dev/null
  echo -e "\n\n"
 }
 
 __T "SHELL OPTIONS"
 {
  __A SHELLOPTS BASHOPTS
  echo -e "\$-: $-"
  __P '-' && builtin shopt -s -p
  __P '-' && builtin shopt -u -p
  echo -e "\n\n"
 }
 
 __T "ENV AND EXPORT"
 {
  __H env && command env | cat -Tsv 2>/dev/null && __P '-'
  builtin export | cat -Tsv 2>/dev/null
  echo -e "\n\n"
 }
 
 if __H perl;
 then
  __T "PERL VARIABLES"
  {
   perl -e'foreach $v (sort(keys(%ENV))) {$vv = $ENV{$v};$vv =~ s|\n|\\n|g;$vv =~ s|"|\\"|g;print "${v}=\"${vv}\"\n"}' | cat -Tsv 2>/dev/null
   echo -e "\n\n"
  }
 fi
 
} | fold - -w$(($__WIDTH+3))
exit $?

Enhanced printenv Script for Server Debugging originally appeared on AskApache.com

Terminal Escape Code Zen

AskApache — Wed, 13 Apr 2011 04:00:27 +0000

AskApache.com

Lightning Strikes Twice!

LOL..

3rd Dimension Broken with Bash!

So 3D... it'll hurt you eyes! lol.. Man I am cracking up here. Ha but seriously those really do look 3D for terminal though.. I am actually really impressed. Sweet. Here is the little function I wrote to output that grey marble.

Just a word to the wise, start learning and going over some of these concepts, especially the code used in functions, I will be back in a followup that details actually using this stu.

This function is one of my favorites because it is so fast and useful. Like when designing a 256color prompt.

aa_256 ()
{
    local o x=`tput op` y=`printf %$((${COLUMNS}-6))s`;
    for i in {0..256};
    do
        o=00$i;
        echo -e ${o:${#o}-3:3} `tputm "setaf $i" "setab $i"`${y// /=}$x;
    done
}

Ya this is actually not very helpful or useful, but there you have it.

tputm ()
{
    local a;
    for a in "$@";
    do
        echo -en "${a}\n";
    done | tput -S
}

Some people call this function the grey bringer of death. Not really.

a256 ()
{
    ( x=`tput op` y=`printf %$((${COLUMNS}-6))slocal `;
    for i in {242..232} 232 232;
    do
        echo -en "`tput setaf $i;tput setab $i`${y}${x}`tput op`";
    done )
}

Fix screen

Helpful Ncurses Programs

Program Name	Description	Example Usage
infotocap	convert a terminfo description into a termcap description
tic	the terminfo entry-description compiler
toe	table of (terminfo) entries	`toe -a\|sort -d`
infocmp	compare or print out terminfo descriptions	`infocmp -a -L -1 -T -x`
capconvert	automated conversion from termcap to terminfo
stty	prints or changes terminal characteristics, such as baud rate.
clear	clears the terminal's screen
capconvert	automated conversion from termcap to terminfo

ALWAYS check out my functions!

This function will print out the terminal, show it's colors, etc.. I have some really nice ones in this article that I use for tmux, screen, and that sort of thing.

c ()
{
    tput clear;
    pm "$TERM: [colors:`tput colors`/`tput pairs`]";
    RC=`tput op` L1=$(L '=' $(( ${COLUMNS} - 25 )));
    for i in `seq ${1:-0} ${2:-16}`;
    do
        o="  $i";
        echo -e " ${o:${#o}-3:3} `tput setaf $i;tput setab $i`${L1}${RC}";
    done
}

Standard Capabilities

X.364 and iBCS2

\033c - RIS - full reset
\0337 - SC - save cursor
\0338 - RC - restore cursor
\033[r - RSR - not an X.364 mnemonic
\033[m - SGR0 - not an X.364 mnemonic
\033[2J - ED2 - clear page

Specified by ISO 2022

\033(0 - ISO DEC G0 - enable DEC graphics for G0
\033(A - ISO UK G0 - enable UK chars for G0
\033(B - ISO US G0 - enable US chars for G0
\033)0 - ISO DEC G1 - enable DEC graphics for G1
\033)A - ISO UK G1 - enable UK chars for G1
\033)B - ISO US G1 - enable US chars for G1

ISO 2022 charset switching:

ISO/IEC 2022:1994
"Character code structure and extension techniques"
ECMA-035
code structure
- =00..=1F C0 set of control characters
- =20..=7F G0 set of 94 or 96 graphic characters
- =80..=9F C1 set of control characters "C1 controls"
- =A0..=FF G1 set of 94 or 96 graphic characters
- ... G2 and G3
registration authority
- ISO/IEC 2375:1985 Procedure for registration of escape sequences
- "ECMA registry"
- ISO-IR registration number
- ftp://dkuug.dk/i18n/iso2375reg.txt
- Internaltional register of coded character sets to be used with escape sequences
- ISO/IEC JTC1/SC2/WG3
ISO-2022 terminals from Siemens
recode iso-2022..utf-8 not yet implemented
CJK charsets

DEC private controls widely supported by emulators

\033= - DECPAM - application keypad mode
\033> - DECPNM - normal keypad mode
\033< - DECANSI - enter ANSI mode
\033[!p - DECSTR - soft reset
\033 F - S7C1T - 7-bit controls

ECMA modes

ISO 2022

2 - AM - keyboard action mode
4 - IRM - insert/replace mode
12 - SRM - send/receive mode
20 - LNM - linefeed mode

DEC modes

1 - CKM - application cursor keys
2 - ANM - set VT52 mode
3 - COLM - 132-column mode
4 - SCLM - smooth scroll
5 - SCNM - reverse video mode
6 - OM - origin mode
7 - AWM - wraparound mode
8 - ARM - auto-repeat mode

ECMA attribute sequences

0 - NORMAL - normal
1 - +BOLD - bold on
2 - +DIM - dim on
3 - +ITALIC - italic on
4 - +UNDERLINE - underline on
5 - +BLINK - blink on
6 - +FASTBLINK - fastblink on
7 - +REVERSE - reverse on
8 - +INVISIBLE - invisible on
9 - +DELETED - deleted on
10 - MAIN-FONT - select primary font
11 - ALT-FONT-1 - select alternate font 1
12 - ALT-FONT-2 - select alternate font 2
13 - ALT-FONT-3 - select alternate font 3
14 - ALT-FONT-4 - select alternate font 4
15 - ALT-FONT-5 - select alternate font 5
16 - ALT-FONT-6 - select alternate font 6
17 - ALT-FONT-7 - select alternate font 7
18 - ALT-FONT-1 - select alternate font 1
19 - ALT-FONT-1 - select alternate font 1
20 - FRAKTUR - Fraktur font
21 - DOUBLEUNDER - double underline
22 - -DIM - dim off
23 - -ITALIC - italic off
24 - -UNDERLINE - underline off
25 - -BLINK - blink off
26 - -FASTBLINK - fastblink off
27 - -REVERSE - reverse off
28 - -INVISIBLE - invisible off
29 - -DELETED - deleted off

Init strings

Name	Description
`is1`	init_1string
`is2`	init_2string
`is3`	init_3string
`rs1`	reset_1string
`rs2`	reset_2string
`rs3`	reset_3string
`smcup`	enter_ca_mode
`rmcup`	exit_ca_mode

Cap strings

Name	Description
so	String of commands to enter standout mode.
se	String of commands to leave standout mode.
sg	Numeric capability, the width on the screen of the magic cookie. This capability is absent in terminals that record appearance modes character by character.
ms	Flag whose presence means that it is safe to move the cursor while the appearance modes are not in the normal state. If this flag is absent, programs should always reset the appearance modes to normal before moving the cursor.
xs	Flag whose presence means that the only way to reset appearance modes already on the screen is to clear to end of line. On a per-character terminal, you must clear the area where the modes are set. On a magic cookie terminal, you must clear an area containing the cookie. See the discussion above.
xt	Flag whose presence means that the cursor cannot be positioned right in front of a magic cookie, and that seis a command to delete the next magic cookie following the cursor. See discussion above.
mb	String of commands to enter blinking mode.
md	String of commands to enter double-bright mode.
mh	String of commands to enter half-bright mode.
mk	String of commands to enter invisible mode.
mp	String of commands to enter protected mode.
mr	String of commands to enter reverse-video mode.
me	String of commands to turn off all appearance modes, including standout mode and underline mode. On some terminals it also turns off alternate character set mode; on others, it may not. This capability must be present if any of mb... mris present.
as	String of commands to turn on alternate character set mode. This mode assigns some or all graphic characters an alternate picture on the screen. There is no standard as to what the alternate pictures look like.
ae	String of commands to turn off alternate character set mode.
sa	String of commands to turn on an arbitrary combination of appearance modes. It accepts 9 parameters, each of which controls a particular kind of appearance mode. A parameter should be 1 to turn its appearance mode on, or zero to turn that mode off. Most terminals do not support the sacapability, even among those that do have various appearance modes. The nine parameters are, in order, `standout` `underline` `reverse` `blink` `half-bright` `double-bright` `blank` `protect` `alt char set`

Variable and Function Index

For stty

BC: tgoto
ospeed: Output Padding
PC: Output Padding
tgetent: Find
tgetflag: Interrogate
tgetnum: Interrogate
tgetstr: Interrogate
tgoto: tgoto
tparam: tparam
tputs: Output Padding
UP: tgoto

Summary of Capability Names

Here are all the terminal capability names in alphabetical order with a brief description of each. For cross references to their definitions, see the index of capability names.

Name	Description
ae	String to turn off alternate character set mode.
al	String to insert a blank line before the cursor.
AL	String to insert `n` blank lines before the cursor.
am	Flag: output to last column wraps cursor to next line.
as	String to turn on alternate character set mode.like.
bc	Very obsolete alternative name for the lecapability.
bl	String to sound the bell.
bs	Obsolete flag: ASCII backspace may be used for leftward motion.
bt	String to move the cursor left to the previous hardware tab stop column.
bw	Flag: leat left margin wraps to end of previous line.
CC	String to change terminal's command character.
cd	String to clear the line the cursor is on, and following lines.
ce	String to clear from the cursor to the end of the line.
ch	String to position the cursor at column `c` in the same line.
cl	String to clear the entire screen and put cursor at upper left corner.
cm	String to position the cursor at line `l`, column `c`.
CM	String to position the cursor at line `l`, column `c`, relative to display memory.
co	Number: width of the screen.
cr	String to move cursor sideways to left margin.
cs	String to set the scroll region.
cS	Alternate form of string to set the scroll region.
ct	String to clear all tab stops.
cv	String to position the cursor at line `l` in the same column.
da	Flag: data scrolled off top of screen may be scrolled back.
db	Flag: data scrolled off bottom of screen may be scrolled back.
dB	Obsolete number: msec of padding needed for the backspace character.
dc	String to delete one character position at the cursor.
dC	Obsolete number: msec of padding needed for the carriage-return character.
DC	String to delete `n` characters starting at the cursor.
dF	Obsolete number: msec of padding needed for the formfeed character.
dl	String to delete the line the cursor is on.
DL	String to delete `n` lines starting with the cursor's line.
dm	String to enter delete mode.
dN	Obsolete number: msec of padding needed for the newline character.
do	String to move the cursor vertically down one line.
DO	String to move cursor vertically down `n` lines.
ds	String to disable the display of the status line.
dT	Obsolete number: msec of padding needed for the tab character.
ec	String of commands to clear `n` characters at cursor.
ed	String to exit delete mode.
ei	String to leave insert mode.
eo	Flag: output of a space can erase an overstrike.
es	Flag: other display commands work while writing the status line.
ff	String to advance to the next page, for a hardcopy terminal.
fs	String to move the cursor back from the status line to its previous position (outside the status line).
gn	Flag: this terminal type is generic, not real.
hc	Flag: hardcopy terminal.
hd	String to move the cursor down half a line.
ho	String to position cursor at upper left corner.
hs	Flag: the terminal has a status line.
hu	String to move the cursor up half a line.
hz	Flag: terminal cannot accept ~as output.
i1	String to initialize the terminal for each login session.
i3	String to initialize the terminal for each login session.
ic	String to insert one character position at the cursor.
IC	String to insert `n` character positions at the cursor.
if	String naming a file of commands to initialize the terminal.
im	String to enter insert mode.
in	Flag: outputting a space is different from moving over empty positions.
ip	String to output following an inserted character in insert mode.
is	String to initialize the terminal for each login session.
it	Number: initial spacing between hardware tab stop columns.
k0	String of input sent by function key 0 or 10.
k1 ... k9	Strings of input sent by function keys 1 through 9.
K1 ... K5	Strings sent by the five other keys in 3-by-3 array with arrows.
ka	String of input sent by the “clear all tabs” key.
kA	String of input sent by the “insert line” key.
kb	String of input sent by the “backspace” key.
kC	String of input sent by the “clear screen” key.
kd	String of input sent by typing the down-arrow key.
kD	String of input sent by the “delete character” key.
ke	String to make the function keys work locally.
kE	String of input sent by the “clear to end of line” key.
kF	String of input sent by the “scroll forward” key.
kh	String of input sent by typing the “home-position” key.
kH	String of input sent by the “home down” key.
kI	String of input sent by the “insert character” or “enter insert mode” key.
kl	String of input sent by typing the left-arrow key.
kL	String of input sent by the “delete line” key.
km	Flag: the terminal has a Meta key.
kM	String of input sent by the “exit insert mode” key.
kn	Numeric value, the number of numbered function keys.
kN	String of input sent by the “next page” key.
ko	Very obsolete string listing the terminal's named function keys.
kP	String of input sent by the “previous page” key.
kr	String of input sent by typing the right-arrow key.
kR	String of input sent by the “scroll reverse” key.
ks	String to make the function keys transmit.
kS	String of input sent by the “clear to end of screen” key.
kt	String of input sent by the “clear tab stop this column” key.
kT	String of input sent by the “set tab stop in this column” key.
ku	String of input sent by typing the up-arrow key.
l0	String on keyboard labelling function key 0 or 10.
l1 ... l9	Strings on keyboard labelling function keys 1 through 9.
le	String to move the cursor left one column.
LE	String to move cursor left `n` columns.
li	Number: height of the screen.
ll	String to position cursor at lower left corner.
lm	Number: lines of display memory.
LP	Flag: writing to last column of last line will not scroll.
mb	String to enter blinking mode.
md	String to enter double-bright mode.
me	String to turn off all appearance modes
mh	String to enter half-bright mode.
mi	Flag: cursor motion in insert mode is safe.
mk	String to enter invisible mode.
mm	String to enable the functioning of the Meta key.
mo	String to disable the functioning of the Meta key.
mp	String to enter protected mode.
mr	String to enter reverse-video mode.
ms	Flag: cursor motion in standout mode is safe.
nc	Obsolete flag: do not use ASCII carriage-return on this terminal.
nd	String to move the cursor right one column.
NF	Flag: do not use XON/XOFF flow control.
nl	Obsolete alternative name for the doand sfcapabilities.
ns	Flag: the terminal does not normally scroll for sequential output.
nw	String to move to start of next line, possibly clearing rest of old line.
os	Flag: terminal can overstrike.
pb	Number: the lowest baud rate at which padding is actually needed.
pc	String containing character for padding.
pf	String to terminate redirection of output to the printer.
po	String to redirect further output to the printer.
pO	String to redirect `n` characters ofoutput to the printer.
ps	String to print the screen on the attached printer.
rc	String to move to last saved cursor position.
RI	String to move cursor right `n` columns.
rp	String to output character `c` repeated `n` times.
rs	String to reset the terminal from any strange modes.
sa	String to turn on an arbitrary combination of appearance modes.
sc	String to save the current cursor position.
se	String to leave standout mode.
sf	String to scroll the screen one line up.
SF	String to scroll the screen `n` lines up.
sg	Number: width of magic standout cookie. Absent if magic cookies are not used.
so	String to enter standout mode.
sr	String to scroll the screen one line down.
SR	String to scroll the screen `n` line down.
st	String to set tab stop at current cursor column on all lines. programs.
ta	String to move the cursor right to the next hardware tab stop column.
te	String to return terminal to settings for sequential output.
ti	String to initialize terminal for random cursor motion.
ts	String to move the terminal cursor into the status line.
uc	String to underline one character and move cursor right.
ue	String to turn off underline mode
ug	Number: width of underlining magic cookie. Absent if underlining doesn't use magic cookies.
ul	Flag: underline by overstriking with an underscore.
up	String to move the cursor vertically up one line.
UP	String to move cursor vertically up `n` lines.
us	String to turn on underline mode
vb	String to make the screen flash.
ve	String to return the cursor to normal.
vi	String to make the cursor invisible.
vs	String to enhance the cursor.
wi	String to set the terminal output screen window.
ws	Number: the width of the status line.
xb	Flag: superbee terminal.
xn	Flag: cursor wraps in a strange way.
xs	Flag: clearing a line is the only way to clear the appearance modes of positions in that line (or, only way to remove magic cookies on that line).
xt	Flag: Teleray 1061; several strange characteristics.

Variable and Function Index

BC: tgoto
ospeed: Output Padding
PC: Output Padding
tgetent: Find
tgetflag: Interrogate
tgetnum: Interrogate
tgetstr: Interrogate
tgoto: tgoto
tparam: tparam
tputs: Output Padding
UP: tgoto

ae: Standout
AL: Insdel Line
al: Insdel Line
am: Wrapping
as: Standout
bc: Cursor Motion
bl: Bell
bs: Cursor Motion
bt: Cursor Motion
bw: Cursor Motion
CC: Basic
cd: Clearing
ce: Clearing
ch: Cursor Motion
cl: Clearing
CM: Cursor Motion
cm: Cursor Motion
co: Screen Size
cr: Cursor Motion
cS: Scrolling
cs: Scrolling
ct: Initialization
cv: Cursor Motion
da: Scrolling
dB: Pad Specs
db: Scrolling
dC: Pad Specs
DC: Insdel Char
dc: Insdel Char
dF: Pad Specs
DL: Insdel Line
dl: Insdel Line
dm: Insdel Char
dN: Pad Specs
DO: Cursor Motion
do: Cursor Motion
ds: Status Line
dT: Pad Specs
ec: Clearing
ed: Insdel Char
ei: Insdel Char
eo: Basic
es: Status Line
ff: Cursor Motion
fs: Status Line
gn: Basic
hc: Basic
hd: Half-Line
ho: Cursor Motion
hs: Status Line
hu: Half-Line
hz: Basic
i1: Initialization
i3: Initialization
IC: Insdel Char
ic: Insdel Char
if: Initialization
im: Insdel Char
in: Insdel Char
ip: Insdel Char
is: Initialization
it: Initialization
K1...K5: Keypad
k1...k9: Keypad
kA...kT: Keypad
ka...ku: Keypad
km: Meta Key
l0...l9: Keypad
LE: Cursor Motion
le: Cursor Motion
li: Screen Size
ll: Cursor Motion
lm: Scrolling
LP: Wrapping
mb: Standout
md: Standout
me: Standout
mh: Standout
mi: Insdel Char
mk: Standout
mm: Meta Key
mo: Meta Key
mp: Standout
mr: Standout
ms: Underlining
ms: Standout
nc: Cursor Motion
nd: Cursor Motion
NF: Initialization
nl: Cursor Motion
ns: Scrolling
nw: Cursor Motion
os: Basic
pb: Pad Specs
pc: Pad Specs
pf: Printer
pO: Printer
po: Printer
ps: Printer
rc: Cursor Motion
RI: Cursor Motion
rp: Basic
rs: Initialization
sa: Standout
sc: Cursor Motion
se: Standout
SF: Scrolling
sf: Scrolling
sg: Standout
so: Standout
SR: Scrolling
sr: Scrolling
st: Initialization
ta: Cursor Motion
te: Initialization
ti: Initialization
ts: Status Line
uc: Underlining
ue: Underlining
ug: Underlining
ul: Underlining
UP: Cursor Motion
up: Cursor Motion
us: Underlining
vb: Bell
ve: Cursor Visibility
vi: Cursor Visibility
vs: Cursor Visibility
wi: Windows
ws: Status Line
xb: Basic
xn: Wrapping
xs: Standout
xt: Standout
xt: Cursor Motion

Basic: Basic characteristics.
Screen Size: Screen size, and what happens when it changes.
Cursor Motion: Various ways to move the cursor.
Wrapping: What happens if you write a character in the last column.
Scrolling: Pushing text up and down on the screen.
Windows: Limiting the part of the window that output affects.
Clearing: Erasing one or many lines.
Insdel Line: Making new blank lines in mid-screen; deleting lines.
Insdel Char: Inserting and deleting characters within a line.
Standout: Highlighting some of the text.
Underlining: Underlining some of the text.
Cursor Visibility: Making the cursor more or less easy to spot.
Bell: Attracts user's attention; not localized on the screen.
Keypad: Recognizing when function keys or arrows are typed.
Meta Key: META acts like an extra shift key.
Initialization: Commands used to initialize or reset the terminal.
Pad Specs: Info for the kernel on how much padding is needed.
Status Line: A status line displays background information.
Half-Line: Moving by half-lines, for superscripts and subscripts.
Printer: Controlling auxiliary printers of display terminals.

Translated Term Capabilities

This is helpful so you can understand what a term cap does, by looking at the long name.

Tput Name	Terminfo Long Name
@7	key_end
AB	set_a_background
AF	set_a_foreground
AL	parm_insert_line
AX	AX
Co	max_colors
DC	parm_dch
DL	parm_delete_line
DO	parm_down_cursor
E0	E0
F1	key_f11
F2	key_f12
G0	G0
HC	5i
IC	parm_ich
Ic	initialize_color
Km	delete_line
LE	parm_left_cursor
NC	no_color_video
ND	MT
NP	NL
RI	parm_right_cursor
S0	S0
UP	parm_up_cursor
YA	NR
YC	YB
YE	YD
YG	YF
ac	acs_chars
ae	exit_alt_charset_mode
al	insert_line
am	auto_right_margin
as	enter_alt_charset_mode
back_color_erase	backspaces_with_bs
bl	bell
bs	eat_newline_glitch
bt	back_tab
bw	auto_left_margin
cb	clr_bol
cc	can_change
cd	clr_eos
ce	clr_eol
cl	clear_screen
cm	cursor_address
co	columns
col_addr_glitch	ceol_standout_glitch
cpi_changes_res	cr_cancels_micro_mode
cr	carriage_return
cs	change_scroll_region
ct	clear_all_tabs
cursor_up	scroll_reverse
da	crt_no_scrolling
db	dest_tabs_magic_smso
dc	delete_character
dl	key_mouse
do	cursor_down
eA	ena_acs
ei	exit_insert_mode
eo	erase_overstrike
es	generic_type
gn	gnu_has_meta_key
hard_copy	hard_cursor
has_meta_key	has_hardware_tabs
has_status_line	has_print_wheel
hl	hc
ho	cursor_home
hs	hue_lightness_saturation
im	enter_insert_mode
in	hz
insert_null_glitch	linefeed_is_newline
is	init_2string
it	init_tabs
k1	key_f1
k2	key_f2
k3	key_f3
k4	key_f4
k5	key_f5
k6	key_f6
k7	key_f7
k8	key_f8
k9	key_f9
k;	key_f10
kB	key_btab
kD	key_dc
kI	key_ic
kN	key_npage
kP	key_ppage
kb	cursor_left
kd	key_down
ke	keypad_local
kh	key_home
kl	key_left
kr	key_right
ks	keypad_xmit
ku	key_up
le	key_backspace
li	lines
mb	enter_blink_mode
md	enter_bold_mode
me	exit_attribute_mode
memory_above	lpi_changes_res
mi	km
move_insert_mode	move_standout_mode
mr	enter_reverse_mode
nc	memory_below
nd	cursor_right
needs_xon_xoff	no_correctly_working_cr
no_pad_char	no_esc_ctlc
non_rev_rmcup	non_dest_scroll_region
nw	newline
nx	ns
op	orig_pair
os	over_strike
pa	max_pairs
prtr_silent	return_does_clr_eol
pt	ms
r2	reset_2string
rc	restore_cursor
row_addr_glitch	semi_auto_right_margin
sa	set_attributes
sc	save_cursor
se	exit_standout_mode
sf	scroll_forward
so	enter_standout_mode
st	set_tab
ta	tab
te	exit_ca_mode
ti	enter_ca_mode
tilde_glitch	status_line_esc_ok
ue	exit_underline_mode
ul	transparent_underline
up	sr
us	enter_underline_mode
vb	flash_screen
ve	cursor_normal
vi	cursor_invisible
vs	cursor_visible
xn	ut
xo	xb
xr	xon_xoff
xt	xs

List of All Terminals

You can view this list with the following command:

$ toe -a|sort -d

VT100 keypad Diagram

VT100 LINK

Here's a diagram of the VT100 keypad keys with their bindings. The top line is the name of the key (some DEC keyboards have the keys labelled somewhat differently, like GOLD instead of PF1, but this is the most "official" name). The second line is the escape sequence it generates in Application Keypad mode (where "$" means the ESC character). The third line contains two items, first the mapping of the key in terminfo, and then in termcap.

   _______________________________________
  |   PF1   |   PF2   |   PF3   |   PF4   |
  |   $OP   |   $OQ   |   $OR   |   $OS   |
  |_kf1__k1_|_kf2__k2_|_kf3__k3_|_kf4__k4_|
  |    7         8         9         -    |
  |   $Ow   |   $Ox   |   $Oy   |   $Om   |
  |_kf9__k9_|_kf10_k;_|_kf0__k0_|_________|
  |    4    |    5    |    6    |    ,    |
  |   $Ot   |   $Ou   |   $Ov   |   $Ol   |
  |_kf5__k5_|_kf6__k6_|_kf7__k7_|_kf8__k8_|
  |    1    |    2    |    3    |         |
  |   $Oq   |   $Or   |   $Os   |  enter  |
  |_ka1__K1_|_kb2__K2_|_ka3__K3_|  $OM    |
  |         0         |   .     |         |
  |        $Op        |  $On    |         |
  |___kc1_______K4____|_kc3__K5_|_kent_@8_|

A better adaptation to modern keyboards such as the PC's, which have a dozen function keys and the keypad 2,4,6,8 keys are labeled with arrows keys, is to use the 5-key arrangement to model the arrow keys as suggested in the terminfo guidelines:

   _______________________________________
  |   PF1   |   PF2   |   PF3   |   PF4   |
  |   $OP   |   $OQ   |   $OR   |   $OS   |
  |_kf1__k1_|_kf2__k2_|_kf3__k3_|_kf4__k4_|
  |    7         8         9         -    |
  |   $Ow   |   $Ox   |   $Oy   |   $Om   |
  |_ka1__K1_|_________|_ka3__K3_|_________|
  |    4    |    5    |    6    |    ,    |
  |   $Ot   |   $Ou   |   $Ov   |   $Ol   |
  |_________|_kb2__K2_|_________|_________|
  |    1    |    2    |    3    |         |
  |   $Oq   |   $Or   |   $Os   |  enter  |
  |_kc1__K4_|_________|_kc3__K5_|  $OM    |
  |         0         |   .     |         |
  |        $Op        |  $On    |         |
  |___________________|_________|_kent_@8_|

SGR parameter values

Name	Description
`0`	default mode (attributes off)
`1`	bold
`2`	dim
`3`	italicized
`4`	underlined
`5`	slow blink
`6`	fast blink
`7`	reverse video
`8`	invisible
`9`	crossed-out (marked for deletion)
`10`	primary font [10 + n (n in 1..9) = nth alternative font]
`20`	Fraktur
`21`	double underline
`22`	turn off 2
`23`	turn off 3
`24`	turn off 4
`25`	turn off 5
`26`	proportional spacing
`27`	turn off 7
`28`	turn off 8
`29`	turn off 9
`30`	black fg
`31`	red fg
`32`	green fg
`33`	yellow fg
`34`	blue fg
`35`	magenta fg
`36`	cyan fg
`37`	white fg
`38`	set fg color as in CCIT T.416
`39`	set default fg color
`40`	black bg
`41`	red bg
`42`	green bg
`43`	yellow bg
`44`	blue bg
`45`	magenta bg
`46`	cyan bg
`47`	white bg
`48`	set bg color as in CCIT T.416
`39`	set default bg color
`50`	turn off 26
`51`	framed
`52`	encircled
`53`	overlined
`54`	turn off 51 & 52
`55`	not overlined
`56-59`	reserved
`61-65`	variable highlights for ideograms.

Sequence     Sequence                             Parameter   or
Mnemonic     Name              Sequence           Value      Mode   terminfo
-----------------------------------------------------------------------------
APC  Applicatn Program Command \E _                -         Delim  -
BEL  Bell *                    ^G                  -         -      bel
BPH  Break Permitted Here *    \E B                -         *      -
BS   Backpace *                ^H                  -         EF     -
CAN  Cancel *                  ^X                  -         -      -   (A)
CBT  Cursor Backward Tab       \E [ Pn Z           1         eF     cbt
CCH  Cancel Previous Character \E T                -         -      -
CHA  Cursor Horizntal Absolute \E [ Pn G           1         eF     hpa (B)
CHT  Cursor Horizontal Tab     \E [ Pn I           1         eF     tab (C)
CMD  Coding Method Delimiter * \E
CNL  Cursor Next Line          \E [ Pn E           1         eF     nel (D)
CPL  Cursor Preceding Line     \E [ Pn F           1         eF     -
CPR  Cursor Position Report    \E [ Pn ; Pn R      1, 1      -      -   (E)
CSI  Control Sequence Intro    \E [                -         Intro  -
CTC  Cursor Tabulation Control \E [ Ps W           0         eF     -   (F)
CUB  Cursor Backward           \E [ Pn D           1         eF     cub
CUD  Cursor Down               \E [ Pn B           1         eF     cud
CUF  Cursor Forward            \E [ Pn C           1         eF     cuf
CUP  Cursor Position           \E [ Pn ; Pn H      1, 1      eF     cup (G)
CUU  Cursor Up                 \E [ Pn A           1         eF     cuu
CVT  Cursor Vertical Tab       \E [ Pn Y           -         eF     -   (H)
DA   Device Attributes         \E [ Pn c           0         -      -
DAQ  Define Area Qualification \E [ Ps o           0         -      -
DCH  Delete Character          \E [ Pn P           1         eF     dch
DCS  Device Control String     \E P                -         Delim  -
DL   Delete Line               \E [ Pn M           1         eF     dl
DLE  Data Link Escape *        ^P                  -         -      -
DMI  Disable Manual Input      \E \                -         Fs     -
DSR  Device Status Report      \E [ Ps n           0         -      -   (I)
DTA  Dimension Text Area *     \E [ Pn ; Pn SPC T  -         PC     -
EA   Erase in Area             \E [ Ps O           0         eF     -   (J)
ECH  Erase Character           \E [ Pn X           1         eF     ech
ED   Erase in Display          \E [ Ps J           0         eF     ed  (J)
EF   Erase in Field            \E [ Ps N           0         eF     -
EL   Erase in Line             \E [ Ps K           0         eF     el  (J)
EM   End of Medium *           ^Y                  -         -      -
EMI  Enable Manual Input       \E b                          Fs     -
ENQ  Enquire                   ^E                  -         -      -
EOT  End Of Transmission       ^D                  -         *      -
EPA  End of Protected Area     \E W                -         -      -   (K)
ESA  End of Selected Area      \E G                -         -      -
ESC  Escape                    ^[                  -         -      -
ETB  End Transmission Block    ^W                  -         -      -
ETX  End of Text               ^C                  -         -      -
FF   Form Feed                 ^L                  -         -      -
FNK  Function Key *            \E [ Pn SPC W       -         -      -
GCC  Graphic Char Combination* \E [ Pn ; Pn SPC B  -         -      -
FNT  Font Selection            \E [ Pn ; Pn SPC D  0, 0      FE     -
GSM  Graphic Size Modify       \E [ Pn ; Pn SPC B  100, 100  FE     -   (L)
GSS  Graphic Size Selection    \E [ Pn SPC C       none      FE     -
HPA  Horz Position Absolute    \E [ Pn `           1         FE     -   (B)
HPB  Char Position Backward    \E [ j              1         FE     -
HPR  Horz Position Relative    \E [ Pn a           1         FE     -   (M)
HT   Horizontal Tab *          ^I                  -         FE     -   (N)
HTJ  Horz Tab w/Justification  \E I                -         FE     -
HTS  Horizontal Tab Set        \E H                -         FE     hts
HVP  Horz & Vertical Position  \E [ Pn ; Pn f      1, 1      FE     -   (G)
ICH  Insert Character          \E [ Pn @           1         eF     ich
IDCS ID Device Control String  \E [ SPC O          -         *      -
IGS  ID Graphic Subrepertoire  \E [ SPC M          -         *      -
IL   Insert Line               \E [ Pn L           1         eF     il
IND  Index                     \E D                -         FE     -
INT  Interrupt                 \E a                -         Fs     -
JFY  Justify                   \E [ Ps SPC F       0         FE     -
IS1  Info Separator #1 *       ^_                  -         *      -
IS2  Info Separator #1 *       ^^                  -         *      -
IS3  Info Separator #1 *       ^]                  -         *      -
IS4  Info Separator #1 *       ^\                  -         *      -
LF   Line Feed                 ^J                  -         -      -
LS1R Locking Shift Right 1 *   \E ~                -         -      -
LS2  Locking Shift 2 *         \E n                -         -      -
LS2R Locking Shift Right 2 *   \E }                -         -      -
LS3  Locking Shift 3 *         \E o                -         -      -
LS3R Locking Shift Right 3 *   \E |                -         -      -
MC   Media Copy                \E [ Ps i           0         -      -   (S)
MW   Message Waiting           \E U                -         -      -
NAK  Negative Acknowledge *    ^U                  -         *      -
NBH  No Break Here *           \E C                -         -      -
NEL  Next Line                 \E E                -         FE     nel (D)
NP   Next Page                 \E [ Pn U           1         eF     -
NUL  Null *                    ^@                  -         -      -
OSC  Operating System Command  \E ]                -         Delim  -
PEC  Pres. Expand/Contract *   \E Pn SPC Z         0         -      -
PFS  Page Format Selection *   \E Pn SPC J         0         -      -
PLD  Partial Line Down         \E K                -         FE     -   (T)
PLU  Partial Line Up           \E L                -         FE     -   (U)
PM   Privacy Message           \E ^                -         Delim  -
PP   Preceding Page            \E [ Pn V           1         eF     -
PPA  Page Position Absolute *  \E [ Pn SPC P       1         FE     -
PPB  Page Position Backward *  \E [ Pn SPC R       1         FE     -
PPR  Page Position Forward *   \E [ Pn SPC Q       1         FE     -
PTX  Parallel Texts *          \E [ \              -         -      -
PU1  Private Use 1             \E Q                -         -      -
PU2  Private Use 2             \E R                -         -      -
QUAD Typographic Quadding      \E [ Ps SPC H       0         FE     -
REP  Repeat Char or Control    \E [ Pn b           1         -      rep
RI   Reverse Index             \E M                -         FE     -   (V)
RIS  Reset to Initial State    \E c                -         Fs     -
RM   Reset Mode *              \E [ Ps l           -         -      -   (W)
SACS Set Add. Char. Sep. *     \E [ Pn SPC /       0         -      -
SAPV Sel. Alt. Present. Var. * \E [ Ps SPC ]       0         -      -   (X)
SCI  Single-Char Introducer    \E Z                -         -      -
SCO  Sel. Char. Orientation *  \E [ Pn ; Pn SPC k  -         -      -
SCS  Set Char. Spacing *       \E [ Pn SPC g       -         -      -
SD   Scroll Down               \E [ Pn T           1         eF     rin
SDS  Start Directed String *   \E [ Pn ]           1         -      -
SEE  Select Editing Extent     \E [ Ps Q           0         -      -   (Y)
SEF  Sheet Eject & Feed *  \E [ Ps ; Ps SPC Y  0,0       -      -
SGR  Select Graphic Rendition  \E [ Ps m           0         FE     sgr (O)
SHS  Select Char. Spacing *    \E [ Ps SPC K       0         -      -
SI   Shift In                  ^O                  -         -      -   (P)
SIMD Sel. Imp. Move Direct. *  \E [ Ps ^           -         -      -
SL   Scroll Left               \E [ Pn SPC @       1         eF     -
SLH  Set Line Home *           \E [ Pn SPC U       -         -      -
SLL  Set Line Limit *          \E [ Pn SPC V       -         -      -
SLS  Set Line Spacing *        \E [ Pn SPC h       -         -      -
SM   Select Mode               \E [ Ps h           none      -      -   (W)
SO   Shift Out                 ^N                  -         -      -   (Q)
SOH  Start Of Heading *        ^A                  -         -      -
SOS  Start of String *         \E X                -         -      -
SPA  Start of Protected Area   \E V                -         -      -   (Z)
SPD  Select Pres. Direction *  \E [ Ps ; Ps SPC S  0,0       -      -
SPH  Set Page Home *           \E [ Ps SPC G       -         -      -
SPI  Spacing Increment         \E [ Pn ; Pn SPC G  none      FE     -
SPL  Set Page Limit *          \E [ Ps SPC j       -         -      -
SPQR Set Pr. Qual. & Rapid. *  \E [ Ps SPC X       0         -      -
SR   Scroll Right              \E [ Pn SPC A       1         eF     -
SRCS Set Reduced Char. Sep. *  \E [ Pn SPC f       0         -      -
SRS  Start Reversed String *   \E [ Ps [           0         -      -
SSA  Start of Selected Area    \E F                -         -      -
SSU  Select Size Unit *        \E [ Pn SPC I       0         -      -
SSW  Set Space Width *         \E [ Pn SPC [       none      -      -
SS2  Single Shift 2 (G2 set)   \E N                -         Intro  -
SS3  Single Shift 3 (G3 set)   \E O                -         Intro  -
ST   String Terminator         \E \                -         Delim  -
STAB Selective Tabulation *    \E [ Pn SPC ^       -         -      -
STS  Set Transmit State        \E S                -         -      -
STX  Start pf Text *           ^B                  -         -      -
SU   Scroll Up                 \E [ Pn S           1         eF     indn
SUB  Substitute *              ^Z                  -         -      -
SVS  Select Line Spacing *     \E [ Pn SPC \       1         -      -
SYN  Synchronous Idle *        ^F                  -         -      -
TAC  Tabul. Aligned Centered * \E [ Pn SPC b       -         -      -
TALE Tabul. Al. Leading Edge * \E [ Pn SPC a       -         -      -
TATE Tabul. Al. Trailing Edge* \E [ Pn SPC `       -         -      -
TBC  Tab Clear                 \E [ Ps g           0         FE     tbc
TCC  Tabul. Centered on Char * \E [ Pn SPC c       -         -      -
TSR  Tabulation Stop Remove  * \E [ Pn SPC d       -         FE     -
TSS  Thin Space Specification  \E [ Pn SC E        none      FE     -
VPA  Vert. Position Absolute   \E [ Pn d           1         FE     vpa
VPB  Line Position Backward *  \E [ Pn k           1         FE     -
VPR  Vert. Position Relative   \E [ Pn e           1         FE     -   (R)
VPR  Vert. Position Relative   \E [ Pn e           1         FE     -   (R)
VT   Vertical Tabulation *     ^K                  -         FE     -
VTS  Vertical Tabulation Set   \E J                -         FE     -

Basic Characteristics

This section documents the capabilities that describe the basic and nature of the terminal, and also those that are relevant to the output of graphic characters.

Name	Description
os	Flag whose presence means that the terminal can overstrike. This means that outputting a graphic character does not erase whatever was present in the same character position before. The terminals that can overstrike include printing terminals, storage tubes (all obsolete nowadays), and many bit-map displays.
eo	Flag whose presence means that outputting a space erases a character position even if the terminal supports overstriking. If this flag is not present and overstriking is supported, output of a space has no effect except to move the cursor. (On terminals that do not support overstriking, you can always assume that outputting a space at a position erases whatever character was previously displayed there.)
gn	Flag whose presence means that this terminal type is a generic type which does not really describe any particular terminal. Generic types are intended for use as the default type assigned when the user connects to the system, with the intention that the user should specify what type he really has. One example of a generic type is the type `network`. Since the generic type cannot say how to do anything interesting with the terminal, termcap-using programs will always find that the terminal is too weak to be supported if the user has failed to specify a real terminal type in place of the generic one. The gnflag directs these programs to use a different error message: “You have not specified your real terminal type”, rather than “Your terminal is not powerful enough to be used”.
hc	Flag whose presence means this is a hardcopy terminal.
rp	String of commands to output a graphic character `c`, repeated `n` times. The first parameter value is the ASCII code for the desired character, and the second parameter is the number of times to repeat the character. Often this command requires padding proportional to the number of times the character is repeated. This effect can be had by using parameter arithmetic with `%`-sequences to compute the amount of padding, then generating the result as a number at the front of the string so that `tputs` will treat it as padding.
hz	Flag whose presence means that the ASCII character cannot be output on this terminal because it is used for display commands. Programs handle this flag by checking all text to be output and replacing each ~with some other character(s). If this is not done, the screen will be thoroughly garbled. The old Hazeltine terminals that required such treatment are probably very rare today, so you might as well not bother to support this flag.
CC	String whose presence means the terminal has a settable command character. The value of the string is the default command character (which is usually ). All the strings of commands in the terminal description should be written to use the default command character. If you are writing an application program that changes the command character, use the CCcapability to figure out how to translate all the display commands to work with the new command character. Most programs have no reason to look at the CCcapability.
xb	Flag whose presence identifies Superbee terminals which are unable to transmit the characters and `Control-C`. Programs which support this flag are supposed to check the input for the code sequences sent by the and keys, and pretend that or `Control-C` (respectively) had been read. But this flag is obsolete, and not worth supporting.

7bit vs. 8bit

These recommendations are optional. IBCS2 allows the leading escape to # be either the 7-bit \E[ or 8-bit \0233 introducer, in accordance with # the ANSI X.364/ISO 6429/ECMA-48 standard.

1.6.1 Describing the Encoding
For example, the cm string for a standard ANSI terminal is written as \E[%i%d;%dH. (\E stands for .) cm by convention always requires two parameters, the vertical and horizontal goal positions, so this string specifies the encoding of two parameters. Here %i increments the two values supplied, and each %d encodes one of the values in decimal. If the cursor position values 20,58 are encoded with this string, the result is \E[21;59H.

First, here are the %-sequences that generate output. Except for %%, each of them encodes one parameter and advances the pointer to the following parameter.

%% Output a single %. This is the only way to represent a literal % in a terminal command with parameters. %% does not use up a parameter.
%d As in printf, output the next parameter in decimal.
%2 Like %02d in printf: output the next parameter in decimal, and always use at least two digits.
%3 Like %03d in printf: output the next parameter in decimal, and always use at least three digits. Note that %4 and so on are _not_ defined.
%. Output the next parameter as a single character whose ASCII code is the parameter value. Like %c in printf.
%+CHAR Add the next parameter to the character CHAR, and output the resulting character. For example, %+ represents 0 as a space, 1 as !, etc.

The following %-sequences specify alteration of the parameters (their values, or their order) rather than encoding a parameter for output. They generate no output; they are used only for their side effects on the parameters. Also, they do not advance the "next parameter" pointer except as explicitly stated. Only %i, %r and %> are defined in standard Unix termcap. The others are GNU extensions.

%i Increment the next two parameters. This is used for terminals that expect cursor positions in origin 1. For example, %i%d,%d would output two parameters with 1 for 0, 2 for 1, etc.
%r Interchange the next two parameters. This is used for terminals whose cursor positioning command expects the horizontal position first.
%s Skip the next parameter. Do not output anything.
%b Back up one parameter. The last parameter used will become once again the next parameter to be output, and the next output command will use it. Using %b more than once, you can back up any number of parameters, and you can refer to each parameter any number of times.
%>C1C2 Conditionally increment the next parameter. Here C1 and C2 are characters which stand for their ASCII codes as numbers. If the next parameter is greater than the ASCII code of C1, the ASCII code of C2 is added to it.
%a OP TYPE POS Perform arithmetic on the next parameter, do not use it up, and do not output anything. Here OP specifies the arithmetic operation, while TYPE and POS together specify the other operand.

Spaces are used above to separate the operands for clarity; the spaces don't appear in the data base, where this sequence is exactly five characters long. The character OP says what kind of arithmetic operation to perform. It can be any of these characters:

= - assign a value to the next parameter, ignoring its old value. The new value comes from the other operand.
+ - add the other operand to the next parameter.
- - subtract the other operand from the next parameter.
* - multiply the next parameter by the other operand.
/ - divide the next parameter by the other operand.

The "other operand" may be another parameter's value or a constant; the character TYPE says which. It can be:

p - Use another parameter. The character POS says which parameter to use. Subtract 64 from its ASCII code to get the position of the desired parameter relative to this one. Thus, the character A as POS means the parameter after the next one; the character ? means the parameter before the next one.
c - Use a constant value. The character POS specifies the value of the constant. The 0200 bit is cleared out, so that 0200 can be used to represent zero.

The following %-sequences are special purpose hacks to compensate for the weird designs of obscure terminals. They modify the next parameter or the next two parameters but do not generate output and do not use up any parameters. %m is a GNU extension; the others are defined in standard Unix termcap.

%n - Exclusive-or the next parameter with 0140, and likewise the parameter after next.
%m - Complement all the bits of the next parameter and the parameter after next.
%B - Encode the next parameter in BCD. It alters the value of the parameter by adding six times the quotient of the parameter by ten. Here is a C statement that shows how the new value is computed: PARM = (PARM / 10) * 16 + PARM % 10;
%D - Transform the next parameter as needed by Delta Data terminals. This involves subtracting twice the remainder of the parameter by 16. PARM -= 2 * (PARM % 16);

Terminal Type Name Conventions

Here is a list of standard suffixes and their conventional meanings:

-w - Short for "wide". This is a mode that gives the terminal more columns than usual. This is normally a user option.
-am - "Automatic margins". This is an alternate description for use when the terminal's margin-wrap switch is on; it contains the am flag. The implication is that normally the switch is off and the usual description for the terminal says that the switch is off.
-nam - "No automatic margins". The opposite of -am, this names an alternative description which lacks the am flag. This implies that the terminal is normally operated with the margin-wrap switch turned on, and the normal description of the terminal says so.
-na - "No arrows". This terminal description initializes the terminal to keep its arrow keys in local mode. This is a user option.
-rv - "Reverse video". This terminal description causes text output for normal video to appear as reverse, and text output for reverse video to come out as normal. Often this description differs from the usual one by interchanging the two strings which turn reverse video on and off.

This is a user option; you can choose either the "reverse video" variant terminal type or the normal terminal type, and termcap will obey.

-s - "Status". Says to enable use of a status line which ordinary output does not touch (*note Status Line::).

Some terminals have a special line that is used only as a status line. For these terminals, there is no need for an -s variant; the status line commands should be defined by default. On other terminals, enabling a status line means removing one screen line from ordinary use and reducing the effective screen height. For these terminals, the user can choose the -s variant type to request use of a status line.

-NLINES - Says to operate with NLINES lines on the screen, for terminals such as the Ambassador which provide this as an option. Normally this is a user option; by choosing the terminal type, you control how many lines termcap will use.
-NPAGESp - Says that the terminal has NPAGES pages worth of screen memory, for terminals where this is a hardware option.
-unk - Says that description is not for direct use, but only for reference in tc capabilities. Such a description is a kind of subroutine, because it describes the common characteristics of several variant descriptions that would use other suffixes in place of -unk.

MS-DOS ANSI.SYS

Here is a description of the color and attribute controls supported in the ANSI.SYS driver under MS-DOS. Most console drivers and ANSI terminal emulators for Intel boxes obey these. They are a proper subset of the ECMA-48 escapes.

Bright black becomes gray, bright brown becomes yellow. These coincide with the prescriptions of the ISO 6429/ECMA-48 standard. * If the 5 attribute is on and you set a background color (40-47) it is supposed to enable bright background.

0        all attributes off
1        foreground bright
4        underscore on
5        blink on/background bright (not reliable with brown)
7        reverse-video
8        set blank (non-display)
10       set primary font
11       set first alternate font (on PCs, display ROM characters 1-31)
12       set second alternate font (on PCs, display IBM high-half chars)
3n       set foreground color       / 0=black, 1=red,     2=green, 3=brown,
4n       set background color       \ 4=blue,  5=magenta, 6=cyan,  7=white

STTY

Stty Control Settings

Name	Description
parenb	`parenb` Generate parity bit in output and expect parity bit in input. May be negated.
parodd	Set odd parity (even if negated). May be negated.
cs5, cs6, cs7, cs8	Set character size to 5, 6, 7, or 8 bits.
hup, hupcl	Send a hangup signal when the last process closes the tty. May be negated.
cstopb	Use two stop bits per character (one if negated). May be negated.
cread	Allow input to be received. May be negated.
clocal	Disable modem control signals. May be negated.
crtscts	Enable RTS/CTS flow control. Disables DTR/DSR flow control. Non-POSIX. May be negated.
cdtrdsr	Enable DTR/DSR flow control. Disables RTS/CTS flow control. Non-POSIX. May be negated. No stty-readable form available. It needs to be supported by device. Usage on device without DTR/DSR support could lead to an error and failure of `stty`.

Stty Input settings

Name	Description
ignbrk	Ignore break characters. May be negated.
brkint	Make breaks cause an interrupt signal. May be negated.
ignpar	Ignore characters with parity errors. May be negated.
parmrk	Mark parity errors (with a 255-0-character sequence). May be negated.
inpck	Enable input parity checking. May be negated.
istrip	Clear high (8th) bit of input characters. May be negated.
inlcr	Translate newline to carriage return. May be negated.
igncr	Ignore carriage return. May be negated.
icrnl	Translate carriage return to newline. May be negated.
iutf8	Assume input characters are UTF-8 encoded. May be negated.
ixon	Enable XON/XOFF flow control (that is, `CTRL-S`/`CTRL-Q`). May be negated.
ixoff, tandem	Enable sending of `stop` character when the system input buffer is almost full, and `start` character when it becomes almost empty again. May be negated.
iuclc	Translate uppercase characters to lowercase. Non-POSIX. May be negated.
ixany	Allow any character to restart output (only the start character if negated). Non-POSIX. May be negated.
imaxbel	Enable beeping and not flushing input buffer if a character arrives when the input buffer is full. Non-POSIX. May be negated.

Stty Output settings

These arguments specify output-related operations.

Name	Description
opost	Postprocess output. May be negated.
olcuc	Translate lowercase characters to uppercase. Non-POSIX. May be negated.
ocrnl	Translate carriage return to newline. Non-POSIX. May be negated.
onlcr	Translate newline to carriage return-newline. Non-POSIX. May be negated.
onocr	Do not print carriage returns in the first column. Non-POSIX. May be negated.
onlret	Newline performs a carriage return. Non-POSIX. May be negated.
ofill	Use fill (padding) characters instead of timing for delays. Non-POSIX. May be negated.
ofdel	Use delete characters for fill instead of null characters. Non-POSIX. May be negated.
nl1, nl0	Newline delay style. Non-POSIX.
cr3, cr2, cr1, cr0	Carriage return delay style. Non-POSIX.
tab3, tab2, tab1, tab0	Horizontal tab delay style. Non-POSIX.
bs1, bs0	Backspace delay style. Non-POSIX.
vt1, vt0	Vertical tab delay style. Non-POSIX.
ff1, ff0	Form feed delay style. Non-POSIX.

Stty Local settings

Name	Description
isig	Enable `interrupt`, `quit`, and `suspend` special characters. May be negated.
icanon	Enable `erase`, `kill`, `werase`, and `rprnt` special characters. May be negated.
iexten	Enable non-POSIX special characters. May be negated.
echo	Echo input characters. May be negated.
echoe, crterase	Echo `erase` characters as backspace-space-backspace. May be negated.
echok	Echo a newline after a `kill` character. May be negated.
echonl	Echo newline even if not echoing other characters. May be negated.
noflsh	Disable flushing after `interrupt` and `quit` special characters. May be negated.
xcase	Enable input and output of uppercase characters by preceding their lowercase equivalents with `\`, when `icanon` is set. Non-POSIX. May be negated.
tostop	Stop background jobs that try to write to the terminal. Non-POSIX. May be negated.
echoprt, prterase	Echo erased characters backward, between `\` and `/`. Non-POSIX. May be negated.
echoctl, ctlecho	Echo control characters in hat notation (`^C`) instead of literally. Non-POSIX. May be negated.
echoke, crtkill	Echo the `kill` special character by erasing each character on the line as indicated by the `echoprt` and `echoe` settings, instead of by the `echoctl` and `echok` settings. Non-POSIX. May be negated.

Stty Combination settings

Name	Description
evenp, parity	Same as `parenb -parodd cs7`. May be negated. If negated, same as `-parenb cs8`.
oddp	Same as `parenb parodd cs7`. May be negated. If negated, same as `-parenb cs8`.
nl	Same as `-icrnl -onlcr`. May be negated. If negated, same as `icrnl -inlcr -igncr onlcr -ocrnl -onlret`.
ek	Reset the `erase` and `kill` special characters to their default values.
sane	Sets all special characters to their default values and: cread -ignbrk brkint -inlcr -igncr icrnl -ixoff -iuclc -ixany imaxbel opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0 isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke
cooked	Same as `brkint ignpar istrip icrnl ixon opost isig icanon`, plus sets the `eof` and `eol` characters to their default values if they are the same as the `min` and `time` characters. May be negated. If negated, same as `raw`.
raw	Same as: -ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -opost -isig -icanon -xcase min 1 time 0 May be negated. If negated, same as `cooked`.
cbreak	Same as `-icanon`. May be negated. If negated, same as `icanon`.
pass8	Same as `-parenb -istrip cs8`. May be negated. If negated, same as `parenb istrip cs7`.
litout	Same as `-parenb -istrip -opost cs8`. May be negated. If negated, same as `parenb istrip opost cs7`.
decctlq	Same as `-ixany`. Non-POSIX. May be negated.
tabs	Same as `tab0`. Non-POSIX. May be negated. If negated, same as `tab3`.
lcase, LCASE	Same as `xcase iuclc olcuc`. Non-POSIX. May be negated.
crt	Same as `echoe echoctl echoke`.
dec	Same as `echoe echoctl echoke -ixany intr ^C erase ^? kill C-u`.

Special characters

The special characters' default values vary from system to system. They are set with the syntax name value, where the names are listed below and the value can be given either literally, in hat notation (^C), or as an integer which may start with 0x to indicate hexadecimal, 0 to indicate octal, or any other digit to indicate decimal.

For GNU stty, giving a value of ^- or undef disables that special character. (This is incompatible with Ultrix stty, which uses a value of u to disable a special character. GNU stty treats a value u like any other, namely to set that special character to .)

Name	Description
`intr`	Send an interrupt signal.
`quit`	Send a quit signal.
`erase`	Erase the last character typed.
`kill`	Erase the current line.
`eof`	Send an end of file (terminate the input).
`eol`	End the line.
`eol2`	Alternate character to end the line. Non-POSIX.
`swtch`	Switch to a different shell layer. Non-POSIX.
`start`	Restart the output after stopping it.
`stop`	Stop the output.
`susp`	Send a terminal stop signal.
`dsusp`	Send a terminal stop signal after flushing the input. Non-POSIX.
`rprnt`	Redraw the current line. Non-POSIX.
`werase`	Erase the last word typed. Non-POSIX.
`lnext`	Enter the next character typed literally, even if it is a special character. Non-POSIX.

Special settings

min N - Set the minimum number of characters that will satisfy a read until the time value has expired, when -icanon is set.
time N - Set the number of tenths of a second before reads time out if the minimum number of characters have not been read, when -icanon is set.
ispeed N - Set the input speed to N.
ospeed N - Set the output speed to N.
rows N - Tell the tty kernel driver that the terminal has N rows. Non-POSIX.
cols N, columns N - Tell the kernel that the terminal has N columns. Non-POSIX.
size - Print the number of rows and columns that the kernel thinks the terminal has. (Systems that don't support rows and columns in the kernel typically use the environment variables LINES and COLUMNS instead; however, GNU stty does not know anything about them.) Non-POSIX.
line N - Use line discipline N. Non-POSIX.
speed - Print the terminal speed.
N - Set the input and output speeds to N. N can be one of: 0 50 75 110 134 134.5 150 200 300 600 1200 1800 2400 4800 9600 19200 38400 exta extb. exta is the same as 19200; extb is the same as 38400. 0 hangs up the line if -clocal is set.

Color Terminals

color256 ()
{
    local L=$(sed 's/[0-9]//g; s/./#/g' <<<`seq -s+0 $(($COLUMNS - 10))`);
    for i in `seq 0 $(tput colors)`;
    do printf " %.3d `tput setab $i`${L}${R} %.3d `tput setab 0;tput setaf 7`\n" $i $i | tr '#' ' ';
    done
}

for F in `find ~/.terminfo/ -type f | sed 's/^.*\///' | xargs -P0 -I'TT' sh -c 'echo "$(tput -T TT colors): TT"'|sed '/^-/d'| grep ^16:|cut -d: -f2`; do
echo -e "\n\n";
pm $F; export TERM=$F; tput -T $F initc; color256; done

find ~/.terminfo/ -type f | sed 's/^.*\///' | xargs -I'TT' sh -c 'echo "$(tput -T TT colors): TT"'|sed '/^-/d'

Terminals with 16 Colors

aixterm-16color
amiga-vnc
hp2397
hp2397a
hp+color
ibm+16color
konsole-16color
nsterm-16color
nsterm-7-c
nsterm-7-c-s
nsterm-acs-c
nsterm-acs-c-s
nsterm-c
nsterm+c
nsterm-c-7
nsterm-c-acs
nsterm-c-s
nsterm-c-s-7
nsterm-c-s-acs
rxvt-16color
screen-16color
screen-16color-bce
screen-16color-bce-s
screen-16color-s
xterm-16color

Terminals with 52 Colors

d430c-dg-ccc
d430c-unix-25-ccc
d430c-unix-ccc
d430c-unix-s-ccc
d430c-unix-sr-ccc
d430c-unix-w-ccc
d430-dg-ccc
d430-unix-25-ccc
d430-unix-ccc
d430-unix-s-ccc
d430-unix-sr-ccc
d430-unix-w-ccc
dg+ccc
dgunix+ccc

Terminals with 64 Colors

hpterm-color
wy370
wy370-101k
wy370-105k
wy370-EPC
wy370-nk
wy370-rv
wy370-vb
wy370-w
wy370-wvb
wyse370
Eterm-88color
rxvt-88color
xterm-88color
xterm+88color

Terminals with 256 colors

Eterm-256color
gnome-256color
konsole-256color
putty-256color
rxvt-256color
screen-256color
screen-256color-bce
screen-256color-bce-s
screen-256color-s
xterm-256color
xterm+256color

View All tput capabilities

http://blogs.msdn.com/adioltean/articles/271063.aspx

function tputs(){ infocmp -1|sed 's/^[   ]*//; /=/!d; s/=.*//g; /[#\|]/d'; }

Some common terminfo parameter sequences, their termcap equivalents, and some terminal types which commonly have such sequences, are:

terminfo                    termcap   Representative Terminals
---------------------------------------------------------------
%p1%c                       %.        adm
%p1%d                       %d        hp, ANSI standard, vt100
%p1%+%c                 %+x       concept
%i                          %iq       ANSI standard, vt100
%p1%?%>%t%p1%+%;    %>xy      concept
%p2 is printed before %p1   %r        hp

Installing your Own Terminfo

MostLike, or if that is offline I uploaded a copy: mostlike - manpages with color looking like most

rm -rvf ~/.terminfo
mkdir -pv ~/.terminfo
curl -o terminfo.master http://nion.modprobe.de/mostlike.txt; curl -Ss ftp://invisible-island.net/ncurses/terminfo.src.gz
export TERMINFO=~/.terminfo;
sudo tic -e screen-256color,screen-256color-s,screen-256color-bce,screen-256color-bce-s,xterm+256color,xterm,xterm-pcolor terminfo.src

256 Colors in Terminal

256 Colors (Background)

W=`tput setaf 7` RC="\E[0;0;0m" L=$(sed 's/[0-9]//g; s/./ /g' <<<`seq -s+0 $(($COLUMNS/2))`);
 
for i in `seq 0 256`;
do
  printf "${W}\n%.3d `tput setab $i`${L}${RC}" $i;
done

256 Colors (Foreground)

W=`tput setaf 7` WB=`tput setab 0` RC="\E[0;0;0m" L=$(sed 's/[0-9]//g; s/./#/g' <<<`seq -s+0 $(($COLUMNS/2))`);
for i in `seq 0 256`;
do
  printf "${WB}\n%.3d `tput setaf $i`${L}${RC}" $i;
done

Terminal Capabilities - Infocmp

infocmp -1Lq|grep -v "$TERM\|#"|tr -d '  '

DEC/ANSI special sequences

List of the DEC/ANSI special sequences recognized:

----------------------------------------- ----------------------------------------- ----------------------------------------- ----------------------------------------- ----------------------------------------- -----------------------------------------

Action	Meaning
`%%`	The percent sign
`%t`	The time the request was received in Universal Coordinated Time since the epoch (Jan. 1, 1970) measured in microseconds. The value is preceded by `t=`.
`%D`	The time from when the request was received to the time the headers are sent on the wire. This is a measure of the duration of the request. The value is preceded by `D=`. The value is measured in microseconds.
`%{VARNAME}e`	The contents of the environment variable `VARNAME`.
`%{VARNAME}s`	The contents of the SSL environment variable `VARNAME`, if `mod_ssl` is enabled.
`RIS`	full reset
`SC`	save cursor
`RC`	restore cursor
`LL`	home-down
`RSR`	reset scroll region
`DECSTR`	soft reset (VT320)
`S7C1T`	7-bit controls (VT220)
`ISO DEC G0`	enable DEC graphics for G0
`ISO UK G0`	enable UK chars for G0
`ISO US G0`	enable US chars for G0
`ISO DEC G1`	enable DEC graphics for G1
`ISO UK G1`	enable UK chars for G1
`ISO US G1`	enable US chars for G1
`DECPAM`	application keypad mode
`DECPNM`	normal keypad mode
`DECANSI`	enter ANSI mode
`ECMA[+-]AM`	keyboard action mode
`ECMA[+-]IRM`	insert replace mode
`ECMA[+-]SRM`	send receive mode
`ECMA[+-]LNM`	linefeed mode
`DEC[+-]CKM`	application cursor keys
`DEC[+-]ANM`	set VT52 mode
`DEC[+-]COLM`	132-column mode
`DEC[+-]SCLM`	smooth scroll
`DEC[+-]SCNM`	reverse video mode
`DEC[+-]OM`	origin mode
`DEC[+-]AWM`	wraparound mode
`DEC[+-]ARM`	auto-repeat mode

Name	Description
acs_chars	``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{\|\|}}~~,
back_tab	`\E[Z,`
bell	`^G,`
carriage_return	`^M,`
change_scroll_region	`\E[%i%p1%d;%p2%dr,`
clear_all_tabs	`\E[3g,`
clear_screen	`\E[H\E[2J,`
clr_bol	`\E[1K,`
clr_eol	`\E[K,`
clr_eos	`\E[J,`
column_address	`\E[%i%p1%dG,`
cursor_address	`\E[%i%p1%d;%p2%dH,`
cursor_down	`^J,`
cursor_home	`\E[H,`
cursor_invisible	`\E[?25l,`
cursor_left	`^H,`
cursor_normal	`\E[?12l\E[?25h,`
cursor_right	`\E[C,`
cursor_up	`\E[A,`
cursor_visible	`\E[?12;25h,`
delete_character	`\E[P,`
delete_line	`\E[M,`
enter_alt_charset_mode	`\E(0,`
enter_am_mode	`\E[?7h,`
enter_blink_mode	`\E[5m,`
enter_bold_mode	`\E[1m,`
enter_ca_mode	`\E[?1049h,`
enter_insert_mode	`\E[4h,`
enter_reverse_mode	`\E[7m,`
enter_secure_mode	`\E[8m,`
enter_standout_mode	`\E[7m,`
enter_underline_mode	`\E[4m,`
erase_chars	`\E[%p1%dX,`
exit_alt_charset_mode	`\E(B,`
exit_am_mode	`\E[?7l,`
exit_attribute_mode	`\E(B\E[m,`
exit_ca_mode	`\E[?1049l,`
exit_insert_mode	`\E[4l,`
exit_standout_mode	`\E[27m,`
exit_underline_mode	`\E[24m,`
flash_screen	`\E[?5h$<100/>\E[?5l,`
init_2string	`\E[\041p\E[?3;4l\E[4l\E>,`
initialize_color	`\E]4;%p1%d;rgb\072%p2%{255}%%{1000}%/%2.2X/%p3%{255}%%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\,`
insert_line	`\E[L,`
key_b2	`\EOE,`
key_backspace	`\177,`
key_btab	`\E[Z,`
key_dc	`\E[3~,`
key_down	`\EOB,`
key_end	`\EOF,`
key_enter	`\EOM,`
key_f1	`\EOP,`
key_f10	`\E[21~,`
key_f11	`\E[23~,`
key_f12	`\E[24~,`
key_f13	`\EO2P,`
key_f14	`\EO2Q,`
key_f15	`\EO2R,`
key_f16	`\EO2S,`
key_f17	`\E[15;2~,`
key_f18	`\E[17;2~,`
key_f19	`\E[18;2~,`
key_f2	`\EOQ,`
key_f20	`\E[19;2~,`
key_f21	`\E[20;2~,`
key_f22	`\E[21;2~,`
key_f23	`\E[23;2~,`
key_f24	`\E[24;2~,`
key_f25	`\EO5P,`
key_f26	`\EO5Q,`
key_f27	`\EO5R,`
key_f28	`\EO5S,`
key_f29	`\E[15;5~,`
key_f3	`\EOR,`
key_f30	`\E[17;5~,`
key_f31	`\E[18;5~,`
key_f32	`\E[19;5~,`
key_f33	`\E[20;5~,`
key_f34	`\E[21;5~,`
key_f35	`\E[23;5~,`
key_f36	`\E[24;5~,`
key_f37	`\EO6P,`
key_f38	`\EO6Q,`
key_f39	`\EO6R,`
key_f4	`\EOS,`
key_f40	`\EO6S,`
key_f41	`\E[15;6~,`
key_f42	`\E[17;6~,`
key_f43	`\E[18;6~,`
key_f44	`\E[19;6~,`
key_f45	`\E[20;6~,`
key_f46	`\E[21;6~,`
key_f47	`\E[23;6~,`
key_f48	`\E[24;6~,`
key_f49	`\EO3P,`
key_f5	`\E[15~,`
key_f50	`\EO3Q,`
key_f51	`\EO3R,`
key_f52	`\EO3S,`
key_f53	`\E[15;3~,`
key_f54	`\E[17;3~,`
key_f55	`\E[18;3~,`
key_f56	`\E[19;3~,`
key_f57	`\E[20;3~,`
key_f58	`\E[21;3~,`
key_f59	`\E[23;3~,`
key_f6	`\E[17~,`
key_f60	`\E[24;3~,`
key_f61	`\EO4P,`
key_f62	`\EO4Q,`
key_f63	`\EO4R,`
key_f7	`\E[18~,`
key_f8	`\E[19~,`
key_f9	`\E[20~,`
key_home	`\EOH,`
key_ic	`\E[2~,`
key_left	`\EOD,`
key_mouse	`\E[M,`
key_npage	`\E[6~,`
key_ppage	`\E[5~,`
key_right	`\EOC,`
key_sdc	`\E[3;2~,`
key_send	`\E[1;2F,`
key_sf	`\E[1;2B,`
key_shome	`\E[1;2H,`
key_sic	`\E[2;2~,`
key_sleft	`\E[1;2D,`
key_snext	`\E[6;2~,`
key_sprevious	`\E[5;2~,`
key_sr	`\E[1;2A,`
key_sright	`\E[1;2C,`
key_up	`\EOA,`
keypad_local	`\E[?1l\E>,`
\E[?1h\E	`,`
memory_lock	`\El,`
memory_unlock	`\Em,`
orig_pair	`\E[39;49m,`
parm_dch	`\E[%p1%dP,`
parm_delete_line	`\E[%p1%dM,`
parm_down_cursor	`\E[%p1%dB,`
parm_ich	`\E[%p1%d@,`
parm_index	`\E[%p1%dS,`
parm_insert_line	`\E[%p1%dL,`
parm_left_cursor	`\E[%p1%dD,`
parm_right_cursor	`\E[%p1%dC,`
parm_rindex	`\E[%p1%dT,`
parm_up_cursor	`\E[%p1%dA,`
print_screen	`\E[i,`
prtr_off	`\E[4i,`
prtr_on	`\E[5i,`
reset_1string	`\Ec,`
reset_2string	`\E[\041p\E[?3;4l\E[4l\E>,`
restore_cursor	`\E8,`
row_address	`\E[%i%p1%dd,`
save_cursor	`\E7,`
scroll_forward	`^J,`
scroll_reverse	`\EM,`
set_a_background	`\E[%?%p1%{8}%`

Term Sorted

./MKtermsort.sh |sed 's/.*\/\*  \(.*\)  \*\//\1/g'|sort -u|grep ^[a-zA-Z]|sortwc|grep -v ^static|sort -d|tr " " ','|sed 's/\([^,]\{1,\}\),/\1<\/var>,/g'

AB,ac,acs_btee,acsc,acs_chars,acs_hline,acs_llcorner,acs_lrcorner,acs_ltee,acs_plus,acs_rtee,acs_ttee,acs_ulcorner,acs_urcorner,acs_vline,ae,AF,al,AL,alt_scancode_esc,am,apstr,arrow_key_map,as,auto_left_margin,auto_right_margin,back_color_erase,backspace_delay,backspace_if_not_bs,backspaces_with_bs,back_tab,batt1,batt2,bc,bce,bel,bell,bicr,binel,birep,bit_image_carriage_return,bit_image_entwining,bit_image_newline,bit_image_repeat,bit_image_type,bitwin,bitype,bl,blink,bold,box1,box2,box_chars_1,bs,bt,BT,btml,btns,buffer_capacity,bufsz,buttons,bw,bx,can_change,carriage_return,carriage_return_delay,cb,cbt,cc,CC,ccc,cd,ce,ceol_standout_glitch,ch,change_char_pitch,change_line_pitch,change_res_horz,change_res_vert,change_scroll_region,char_padding,char_set_names,chr,chts,ci,civis,cl,clear,clear_all_tabs,clear_margins,clear_screen,clr_bol,clr_eol,clr_eos,cm,CM,cmdch,cnorm,co,Co,code_set_init,col_addr_glitch,colb0,colb1,colb2,colb3,colb4,colb5,colb6,colb7,colf0,colf1,colf2,colf3,colf4,colf5,colf6,colf7,color_names,colornm,colors,cols,column_address,columns,command_character,cpi,cpi_changes_res,cpix,cps,cr,cr_cancels_micro_mode,create_window,crt_no_scrolling,crxm,cs,csin,csnm,csr,ct,cub,cub1,cud,cud1,cuf,cuf1,cup,cursor_address,cursor_down,cursor_home,cursor_invisible,cursor_left,cursor_mem_address,cursor_normal,cursor_right,cursor_to_ll,cursor_up,cursor_visible,cuu,cuu1,cv,cvr,cvvis,CW,cwin,da,daisy,db,dB,dc,dC,DC,dch,dch1,dclk,defbi,defc,define_bit_image_region,define_char,delete_character,delete_line,dest_tabs_magic_smso,device_type,devt,DI,dial,dial_phone,dim,dispc,display_clock,display_pc_char,dis_status_line,DK,dl,DL,dl1,dm,dN,do,DO,docr,dot_horz_spacing,dot_vert_spacing,down_half_line,ds,dsl,dT,dv,eA,eat_newline_glitch,ec,ech,ed,ehhlm,ei,el,el1,elhlm,elohlm,ena_acs,enacs,endbi,end_bit_image_region,enter_alt_charset_mode,enter_am_mode,enter_blink_mode,enter_bold_mode,enter_ca_mode,enter_delete_mode,enter_dim_mode,enter_doublewide_mode,enter_draft_quality,enter_horizontal_hl_mode,enter_insert_mode,enter_italics_mode,enter_left_hl_mode,enter_leftward_mode,enter_low_hl_mode,enter_micro_mode,enter_near_letter_quality,enter_normal_quality,enter_pc_charset_mode,enter_protected_mode,enter_reverse_mode,enter_right_hl_mode,enter_scancode_mode,enter_secure_mode,enter_shadow_mode,enter_standout_mode,enter_subscript_mode,enter_superscript_mode,enter_top_hl_mode,enter_underline_mode,enter_upward_mode,enter_vertical_hl_mode,enter_xon_mode,eo,erase_chars,erase_overstrike,erhlm,es,eslok,ethlm,evhlm,exit_alt_charset_mode,exit_am_mode,exit_attribute_mode,exit_ca_mode,exit_delete_mode,exit_doublewide_mode,exit_insert_mode,exit_italics_mode,exit_leftward_mode,exit_micro_mode,exit_pc_charset_mode,exit_scancode_mode,exit_shadow_mode,exit_standout_mode,exit_subscript_mode,exit_superscript_mode,exit_underline_mode,exit_upward_mode,exit_xon_mode,F1,F2,F3,F4,F5,F6,F7,F8,F9,Fa,FA,Fb,FB,Fc,FC,Fd,FD,Fe,FE,ff,Ff,FF,Fg,FG,fh,Fh,FH,Fi,FI,fixed_pause,Fj,FJ,Fk,FK,Fl,FL,flash,flash_hook,flash_screen,fln,Fm,FM,Fn,FN,Fo,FO,font0,font1,font2,font3,font4,font5,font6,font7,form_feed,Fp,FP,Fq,FQ,Fr,FR,from_status_line,fs,FS,fsl,FT,FU,FV,FW,FX,FY,FZ,G1,G2,G3,G4,GC,GD,generic_type,getm,get_mouse,GH,GL,Gm,gn,gnu_has_meta_key,goto_window,GR,GU,GV,hangup,hard_copy,hard_cursor,has_hardware_tabs,has_meta_key,has_print_wheel,has_status_line,hc,HC,hd,hl,hls,ho,home,hook,horizontal_tab_delay,hpa,hs,ht,hts,hu,HU,hue_lightness_saturation,hup,hz,i1,i2,i3,ic,Ic,IC,ich,ich1,if,il,il1,im,in,ind,indn,init_1string,init_2string,init_3string,initc,init_file,initialize_color,initial ize_pair,initp,init_prog,init_tabs,insert_character,insert_line,insert_null_glitch,insert_padding,invis,ip,iP,Ip,iprog,is,is1,is2,is3,it,k;,k0,k1,K1,k2,K2,k3,K3,k4,K4,k5,K5,k6,k7,k8,k9,ka,kA,ka1,ka3,kact,kb,kB,kb2,kbeg,kBEG,kbs,kbtab,kC,kc1,kc3,kcan,kCAN,kcbt,kclo,kclr,kcmd,kCMD,kcpn,kcpy,kCPY,kcrt,kCRT,kctab,kcub1,kcud1,kcuf1,kcuu1,kd,kD,kDC,kdch1,kDL,kdl1,kdo,ke,kE,ked,kel,kend,kEND,kent,kEOL,kext,kEXT,key_a1,key_a3,key_b2,key_backspace,key_beg,key_btab,key_c1,key_c3,key_cancel,key_catab,key_clear,key_close,key_command,key_copy,key_create,key_ctab,key_dc,key_dl,key_down,key_eic,key_end,key_enter,key_eol,key_eos,key_exit,key_f0,key_f1,key_f10,key_f11,key_f12,key_f13,key_f14,key_f15,key_f16,key_f17,key_f18,key_f19,key_f2,key_f20,key_f21,key_f22,key_f23,key_f24,key_f25,key_f26,key_f27,key_f28,key_f29,key_f3,key_f30,key_f31,key_f32,key_f33,key_f34,key_f35,key_f36,key_f37,key_f38,key_f39,key_f4,key_f40,key_f41,key_f42,key_f43,key_f44,key_f45,key_f46,key_f47,key_f48,key_f49,key_f5,key_f50,key_f51,key_f52,key_f53,key_f54,key_f55,key_f56,key_f57,key_f58,key_f59,key_f6,key_f60,key_f61,key_f62,key_f63,key_f7,key_f8,key_f9,key_find,key_help,key_home,key_ic,key_il,key_left,key_ll,key_mark,key_message,key_mouse,key_move,key_next,key_npage,key_open,key_options,keypad_local,keypad_xmit,key_ppage,key_previous,key_print,key_redo,key_reference,key_refresh,key_replace,key_restart,key_resume,key_right,key_save,key_sbeg,key_scancel,key_scommand,key_scopy,key_screate,key_sdc,key_sdl,key_select,key_send,key_seol,key_sexit,key_sf,key_sfind,key_shelp,key_shome,key_sic,key_sleft,key_smessage,key_smove,key_snext,key_soptions,key_sprevious,key_sprint,key_sr,key_sredo,key_sreplace,key_sright,key_srsume,key_ssave,key_ssuspend,key_stab,key_sundo,key_suspend,key_undo,key_up,kF,kf0,kf1,kf10,kf11,kf12,kf13,kf14,kf15,kf16,kf17,kf18,kf19,kf2,kf20,kf21,kf22,kf23,kf24,kf25,kf26,kf27,kf28,kf29,kf3,kf30,kf31,kf32,kf33,kf34,kf35,kf36,kf37,kf38,kf39,kf4,kf40,kf41,kf42,kf43,kf44,kf45,kf46,kf47,kf48,kf49,kf5,kf50,kf51,kf52,kf53,kf54,kf55,kf56,kf57,kf58,kf59,kf6,kf60,kf61,kf62,kf63,kf7,kf8,kf9,kfnd,kFND,kh,kH,khlp,kHLP,kHOM,khome,khts,kI,kIC,kich1,kil1,kind,kl,kL,kLFT,kll,km,kM,Km,kmous,kmov,kMOV,kmpf1,kmpf2,kmpf3,kmpf4,kmpf5,kmpf6,kmpf7,kmpf8,kmpf9,kmpt1,kmpt2,kmpt3,kmpt4,kmpt5,kmpt6,kmpt7,kmpt8,kmpt9,kmrk,kmsg,kMSG,kn,kN,knl,knp,knpn,knxt,kNXT,ko,kopn,kopt,kOPT,kP,kpp,kppn,kprt,kPRT,kprv,kPRV,kquit,kr,kR,krdo,kRDO,kref,kres,kRES,krfr,kri,kRIT,krmir,krpl,kRPL,krst,ks,kS,ksav,kSAV,kscl,kscr,ksel,ksf1,ksf10,ksf2,ksf3,ksf4,ksf5,ksf6,ksf7,ksf8,ksf9,kslt,kspd,kSPD,kt,kT,ktab,ktbc,ku,kund,kUND,l0,l1,l2,l3,l4,l5,l6,l7,l8,l9,la,label_format,label_height,label_off,label_on,label_width,lab_f0,lab_f1,lab_f10,lab_f2,lab_f3,lab_f4,lab_f5,lab_f6,lab_f7,lab_f8,lab_f9,le,LE,Lf,LF,lf0,lf1,lf10,lf2,lf3,lf4,lf5,lf6,lf7,lf8,lf9,lh,li,linefeed_if_not_lf,linefeed_is_newline,lines,lines_of_memory,ll,lm,LO,lpi,lpi_changes_res,lpix,lvert,lw,ma,maddr,magic_cookie_glitch,magic_cookie_glitch_ul,max_attributes,max_colors,maximum_windows,max_micro_address,max_micro_jump,max_pairs,mb,MC,mc0,mc4,mc5,mc5i,mc5p,mcs,mcub,mcub1,mcud,mcud1,mcuf,mcuf1,mcuu,mcuu1,md,me,meml,memory_above,memory_below,memory_lock,memory_unlock,memu,meta_off,meta_on,mgc,mh,mhpa,mi,Mi,micro_col_size,micro_column_address,micro_down,micro_left,micro_line_size,micro_right,micro_row_address,micro_up,minfo,mir,mjump,mk,ml,ML,mls,mm,mo,mouse_info,move_insert_mode,move_standout_mode,mp,mr,MR,mrcup,ms,msgr,MT,mu,mvpa,MW,nc,NC,ncv,nd,ND,ndscr,needs_xon_xoff,nel,newline,new_line_delay,nl,Nl,NL,nlab,no_color_video,no_correctly_working_cr,no_esc_ctlc,non_dest_scroll_region,non_rev_rmcup,no_pad_char,NP,npc,npins,NR,nrrmc,ns,number_of_function_keys,number_of_pins,num_labels,nw,nx,nxon,oc,op,orc,order_of_pins,orhi,orig_colors,orig_pair,orl,orvi,os,OTbc,OTbs,OTdB,OTdC,OTdN,OTdT,OTG1,OTG2,OTG3,OTG4,OTGC,OTGD,OTGH,OTGL,OTGR,OTGU,OTGV,other_non_function_keys,OTi2,OTkn,OTko,OTma,OTMT,OTnc,OTnl,OTNL,OTns,OTpt,OTrs,OTug,OTxr,output_res_char,output_res_horz_inch,output_res_line,output_res_vert_inch,over_strike,pa,PA,pad,pad_char,padding_baud_rate,pairs,parm_dch,parm_delete_line,parm_down_cursor,parm_down_micro,parm_ich,parm_index,parm_insert_line,parm_left_cursor,parm_left_micro,parm_right_cursor,parm_right_micro,parm_rindex,parm_up_cursor,parm_up_micro,pause,pb,pc,pc_term_options,pctrm,pf,pfkey,pfloc,pfx,pfxl,pk,pkey_key,pkey_local,pkey_plab,pkey_xmit,pl,plab_norm,pln,pn,po,pO,porder,print_rate,print_screen,prot,prtr_non,prtr_off,prtr_on,prtr_silent,ps,pt,PU,pulse,px,QD,qdial,quick_dial,r1,r2,r3,RA,rbim,rc,RC,rcsd,remove_clock,rep,repeat_char,req_for_input,req_mouse_pos,reqmp,reset_1string,reset_2string,reset_3string,reset_file,restore_cursor,return_does_clr_eol,rev,rf,RF,rfi,ri,RI,rin,ritm,rlm,rmacs,rmam,rmclk,rmcup,rmdc,rmicm,rmir,rmkx,rmln,rmm,rmp,rmpch,rmsc,rmso,rmul,rmxon,row_address,row_addr_glitch,rp,rP,RQ,rs,rs1,rs2,rs3,rshm,rsubm,rsupm,rum,rvert,rwidm,RX,s0,s0ds,s1,S1,s1ds,s2,S2,s2ds,s3,S3,s3ds,S4,S5,S6,S7,S8,sa,sA,SA,sam,save_cursor,Sb,sbim,sc,SC,scancode_escape,scesa,scesc,sclk,scp,scroll_forward,scroll_reverse,scs,scsd,sdrfq,se,select_char_set,semi_auto_right_margin,set0_des_seq,set1_des_seq,set2_des_seq,set3_des_seq,set_a_attributes,setab,set_a_background,setaf,set_a_foreground,set_attributes,setb,set_background,set_bottom_margin,set_bottom_margin_parm,set_clock,setcolor,set_color_band,set_color_pair,setf,set_foreground,set_left_margin,set_left_margin_parm,set_lr_margin,set_page_length,set_pglen_inch,set_right_margin,set_right_margin_parm,set_tab,set_tb_margin,set_top_margin,set_top_margin_parm,set_window,sf,Sf,SF,sg,sgr,sgr0,sgr1,sitm,sL,slength,slines,slm,smacs,smam,smcup,smdc,smgb,smgbp,smgl,smglp,smglr,smgr,smgrp,smgt,smgtb,smgtp,smicm,smir,smkx,smln,smm,smpch,smsc,smso,smul,smxon,snlq,snrmq,so,sp,spinh,spinv,sr,SR,sshm,ssubm,ssupm,st,start_bit_image,start_char_set_def,status_line_esc_ok,stop_bit_image,stop_char_set_def,subcs,subscript_characters,sum,supcs,superscript_characters,swidm,SX,ta,tab,tbc,te,termcap_init2,termcap_reset,these_cause_cr,ti,tilde_glitch,TO,tone,topl,to_status_line,transparent_underline,ts,tsl,u0,u1,u2,u3,u4,u5,u6,u7,u8,u9,uc,ue,ug,ul,underline_char,up,UP,up_half_line,us,user0,user1,user2,user3,user4,user5,user6,user7,user8,user9,ut,vb,ve,vi,virtual_terminal,vpa,vs,vt,WA,wait,wait_tone,WG,wi,widcs,wide_char_size,width_status_line,wind,wingo,wnum,ws,wsl,xb,xenl,XF,Xh,xhp,xhpa,xl,Xl,xmc,xn,XN,xo,Xo,xoffc,xoff_character,xon,xonc,xon_character,xon_xoff,xr,Xr,xs,xsb,xt,Xt,Xv,xvpa,Xy,Ya,YA,Yb,YB,Yc,YC,Yd,YD,Ye,YE,Yf,YF,Yg,YG,Yh,Yi,Yj,Yk,Yl,Ym,Yn,Yo,Yp,Yv,Yw,Yx,Yy,Yz,YZ,Za,ZA,Zb,ZB,Zc,ZC,Zd,ZD,Ze,ZE,zerom,zero_motion,Zf,ZF,Zg,ZG,Zh,ZH,Zi,ZI,Zj,ZJ,Zk,ZK,Zl,ZL,Zm,ZM,Zn,ZN,Zo,ZO,Zp,ZP,Zq,ZQ,Zr,ZR,Zs,ZS,Zt,ZT,Zu,ZU,Zv,ZV,Zw,ZW,Zx,ZX,Zy,ZY,Zz,ZZ

TERMINAL TYPE DESCRIPTIONS SOURCE FILE

ANSI, UNIX CONSOLE, AND SPECIAL TYPES
Specials
ANSI.SYS/ISO 6429/ECMA-48 Capabilities
ANSI/ECMA-48 terminals and terminal emulators
DOS ANSI.SYS variants
ANSI console types
BeOS
Linux consoles
Mach
OSF Unix
QNX
NetBSD consoles
FreeBSD console entries
386BSD and BSD/OS Consoles
DEC VT52
DEC VT100 and compatibles
VT100 emulations
X terminal emulators
MGR
UNIX VIRTUAL TERMINALS, VIRTUAL CONSOLES, AND TELNET CLIENTS
Pilot Pro Palm-Top
COMMERCIAL WORKSTATION CONSOLES
Alpha consoles
Sun consoles
Iris consoles
NeWS consoles
NeXT consoles
Sony NEWS workstations
Common Desktop Environment
Non-Unix Consoles
COMMON TERMINAL TYPES
Altos
Hewlett-Packard (hp)
Honeywell-Bull
Lear-Siegler (adm)
Prime
Qume (qvt)
Televideo (tvi)
Visual (vi)
Wyse (wy)
Kermit terminal emulations
NON-ANSI TERMINAL EMULATIONS
Avatar
RBcomm
LCD DISPLAYS
Matrix Orbital
OLDER TERMINAL TYPES
AT&T (att, tty)
Ampex (Dialogue)
Ann Arbor (aa)
Applied Digital Data Systems (adds)
C. Itoh Electronics
Control Data (cdc)
Getronics
Human Designed Systems (Concept)
Contel Business Systems.
Data General (dg)
Datamedia (dm)
Falco

Just something sorta cool for you to check out

{
 local t=`tputm 'clear' 'setaf 75'` l a b f=/tmp/ps IFS=' ';
exec 6<>$f;ps L|tr -s ' ' &>$f;
while read -u6 l;do a=${l/% */} b=${l/* /};
figlet -rtw $((${COLUMNS} /2 )) -f big "$l";
tput sgr0;
command ps wwo pid:6,user:8,vsize:8,comm:20,$a:50 k -$a -A;cont;
done;
};

Terminal Escape Code Zen originally appeared on AskApache.com

Advanced Linux Debugging using a Bootloader (GRUB)

AskApache — Thu, 07 Apr 2011 16:35:41 +0000

AskApache.com

This is part of the GRUB article on the Arch Linux wiki that I contributed. As of now, just a copy.

The grub menu.lst provides for a convenient way to add a number of entries with extended kernel parameters to configure all sorts of advanced settings to enable you to quickly and conveniently boot into your existing system with varying levels of debugging output. It's very easy and useful to create several levels of debugging just by adding additional entries to your grub configuration. And if you ever have issues or problems down the road due to a power-failure or hardware failure, it can save you hours of trouble, and of course nothing can beat debugging output when it comes to learning about your system.

Useful Menu.lst Entries

If you are interested in debugging, then you deserve some grub entries for powerusers, here are a few that I like (just add to your menu.lst).

title Shutdown the Computer
halt
 
title Reboot the Computer
reboot
 
title Command Line
commandline
 
title Install GRUB to hd0 MBR
root (hd0,0)
setup (hd0)
 
title Matrix
color green/black light-green/green
 
title Scan for /boot/grub/menu.lst
find --set-root --ignore-floppies /boot/grub/menu.lst
configfile /boot/grub/menu.lst
 
title Scan for /boot/menu.lst
find --set-root --ignore-floppies /menu.lst
configfile /boot/menu.lst
 
# http://www.vortex.prodigynet.co.uk/x86test/
title    Run x86test (CPU Info)
kernel /boot/x86test_zImage.bin
#wget http://www.vortex.prodigynet.co.uk/x86test/x86test_zImage.bin
 
# http://www.memtest.org/
title    Run memtest86+ (Memory Testing)
kernel /boot/memtest86+-1.70.bin

Light Debug

A quick way to see more verbose messages on your console is to bootup your normal grub entry after appending verbose to the kernel line. This simple word added to your kernel line turns on more logging thanks to the /etc/rc.sysinit file, which at the top of the file runs:

if /bin/grep -q " verbose" /proc/cmdline; then /bin/dmesg -n 8; fi

Very simple way to get a bit more messages and debug output in your logs.

title  Arch Linux DEBUG Light
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait verbose
initrd /kernel26.img

Medium Debug

This example menu.lst entry turns on real logging that is set by the kernel and not in an init script. Adding the debug kernel parameter to your kernel line is recognized by a lot of linux internals and enables quite a bit of debugging compared to the default.

title Arch Linux DEBUG Medium
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootdelay=5 panic=10 debug
initrd /kernel26.img

Heavy Debug

An even more impressive kernel parameter is the ignore_loglevel, which causes the system to ignore any loglevel and keeps the internal loglevel at the maximum debugging level, basically rendering dmesg unable to lower the debug level.

title Arch Linux DEBUG Heavy
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootdelay=5 panic=10 debug ignore_loglevel
initrd /kernel26.img

Extreme Debug

If the "Heavy Debug" seemed like a lot of output, thats about 1/2 of the logging that occurs with this example. This does a couple things, it uses the earlyprintk parameter to setup your kernel for "early" "printing" of messages to your "vga" screen. The ,keep just lets it stay on the screen longer. This will let you see logs that normally are hidden due to the boot-up process. This also changes the log buffer length to 10MB, and also instructs that any fatal signals be printed with print_fatal_signals. The last one, sched_debug, you can look up in the very excellent kernel documentation on kernel parameters.

title Arch Linux DEBUG Extreme
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro debug ignore_loglevel log_buf_len=10M print_fatal_signals=1 LOGLEVEL=8 earlyprintk=vga,keep sched_debug
initrd /kernel26.img

Insane Debug

The first few debugging examples showed some really nice kernel parameters to turn on really verbose debugging. This kind of debugging is absolutely critical if you want to max out your system or just learn more about what is going on behind the scenes. But there is a final trick that is my favorite, it's the ability to set both environment variables, and more importantly, module parameters at boot.

As an example, here is the boot line that I am using at the moment on an older Dell Desktop, just to illustrate module parameters and environment vars.

title  Arch Linux X-256
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait pause_on_oops=5 panic=60 i915.modeset=1 no_console_suspend ipv6.disable=1 TERM=xterm-256color quiet 5
initrd /kernel26.img

Since it's low on both memory and CPU, I disable ipv6. I also turn on kernel modesetting for the i915 video card, set my terminal to be xterm-256color, and boot straight into X. This lets me use a very optimized arch-linux configuration, amazing how fast thanks to using slim as the login manager, ratpoison as my window manager, and terminal with tmux as my login shell, all from boot, as the pstree shows (plus Synergy!).

init,1
  |-slim,3096
  |   |-X,3098 -nolisten tcp vt07 -auth /var/run/slim.auth
  |   `-ratpoison,3107,askapache
  |       |-terminal,5341 -x sh -c exec /usr/bin/tmux -2 -l -u -q attach -d -t tmux-askapache
  |       |   |-bash,11165
  |       |   |-tmux,5345 -2 -l -u -q attach -d -t tmux-askapache
  |       |   `-{terminal},5346
  |       `-xscreensaver,3113 -no-splash
  |-synergyc,6121,galileo -f --name galileo-fire --restart 10.66.66.2:26666
  |
  `-tmux,5348,askapache -2 -l -u -q attach -d -t tmux-askapache
      |-bash,5351
      |   `-ssh,9969 lug@askapache.com
      `-bash,5868
         `-vim,11149 -p sda1/grub/menu.lst /boot/grub/menu.lst

That kind of optimized system is only possible if you first can figure out your system, by debugging both the kernel as previously illustrated, debugging the init process, and most importantly, by debugging the modules enabled for your system's hardware/firmware/software. Debugging modules is challenging but worth the effort, and then you are able to do some truly insane debugging from grub like the following example, note that the actual grub entry is all on one line, but I split it into 4 lines so you could see it all. This basically turns on every module on this little Dell desktop to be at the absolute max debug level. There is so much logging when I boot this that the system grinds to a halt and is slower than a TI-89 calculator (See Improve Boot Performance).

title  Arch Linux DEBUG INSANE
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait ignore_loglevel debug debug_locks_verbose=1 sched_debug initcall_debug mminit_loglevel=4 udev.log_priority=8
       loglevel=8 earlyprintk=vga,keep log_buf_len=10M print_fatal_signals=1 apm.debug=Y i8042.debug=Y drm.debug=1 scsi_logging_level=1 usbserial.debug=Y
       option.debug=Y pl2303.debug=Y firewire_ohci.debug=1 hid.debug=1 pci_hotplug.debug=Y pci_hotplug.debug_acpi=Y shpchp.shpchp_debug=Y apic=debug
       show_lapic=all hpet=verbose lmb=debug pause_on_oops=5 panic=10 sysrq_always_enabled
initrd /kernel26.img

A couple key items from that grub entry are sysrq_always_enabled which forces on the sysrq magic, which really is a lifesaver when debugging at this level as your machine will freeze/stop-responding sometimes and it's nice to use sysrq to kill all tasks, change the loglevel, unmount all filesystems, or do a hard reboot. Another key parameter is the initcall_debug, which debugs the init process in excruciating detail. Very useful at times. The last parametery I find very useful is the udev.log_priority=8 to turn on udev logging.

Break Into Init

For instance, If you add break=y to your kernel cmdline, init will pause early in the boot process (after loading modules) and launch an interactive sh shell which can be used for troubleshooting purposes. (Normal boot continues after logout.) This is very similar to the shell that shows up if your computer gets turned off before it is able to shutdown properly. But using this parameter lets you enter into this mode differently at will.

title  Arch Linux Init Break
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro rootwait break=y
initrd /kernel26.img

Debugging init

This awesome parameter udev.log_priority=8 does the same thing as editing the file /etc/udev/udev.conf except it executes earlier, turning on debugging output for udev. If you want to know your hardware, that is the key parameter right there. Another trick is if you change the /etc/udev/udev.conf to be verbose, then you can make your initrd image include that file to turn on verbose udeb debugging by adding it to your {{Filename|/etc/mkinitcpio.conf} like:

FILES="/etc/modprobe.d/modprobe.conf /etc/udev/udev.conf"

, which on arch is as easy as

# mkinitcpio -p kernel26

Debugging udev is key because the initrd performs a root change at the end of its run to usually launch a program like /sbin/init as part of a chroot, and unless the new file system has a valid /dev directory, udev must be initialized before invoking chroot in order to provide /dev/console.

exec chroot . /sbin/init dev/console 2>&1

So basically, you aren't able to view the logs that are generated before /dev/console is initialized by udev or by a special initrd you compiled yourself. One method the kernel developers use to be able to still get the log messages generated before /dev/console is available is to provide an alternative console that you can enable or disable from grub.

Net Console

If you read through the kernel documentation regarding debugging, you will hear about Netconsole, which can be loaded from the kernel line in GRUB, compiled into your kernel, or loaded at runtime as a module. Having a netconsole entry in your menu.lst is most excellent for debugging slower computers like old laptops or thin-clients. It's easy to use. Just setup a 2nd computer (running arch) to accept syslog requests on a remote port, very fast and quick to do on arch-linux, 1 line to syslog.conf. Then you could use a log-color-parser like ccze to view all syslog logs, or just tail your everything.log. Then on your laptop, boot up and select the netconsole entry from the grub menu, and you will start seeing as much logging as you want on your syslog system. This logging lets you view even earlier log output than is available with the earlyprintk=vga kernel parameter, as netconsole is used by kernel hackers and developers, so it's very powerful.

title  Arch Linux DEBUG Netconsole
kernel /vmlinuz26 root=/dev/disk/by-label/ROOT ro netconsole=514@10.0.0.2/12:34:56:78:9a:bc debug ignore_loglevel
initrd /kernel26.img

Hijacking cmdline

If you do not have access to GRUB or the kernel boottime cmdline, like on a server or virtual machine, as long as you have root permissions you can still enable this kind of simplistic verbose logging using a neat hack. While you cannot modify the /proc/cmdline even as root, you can place your own cmdline file on top of /proc/cmdline, so that accessing /proc/cmdline actually accesses your file.

For example if I cat /proc/cmdline, I have the following:

root=/dev/disk/by-label/ROOT ro console=tty1 logo.nologo quiet

So I use a simple sed command to replace quiet with verbose like:

sed 's/ quiet/ verbose/' /proc/cmdline > /root/cmdline

Then I bind mount /root/cmdline so that it becomes /proc/cmdline, using the -n option to mount so that this mount won't be recorded in the systems mtab.

mount -n --bind -o ro /root/cmdline /proc/cmdline

Now if I cat /proc/cmdline, I have the following:

root=/dev/disk/by-label/ROOT ro console=tty1 logo.nologo verbose

Advanced Linux Debugging using a Bootloader (GRUB) originally appeared on AskApache.com

Creating an Offline Version of Page

AskApache — Fri, 22 Oct 2010 20:06:49 +0000

AskApache.com

A large client has a secure website where they assemble and create presentations consisting of a single Table of Contents page with many pdf's attached to it. They use the site to make presentations. This is a big client, a rich client, and they needed a way to guarantee they would be able to access the site. So I got this request:

Can you make an offline version of the Page that is always updated and available for download so we can download the offline version and present using that in case the website is down or more often, in case Internet Access is unavailable?

Update: I used COOKIE based authentication to secure this clients site, so that only logged in users can see anything at all, so how do I enable the curl requests to authenticate as well using the COOKIE of the requesting user in each request made by curl? Just add the users HTTP_COOKIE to the headers array used by curl like so:

array(
...
"Cookie: {$_SERVER['HTTP_COOKIE']}"
)

That now means the scraped version of the page is an exact duplicate that the user is looking at. Very sweet!

I can GET anything

So, here's what I hacked together last night, that is being used today. It's essentially 2 files.

A php file that scrapes uses curl to scrape all the urls for the page (favicon, css, images, pdfs, etc..)
A simple bash shell script acting as a cgi that creates a zip file of all the urls, and a self-extracting exe file for those without a winzip tool

The PHP File

This is a simple script that is given 2 parameters:

The url to scrape
The type of download to return

scrapeit.php

 $ch_info,
'curl_errno' => curl_errno($ch),
'curl_error' => curl_error($ch)
), 1));
curl_close($ch);
return $g;
}
 
/**
* _mkdir() makes a directory
*
* @return
*/
function _mkdir($path, $mode = 0755)
{
$old = umask(0);
$res = @mkdir($path, $mode);
umask($old);
return $res;
}
 
/**
* rmkdir()  recursively makes a directory tree
*
* @return
*/
function rmkdir($path, $mode = 0755)
{
return is_dir($path) || (rmkdir(dirname($path), $mode) && _mkdir($path, $mode));
}

The following should be in a couple functions, but I was running on a tight time schedule, and hey this $hitt aint free... wait yes it is, always.

// Ok lets get it on!
// first lets setup some variables
if (!isset($_GET['url']) || empty($_GET['url']))die();
$td = $th = $urls = array();
$FDATE = date("m-d-y-Hms");
$FTMP = '/home/askapache/sites/askapache.com/tmp';
$fetch_url = $_GET['url'];
$fu = parse_url($fetch_url);
$fd = substr($FTMP . $fu['path'], 0, - 1);
$FEXE = "{$fd}-{$FDATE}.exe";
$FZIP = "{$fd}-{$FDATE}.zip";
 
// now this is a shortcut to download the css file and add all the images in it to the img_urls array
$img_urls = array();
$gg = preg_match_all("/url\(([^\)]*?)\)/Ui", gogeturl('https://www.askapache.com/askapache-0128770124.css'), $th);
$imgs = array_unique($th[1]);
foreach($imgs as $img)
{
// only because all the links are relative
$img_urls[] = 'https://www.askapache.com' . $img;
}
 
// now fetch the main page, and assemble an array of all the external resources into the urls array
$gg = preg_match_all("/(background|href)=([\"\'])([^\"\'#]+?)([\"\'])/Ui", gogeturl($fetch_url), $th);
foreach($th[3] as $url)
{
if (strpos($url, '.js') !== false)continue;
if (strpos($url, 'wp-login.php') !== false || $url == 'https://www.askapache.com/') continue;
if (strrpos($url, '/') == strlen($url) - 1)continue;
if (strpos($url, 'https://www.askapache.com/') === false)
{
  if ($url[0] == '/') $urls[] = 'https://www.askapache.com' . $url;
  else continue;
}
else $urls[] = $url;
}
 
// now create a uniq array of urls, then download and save each of them
$urls = array_merge(array_unique($img_urls), array_unique($urls));
foreach($urls as $url)
{
  $pu = parse_url($url);
  rmkdir(substr($fd . $pu['path'], 0, strrpos($fd . $pu['path'], '/')));
  gogeturl2($url, $fd . $pu['path']);
}
 
// deletes dir ie. /this-page/this-page/ when it should be /this-page/index.html
if (is_dir($fd . $fu['path'])) rmdir($fd . $fu['path']);
 
// now save the page as index.html
gogeturl2($fetch_url, $fd . '/index.html');
 
// fixup to be able3 to parse
$g = file_get_contents($fd . '/index.html');
$g = str_replace('', 'script>!-->', $g);
$g = str_replace('href="https://www.askapache.com/', 'href="/', $g);
$g = str_replace('src="https://www.askapache.com/', 'src="/', $g);
$g = str_replace('href="/', 'href="', $g);
$g = str_replace('src="/', 'src="', $g);
$g = str_replace("href='https://www.askapache.com/", "href='/", $g);
$g = str_replace("src='https://www.askapache.com/", "src='/", $g);
$g = str_replace("href='/", "href='", $g);
$g = str_replace("src='/", "src='", $g);
file_put_contents($fd . '/index.html', $g);
 
// fixup for css file
foreach($urls as $url)
{
if (strpos($url, '.css') !== false)
{
  $fuu = parse_url($url);
  $css = file_get_contents($fd . $fuu['path']);
  $css = str_replace('url(/', 'url(../', $css);
  file_put_contents($fd . $fuu['path'], $css);
}
}
 
// my favorite technique, using fsockopen to initiate a shell script server-side.
// passing the args in the HTTP Headers... genius!!
// close the sucker fast with HTTP/1.0 and connection: close
$fp = fsockopen ($_SERVER['SERVER_NAME'], 80, $errno, $errstr, 5);
fwrite($fp, "GET /cgi-bin/sh/zip.sh HTTP/1.0\r\nHost: www.askapache.com\r\nX-Pad: {$fd}\r\nX-Allow: {$FDATE}\r\nConnection: Close\r\n\r\n");
fclose($fp);
 
// loop until the file created by /cgi-bin/sh/zip.sh is found
$c = 0;
do
{
$c++;
sleep(1);
clearstatcache();
if (is_file("{$FEXE}")) continue;
}
while ($c < 20);
 
// either zip or exe
$type = $_GET['type'];
if ($type == 'zip') $file = $FZIP;
else $file = $FEXE;
 
// wow great debugging dude
error_log($file);
 
// if the file is there, do a 302 redirect to initiate download
if (file_exists("{$file}"))
{
  @header("HTTP/1.1 302 Moved", 1, 302);
  @header("Status: 302 Moved", 1, 302);
  @header('Location: https://www.askapache.com/' . basename($file));
  exit;
}
 
exit;
?>

zip.sh

#!/bin/sh
 
# all you need for cgi
  echo -e "Content-type: text/plain\n\n"
 
# blank that run log
  echo "" > /home/askapache/sites/askapache.com/tmp/run.log
 
# redirect 1 and 2 to the run log for the whole script
  exec &>/home/askapache/sites/askapache.com/tmp/run.log
 
# basename
  N=${HTTP_X_PAD//*\/}
 
# date-based
  NN=${HTTP_X_ALLOW}
 
# create recursively the dir tree
  mkdir -pv $HTTP_X_PAD
 
# cd to the tmp
  cd /home/askapache/sites/askapache.com/tmp
 
# the zip version with date
  F=$N-$NN.zip
 
# the exe version with date
  NN=$N-$NN.exe
 
# for debugging, only goes to run log
  echo "F=$F"
  echo "NN=$NN"
  echo "N=$N"
 
# create a relative (r is recursive) archive of the entire dir
  /usr/bin/zip -rvv $F $N
 
# add the self-extracting stub to the archive
  /bin/cat unzipsfx.exe $F > $NN
 
# fix the sfx stub
  /usr/bin/zip -A $NN
 
# move both the exe and zip to the web-docroot to be dl'd directly
  cp -vf $NN /home/askapache/sites/askapache.com/htdocs/
  cp -vf $F /home/askapache/sites/askapache.com/htdocs/
 
# sleep for 60 seconds and then rm all the files, so you better download that file fast
  sleep 60 && rm -rvf $HTTP_X_PAD $F $NN /home/askapache/sites/askapache.com/htdocs/$NN /home/askapache/sites/askapache.com/htdocs/$F
 
#suh suh cya
exit 0;

Creating SFX Archives

The best way is 7z, but I couldn't get p7zip's sfx module to work and didn't have time to compile it. Instead I just used the stub available here: curl -O -L ftp://ftp.info-zip.org/pub/infozip/win32/unz552xn.exe which works great but no customization like password, icons, etc.. oh well.

Implementation

Simple, just create a link to the php file with the url and type parameters. /cgi-bin/php/scrapeit.php?url=http://www.askapache.com/htaccess/htaccess.html&type=exe or if you integrate into wordpress like I did you can add this to your header or admin_bar and use the get_permalink() for the url arg.

Lock It Down

This was used on a private site so what I did was add some code to the scrapeit.php file that just copied the HTTP_COOKIE value sent by the requesting user and sent that as part of the request in the fsockopen request. That means only logged in users can do this, and furthermore, if a user doesn't have access to a page and tries to use this to circumvent, they can't. And also htaccess was used to limit the scripts to only allow the ip's running the server to make connections.

Conclusion

What can't be done with linux, bash, http, php, and a little server-side finesse? My clients are very happy, and I had some fun!

Creating an Offline Version of Page originally appeared on AskApache.com

Ultimate Htaccess Part II

AskApache — Sat, 16 Oct 2010 16:11:33 +0000

AskApache.com

Here is even more information from the Ultimate Htaccess Part I. For now this is very rough and you will want to come back later to read it.

Merging Notes

The order of merging is:

(except regular expressions) and .htaccess done simultaneously (with .htaccess, if allowed, overriding )
(and )
and done simultaneously
and done simultaneously

Below is an artificial example to show the order of merging. Assuming they all apply to the request, the directives in this example will be applied in the order:

A > B > C > D > E


E


D



B



C


A

Htaccess Software

Apache HTTP Server comes with the following programs.

httpd: Apache hypertext transfer protocol server
apachectl: Apache HTTP server control interface
ab: Apache HTTP server benchmarking tool
apxs: APache eXtenSion tool
dbmmanage: Create and update user authentication files in DBM format for basic authentication
fcgistarter: Start a FastCGI program
htcacheclean: Clean up the disk cache
htdigest: Create and update user authentication files for digest authentication
htdbm: Manipulate DBM password databases.
htpasswd: Create and update user authentication files for basic authentication
httxt2dbm: Create dbm files for use with RewriteMap
logresolve: Resolve hostnames for IP-addresses in Apache logfiles
log_server_status: Periodically log the server's status
rotatelogs: Rotate Apache logs without having to kill the server
split-logfile: Split a multi-vhost logfile into per-host logfiles
suexec: Switch User For Exec

Options All
AuthType "Basic"
AuthName "Protected Access"
 
#SetEnv APPLICATION_ENV production
SetEnv APPLICATION_ENV development
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -s [OR]
 
ErrorDocument 404 /index.php
 
AddType text/html url
 
order allow,deny
allow from all
Options +ExecCGI
 
AddHandler python-program .py
PythonHandler mod_python.publisher
PythonDebug On
PythonAutoReload On
 
RewriteEngine on
RewriteRule .* - [E=HTTP_IF_MODIFIED_SINCE:%{HTTP:If-Modified-Since}]
RewriteRule .* - [E=HTTP_IF_NONE_MATCH:%{HTTP:If-None-Match}]
 
# updating tab.ser takes lots of memory
php_value memory_limit 64000000
 
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -s [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^.*$ - [NC,L]
 
php_flag magic_quotes_gpc Off
php_flag magic_quotes_runtime Off
 
# redirect moved files
RedirectMatch Permanent ^/fop/anttask(.*) http://xmlgraphics.apache.org/fop/0.20.5/anttask$1
RedirectMatch Permanent ^/fop/compiling(.*) http://xmlgraphics.apache.org/fop/0.20.5/compiling$1
RedirectMatch Permanent ^/fop/configuration(.*) http://xmlgraphics.apache.org/fop/0.20.5/configuration$1
RedirectMatch Permanent ^/fop/embedding(.*) http://xmlgraphics.apache.org/fop/0.20.5/embedding$1
 

ExpiresActive On
ExpiresDefault "access plus 52 weeks"

 
# $Id: .htaccess 1255 2003-12-21 05:30:57Z rufustfirefly $
# $Author: rufustfirefly $
#
# Since sometimes Apache isn't properly configured to view the
# proper PHP script, we do this instead, but only if PHP4 is
 


Order Allow,Deny
Allow from 127.0.0.1
RewriteEngine On
# map phocoa framework wwwroot - enabled PHOCOA Versioning
RewriteRule ^/www/framework(/[0-9\.]*)?/?(.*) /wwwroot/www/framework/$2 [L]
#enable a normal wwwroot
 
RewriteEngine on
RewriteRule Boot/(.*) Boot/$1 [QSA,L]
 
RewriteEngine on
RewriteRule ^xbl.js build/xbl.php
RewriteRule ^ie-dom.js build/ie-dom.php
 
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php [QSA,L]
 
# Rewrite rules for Zend Framework
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) index.php/$1 [L]
 
AuthUserFile /dev/null
AuthGroupFile /dev/null
AuthName "You are not welcomed here"
AuthType Basic

 
AddHandler application/x-httpd-php-source php
 
DirectoryIndex index.html index.php
AddDefaultCharset UTF-8
RewriteEngine off
# Web Optimizer options
 
ErrorDocument 404 /~george/blog/index.php
AddType application/x-httpd-php .php .rdf
 
# redirect moved files
RedirectMatch Permanent ^/fop/anttask(.*) http://xmlgraphics.apache.org/fop/0.20.5/anttask$1
RedirectMatch Permanent ^/fop/compiling(.*) http://xmlgraphics.apache.org/fop/0.20.5/compiling$1
RedirectMatch Permanent ^/fop/configuration(.*) http://xmlgraphics.apache.org/fop/0.20.5/configuration$1
RedirectMatch Permanent ^/fop/embedding(.*) http://xmlgraphics.apache.org/fop/0.20.5/embedding$1
AddHandler x-httpd-php504 .php
AddType application/x-httpd-php .html
 
RewriteEngine on
# Remember to change RewriteBase to where your site really is - for example if
# you access the site by www.somesite.com/~myname/thesite, you do as so:
# RewriteBase /~myname/thesite/
 

Order deny,allow
Deny From All

 
#
# Apache/PHP/Drupal settings:
#
# Protect files and directories from prying eyes.
 
DirectoryIndex membersite.pl
Options +ExecCGI
AddHandler cgi-script .pl
 
#AddType application/x-httpd-php .php3 .phtml .php .php4 .php5 .html .html .xml
Options -Indexes
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
 
# Comment/uncomment the lines below depending of your needs
# To deny any access to this directory
Deny from all
 

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ index.php/$1 [L]
 

Order allow,deny
Deny from all

 
AddHandler application/x-httpd-php5 .php5 .php4 .php .php3 .php2 .phtml
AddType application/x-httpd-php5 .php5 .php4 .php .php3 .php2 .phtml
AddHandler application/x-httpd-php5 .snow
AddType application/x-httpd-php5 .snow
AddHandler image/jpeg me
Redirect permanent /~hoge/hns-lite/i/ http://www.example.ne.jp/~hoge/hns-lite/
 
# Make index.php the default
DirectoryIndex index.php
Options -Indexes
 
# In case it's on, turn off magic quotes:
php_value magic_quotes_gpc off
# On production servers, uncomment these lines and update the path
# to a suitable log file:
 
Deny from all
Satisfy All
 
##############################
# MicroMVC Apache settings
# .htaccess v1.0.0 9/29/2007
##############################
 

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
 

DENY FROM ALL

 
RewriteEngine on
#RewriteCond %{REMOTE_HOST} ^yupi-cms*
RewriteCond %{REQUEST_FILENAME} !-f
#RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php/$1 [L]
 
RewriteEngine On
RewriteBase /
# Rewrite www.domain.com to domain.com
RewriteCond %{HTTP_HOST} ^www\.(.*)
 
#
# OntoWiki requires Apache's rewrite engine to work
#

RewriteEngine Off

# Turn on URL rewriting
RewriteEngine On
 
RewriteEngine on
# Remember to change RewriteBase to where your site really is - for example if
# you access the site by www.somesite.com/~myname/thesite, you do as so:
# RewriteBase /~myname/thesite/
 
# this file is placed here to restrict the access to this directory
# for non authorized users
# Note that the AllowOverride directive (in the Apache's access.conf) must be
# set to "Limit" or "All"
 
# This folder does not require access over HTTP
Order deny,allow
Deny from all
Allow from none
 
RewriteEngine on
# Remember to change RewriteBase to where your site really is - for example if
# you access the site by www.somesite.com/~myname/thesite, you do as so:
# RewriteBase /~myname/thesite/
 
#
# Latitude .htaccess Rules
# Special thanks to Drupal for a template (http://drupal.org/)
#
 
AddHandler php5-script .php
Options +FollowSymLinks +ExecCGI

RewriteEngine On
 
Options +Indexes
IndexOptions FancyIndexing
IndexOptions +NameWidth=*
IndexOptions +DescriptionWidth=*
 
RewriteEngine off
Options -Indexes
 
############
# SECURITY #
###########################################################

Options +Indexes
IndexOptions FancyIndexing
IndexOptions +NameWidth=*
IndexOptions +DescriptionWidth=*
 
AddHandler fcgid-script .abc
#update this path according to your setup
#you can place the executable fcgishell anywhere on your server
 
AddEncoding x-compress Z
AddEncoding x-gzip gz
 
AddHandler cgi-script rb
# Send any page with a slow.html suffix with a delay
AddHandler slow-down .slow
Action slow-down @PATH_PREFIX@/cgi/send-delayed.rb
 
DirectoryIndex index.htm
AddHandler server-parsed .htm
 
# for current work

php_value memory_limit 128M
php_value max_execution_time 150
php_flag magic_quotes_gpc off
 

Deny from all

 
ErrorDocument 404 /cgi-bin/error.cgi
 
deny from all

order allow,deny
deny from all

 
Options -Indexes

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
# General Apache options
AddHandler fastcgi-script .fcgi
AddHandler cgi-script .cgi
Options +FollowSymLinks +ExecCGI
 
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
# RewriteRule !\.(js|ico|gif|jpg|png|css|txt)$ - [C]
RewriteRule (.*) http://127.0.0.1:8080/$1 [P]
 
RewriteEngine on
RewriteCond $1 !^(index\.php|images|robots\.txt)
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?$1
 
Order Deny,Allow
Deny from all
#dozwoleni userzy
Allow from localhost
 
# deny access to private folders and files, start with prefix "_"

deny from all

# deny access to all executable files (.php)
 
php_flag register_globals off
php_flag magic_quotes_gpc off
php_flag short_open_tag off
php_flag allow_call_time_pass_reference off
php_flag safe_mode on
 

RewriteEngine on
RewriteBase /jblog/
RewriteCond %{REQUEST_FILENAME} !-f
 
deny from all
Satisfy all
 
#
# Apache/PHP/Drupal settings:
#
# Protect files and directories from prying eyes.
 
RewriteEngine On
RewriteRule ^(\w*)/post$ index.php?at=$1&do=post [L,QSA]
RewriteRule ^edit/(\d*)$ index.php?do=edit&to=$1 [L,QSA]
RewriteRule ^comment/(\d*)$ index.php?do=comment&to=$1 [L,QSA]

Technical Look at .htaccess

Source: Apache API notes

Per-directory configuration structures

Let's look out how all of this plays out in mod_mime.c, which defines the file typing handler which emulates the NCSA server's behavior of determining file types from suffixes. What we'll be looking at, here, is the code which implements the AddType and AddEncoding commands. These commands can appear in .htaccess files, so they must be handled in the module's private per-directory data, which in fact, consists of two separate tables for MIME types and encoding information, and is declared as follows:

table *forced_types;      /* Additional AddTyped stuff */
table *encoding_types;    /* Added with AddEncoding... */
mime_dir_config;

When the server is reading a configuration file, or section, which includes one of the MIME module's commands, it needs to create a mime_dir_config structure, so those commands have something to act on. It does this by invoking the function it finds in the module's `create per-dir config slot', with two arguments: the name of the directory to which this configuration information applies (or NULL for srm.conf), and a pointer to a resource pool in which the allocation should happen.

(If we are reading a .htaccess file, that resource pool is the per-request resource pool for the request; otherwise it is a resource pool which is used for configuration data, and cleared on restarts. Either way, it is important for the structure being created to vanish when the pool is cleared, by registering a cleanup on the pool if necessary).

For the MIME module, the per-dir config creation function just ap_pallocs the structure above, and a creates a couple of tables to fill it. That looks like this:

void *create_mime_dir_config (pool *p, char *dummy)
mime_dir_config *new = (mime_dir_config *) ap_palloc (p, sizeof(mime_dir_config));
 
new->forced_types = ap_make_table (p, 4);
new->encoding_types = ap_make_table (p, 4);

Now, suppose we've just read in a .htaccess file. We already have the per-directory configuration structure for the next directory up in the hierarchy. If the .htaccess file we just read in didn't have any AddType or AddEncoding commands, its per-directory config structure for the MIME module is still valid, and we can just use it. Otherwise, we need to merge the two structures somehow.

To do that, the server invokes the module's per-directory config merge function, if one is present. That function takes three arguments: the two structures being merged, and a resource pool in which to allocate the result. For the MIME module, all that needs to be done is overlay the tables from the new per-directory config structure with those from the parent:

void *merge_mime_dir_configs (pool *p, void *parent_dirv, void *subdirv)
mime_dir_config *parent_dir = (mime_dir_config *)parent_dirv;
mime_dir_config *subdir = (mime_dir_config *)subdirv;
mime_dir_config *new =  (mime_dir_config *)ap_palloc (p, sizeof(mime_dir_config));
new->forced_types = ap_overlay_tables (p, subdir->forced_types, parent_dir->forced_types);
new->encoding_types = ap_overlay_tables (p, subdir->encoding_types, parent_dir->encoding_types);

As a note --- if there is no per-directory merge function present, the server will just use the subdirectory's configuration info, and ignore the parent's. For some modules, that works just fine (e.g., for the includes module, whose per-directory configuration information consists solely of the state of the XBITHACK), and for those modules, you can just not declare one, and leave the corresponding structure slot in the module itself NULL.

Command handling

Now that we have these structures, we need to be able to figure out how to fill them. That involves processing the actual AddType and AddEncoding commands. To find commands, the server looks in the module's command table. That table contains information on how many arguments the commands take, and in what formats, where it is permitted, and so forth. That information is sufficient to allow the server to invoke most command-handling functions with pre-parsed arguments. Without further ado, let's look at the AddType command handler, which looks like this (the AddEncoding command looks basically the same, and won't be shown here):

char *add_type(cmd_parms *cmd, mime_dir_config *m, char *ct, char *ext)
if (*ext == '.') ++ext;
ap_table_set (m->forced_types, ext, ct);

This command handler is unusually simple. As you can see, it takes four arguments, two of which are pre-parsed arguments, the third being the per-directory configuration structure for the module in question, and the fourth being a pointer to a cmd_parms structure. That structure contains a bunch of arguments which are frequently of use to some, but not all, commands, including a resource pool (from which memory can be allocated, and to which cleanups should be tied), and the (virtual) server being configured, from which the module's per-server configuration data can be obtained if required.

Another way in which this particular command handler is unusually simple is that there are no error conditions which it can encounter. If there were, it could return an error message instead of NULL; this causes an error to be printed out on the server's stderr, followed by a quick exit, if it is in the main config files; for a .htaccess file, the syntax error is logged in the server error log (along with an indication of where it came from), and the request is bounced with a server error response (HTTP error status, code 500).

The MIME module's command table has entries for these commands, which look like this:

command_rec mime_cmds[] =
{ "AddType", add_type, NULL, OR_FILEINFO, TAKE2, "a mime type followed by a file extension" },
{ "AddEncoding", add_encoding, NULL, OR_FILEINFO, TAKE2, "an encoding (e.g., gzip), followed by a file extension" },

The entries in these tables are:

The name of the command
The function which handles it a (void *) pointer, which is passed in the cmd_parms structure to the command handler --- this is useful in case many similar commands are handled by the same function.
A bit mask indicating where the command may appear. There are mask bits corresponding to each AllowOverride option, and an additional mask bit, RSRC_CONF, indicating that the command may appear in the server's own config files, but not in any .htaccess file.
A flag indicating how many arguments the command handler wants pre-parsed, and how they should be passed in. TAKE2 indicates two pre-parsed arguments. Other options are TAKE1, which indicates one pre-parsed argument, FLAG, which indicates that the argument should be On or Off, and is passed in as a boolean flag, RAW_ARGS, which causes the server to give the command the raw, unparsed arguments (everything but the command name itself). There is also ITERATE, which means that the handler looks the same as TAKE1, but that if multiple arguments are present, it should be called multiple times, and finally ITERATE2, which indicates that the command handler looks like a TAKE2, but if more arguments are present, then it should be called multiple times, holding the first argument constant.
Finally, we have a string which describes the arguments that should be present. If the arguments in the actual config file are not as required, this string will be used to help give a more specific error message. (You can safely leave this NULL).

Finally, having set this all up, we have to use it. This is ultimately done in the module's handlers, specifically for its file-typing handler, which looks more or less like this; note that the per-directory configuration structure is extracted from the request_rec's per-directory configuration vector by using the ap_get_module_config function.

Side notes --- per-server configuration, virtual servers, etc.

The basic ideas behind per-server module configuration are basically the same as those for per-directory configuration; there is a creation function and a merge function, the latter being invoked where a virtual server has partially overridden the base server configuration, and a combined structure must be computed. (As with per-directory configuration, the default if no merge function is specified, and a module is configured in some virtual server, is that the base configuration is simply ignored).

The only substantial difference is that when a command needs to configure the per-server private module data, it needs to go to the cmd_parms data to get at it. Here's an example, from the alias module, which also indicates how a syntax error can be returned (note that the per-directory configuration argument to the command handler is declared as a dummy, since the module doesn't actually have per-directory config data):

Ultimate Htaccess Part II originally appeared on AskApache.com