linux.bash
Crazy POWERFUL Bash Prompt
This amazing bash linux prompt does more than meets the eye. If you want to know how to become really good with technology, linux is the secret sauce behind the AskApache articles. Open Source is elixir of the web. Thanks to everyone who helped me for the past 20 years. I use linux/bsd because homey don't play, so this is geared to be as productive a prompt as I can make it.
Don't have much time.. or just don't care? Not a problem, here are the 3 lines to copy and paste - you can just paste them right in your shell to test it, or add to a startup script.
export AA_P="export PVE=\"\\033[m\\033[38;5;2m\"\$(( \`sed -n \"s/MemFree:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\` / 1024 ))\"\\033[38;5;22m/\"\$((\`sed -n \"s/MemTotal:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\`/ 1024 ))MB\"\\t\\033[m\\033[38;5;55m\$(< /proc/loadavg)\\033[m\";echo -en \"\""
export PROMPT_COMMAND="history -a;((\$SECONDS % 10==0 ))&&eval \"\$AA_P\";echo -en \"\$PVE\";"
export PS1="\\[\\e[m\\n\\e[1;30m\\][\$\$:\$PPID \\j:\\!\\[\\e[1;30m\\]]\\[\\e[0;36m\\] \\T \\d \\[\\e[1;30m\\][\\[\\e[1;34m\\]\\u@\\H\\[\\e[1;30m\\]:\\[\\e[0;37m\\]\${SSH_TTY} \\[\\e[0;32m\\]+\${SHLVL}\\[\\e[1;30m\\]] \\[\\e[1;37m\\]\\w\\[\\e[0;37m\\] \\n(\$SHLVL:\\!)\\\$ " && eval $AA_P
Custom bash_profile for Advanced Shell Users
Looking for some advanced uses for the shell? Here is some of my best. The shell is where 70% of my work takes place, and I have at least one terminal open almost 100% of the time, for viewing tailing color-coded logs, and of course for the SSH Tunnels that I use to route various networking through, like my email. So I decided that to standardize and create a bash_profile containing the most time-saving and helpful functions that I could use on all the various hosting environments would really be some sweet sugar, so here is my constant Work-in-progress.
It works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.
I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:
#!/bin/bash source ~/.bash_profile &>/dev/nulll pm "PM is a function to output nice messages with color" yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS" yn "Show Calendar" && aa_calendar yn "Show Fortune" && aa_fortune
Bash alternative to Reflector for Ranking Mirrors
A pure bash alternative to the python reflector, using curl, xargs, and sort for ranking Arch Linux mirrors.
Nice and simple. Short and sweet.
Magic in the Terminal: Screen, Bash, and SSH
Oh ya lets get it on! short but sweet
Bash Functions and Aliases for Traps, Kills, and Signals
Bash Functions and Aliases for Traps, Kills, and Signals. Useful for translating signal numbers/signal names and getting more info about signals.
Bash Script to Create index.html of Dir Listing
If you use Apache to auto-generate directory index listings of files/dirs, and you have a large number of files and directories in the root directory and/or slow IO speed, then generating the index could take Apache over a minute!
Custom Webmaster Shell Environment
Webmasters often use SSH to remotely admin web sites and servers, learn how to make your shell colorful and powerful using my sample .bash_profile and you can be up and running in 30 seconds!
RXVT Customization with ~/.Xresources
After many years of using any and all terminal emulators out there, from xterm to the Gnome terminal, to KDE Konsole to xfce4-terminal, lxterminal, vte, yakuake, rote, roxterm, sakura, terminator, and putty I finally I settled in for the long-haul with rxvt (rxvt-unicode).
My BOX: Slim -> Ratpoison -> URxvt -> Tmux -> Bash
Mod_Status tricks to View Apache Module Directives
Quick and easy method to get a list of all Apache Modules currently loaded, a list of all the directives each module provides, a list of currently used directives, etc... These directives can be used in httpd.conf and/or .htaccess files so it is definately useful to know which ones are available and which ones are currently being used.
Fix Linux Users Home Permissions with a Cron Job
As a security nut myself, and also a Linux admin, one of my biggest pet peeves is when I've taken the time and care to segment all the users on a server into separate home directories, and then some developer comes along, logs in as root, and changes the ownership of files. Other things can cause this, like Apache, PHP, Mutt, etc.. So I've always used a cron job that executes daily (and on demand) which automatically fixes all the permissions back to what they should be.
Generating AddDescription for all Files
Wget Trick to Download from Restrictive Sites
Before
After trick
I am often logged in to my servers via SSH, and I need to download a file like a WordPress plugin. I've noticed many sites now employ a means of blocking robots like wget from accessing their files. Most of the time they use .htaccess to do this. So a permanent workaround has wget mimick a normal browser.
View all MySQL Variables for Pasting into my.cnf
This is really useful for me because I work with dozens of different database servers. The first thing I do is run this command and paste it into the servers /etc/my.cnf file. That way I will always know the original value and it just makes life much easier.
$ mysql -NBe 'SHOW VARIABLES' |sed 's,\t,^=,'|column -ts^|tr "\n" '@'|eval $(echo "sed '" "s,@\("{a..z}"\),\n\n\1,;" "'")|tr '@' "\n"|sed 's,^,# ,g'
HOWTO: Uninstall CPANEL over SSH
The following is just a quick but detailed reference of some of the commands I used to successfully uninstall cpanel permanently. This is for advanced users of the shell. If you run a single one of these commands without fully understanding it, you will probably kill your server, lose everything on it permanently, not have a website or email for weeks.. So backup everything. FIRST. You should also contact your hosting provider support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for web-hosting companies, and you are less than a drop in their bucket.
TMUX
Stop wasting your lives with Mac Terminals.. or Macs. Get a real machine and then get a real shell multiplexer! For many years we all loved GNU Screen, but tmux is by far a better option today. The only time I am in the shell and not using a multiplexer, is when I'm not on one of my machines. My Arch Linux machines all run URxvt and my .bash_profiles all start tmux automataically, whether in X or single-user mode, tmux is where it's at.
Install multiple OS Without Cds
Orig published 2006. I had a CD-RW drive but being a computer security researcher I had no money for blank cd-recordables. What follows is how I managed to install various operating systems on my computer (1 hard drive) without having to burn to a CD the ISO and then boot from that.
And also:
==Phrack Inc.==
Volume 0x0b, Issue 0x3f, Phile #0x0a of 0x14
|=-----------------=[ Hacking Grub for fun and profit ]=-----------------=|
Chmod, Umask, Stat, Fileperms, and File Permissions
Unix file permissions are one of the more difficult subjects to grasp.. Well, ok maybe "grasp" isn't the word.. Master is the right word.. Unix file permissions is a hard topic to fully master, mainly I think because there aren't many instances when a computer user encounters them seriously, and bitwise is oldschool. This contains a listing of all possible permission masks and bits from a linux, php, and web hosting view.... cuz you guys AskApache Regs Rock!
Search And Replace shell script helpful for Upgrades
Search all files in a directory, replacing all occurances of string with a replacement string.
Linux Debugging using a Bootloader with Kernel Parameters
As an example, here is the boot line that I am using at the moment on an older Dell Desktop, just to illustrate module parameters and environment vars.
title Arch Linux X-256 linux /vmlinuz-linux root=UUID rw rootfstype=ext4 elevator=noop selinux=0 plymouth.enable=0 raid=noautodetect ipv6.disable=1 video=DP1:d rcutree.rcu_idle_gp_delay=1 i915.panel_ignore_lid=-1 i915.modeset=1 TERM=rxvt-256color
Creating an Offline Version of Page
So, here's what I hacked together last night, that is being used today. It's essentially 2 files.
- A php file that scrapes uses curl to scrape all the urls for the page (favicon, css, images, pdfs, etc..)
- A simple bash shell script acting as a cgi that creates a zip file of all the urls, and a self-extracting exe file for those without a winzip tool
Optimized Vimrc with 256 Colorscheme
This is not your everyday vimrc. This is my personalized vimrc.. works especially great on remote servers, in tmux, screen, etc.
This doesn't require a git checkout, a vim update, a bash update, an OS update, a plugin. No. 1 vimrc, 1 colorscheme, just use curl to download and it's on. I use this on Arch Linux, NetBSD, FreeBSD, OpenBSD, Debian, Ubuntu, Cent OS, Red Hat, and that's mostly it so far.
If you aren't already using vim as your primary editor in a powerful way, you are gonna owe me for this.
Actual Htaccess Files from My Server
#### No https except to wp-admin -
# If the request is empty ( implies fopen or normal file access by a php script )
RewriteCond %{THE_REQUEST} ^$ [OR]
# OR if the request if for wp-admin or wp-login.php
RewriteCond %{REQUEST_URI} ^/(wp-admin|wp-login.php).*$ [NC,OR]
# OR if the Referer is https
RewriteCond %{HTTP_REFERER} ^https://www.askapache.com/.*$ [NC]
# THEN skip the following rule, basically all this does is force https or badhost to be redirected
# BUT because of the above 3 rewritecond's, this won't break poorly written admin scripts
RewriteRule .* - [S=1]
RewriteCond %{HTTPS} =on [OR]
RewriteCond %{HTTP_HOST} !^www.askapache.com$ [NC]
RewriteRule .* https://www.askapache.com%{REQUEST_URI} [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9} /(wp-admin/.*|wp-login.php.*) HTTP/ [NC]
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]
ASCII Chart
Wanted to stick this here for a reference, mostly for me. I use ASCII alot in bash, preg_matches, preg_replace, etc..
Unicode Character Reference
Just a quick reference to all those delicious unicode characters and how they render on the web‽‽
Building strace-plus
strace+ is an improved version of strace that collects stack traces associated with each system call. Since system calls require an expensive user-kernel context switch, they are often sources of performance bottlenecks. strace+ allows programmers to do more detailed system call profiling and determine, say, which call sites led to costly syscalls and thus have potential for optimization.
DreamHost ProcWatch
Pimp out your FeedBurner Count
I've had a lot of people ask about the FeedBurner FeedCount image on AskApache. Specifically how to set it up with custom messages and different colors each page view... It is pretty sweet..
Elite Log File Scrolling with Color Syntax
Scrolls the latest log entries for multiple log files to the current screen or to any other monitor or TTY in color using syntax highlighting, making debugging easier and saving a lot of time for multi-monitor workstations.
THE Ultimate Htaccess
.htaccess is a very ancient configuration file for web servers, and is one of the most powerful configuration files most webmasters will ever come across. This htaccess guide shows off the very best of the best htaccess tricks and code snippets from hackers and server administrators.
You've come to the right place if you are looking to acquire mad skills for using .htaccess files!
Optimizing Servers and Processes for Speed with ionice, nice, ulimit
To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.
Shell Scripts for Dreamhost
Example shell scripts (bash or sh) that automate and easyate tasks from within the dreamhost ssh env
Upgrading to DreamHost Private Servers
DreamHost PS gives you your own "virtual machine", protecting your CPU and RAM on your physical machine for faster websites. Here's what I like and dislike about DreamHostPS, and some of the issues and solutions for migrating.
Serve External Javascript Files locally for Increased Speed
One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally. In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.
Get Number of Running Proccesses with PHP
Recently I had to setup a script to curl 10k urls, but it could only do 500 requests at any one time. In order to work under that limit, I created a function that returns the number of currently running processes on the machine in an extremely fast and efficient way, thus allowing the curl_multi requests to queu themselves such as GNU xargs.
Htaccess File Sample
Comprehensive .htaccess example file with advanced examples in 1 htaccess sample skeleton .htaccess file with the very best apache htaccess examples... Updated frequently based on detailed info from the Apache htaccess tutorial.
PHP and AJAX shell console
Ever wanted to execute commands on your server through php? Now you can. I'm calling this file (see below) shell.php and it allows you to run commands on your web server with the same permissions that your php executable has.
Apache Web Server Speed Configuration Hacks
Apache server performance can be improved by adding additional hardware resources such as RAM, faster CPU, etc.Securing php.ini and php.cgi with .htaccess
If you have a php.cgi or php.ini file in your /cgi-bin/ directory or other pub directory, try requesting them from your web browser. If your php.ini shows up or worse you are able to execute your php cgi, you'll need to secure it ASAP. This shows several ways to secure these files, and other interpreters like perl, fastCGI, bash, csh, etc.
