Shell Scripting
Unicode Character Reference
Just a quick reference to all those delicious unicode characters and how they render on the web‽‽
Awk Tutorial and Introduction
While researching a unix/linux tool awk I came upon one of the most thorough and helpful tutorials I've ever seen devoted to a particular topic. It's old-school just the way I like it. I contacted the author, Bruce Barnett because I just HAD to have this article for my readers, who are predominantly running solaris/unix/bsd/linux and he kindly gave permission.
Actual Htaccess Files from My Server
#### No https except to wp-admin -
# If the request is empty ( implies fopen or normal file access by a php script )
RewriteCond %{THE_REQUEST} ^$ [OR]
# OR if the request if for wp-admin or wp-login.php
RewriteCond %{REQUEST_URI} ^/(wp-admin|wp-login.php).*$ [NC,OR]
# OR if the Referer is https
RewriteCond %{HTTP_REFERER} ^https://www.askapache.com/.*$ [NC]
# THEN skip the following rule, basically all this does is force https or badhost to be redirected
# BUT because of the above 3 rewritecond's, this won't break poorly written admin scripts
RewriteRule .* - [S=1]
RewriteCond %{HTTPS} =on [OR]
RewriteCond %{HTTP_HOST} !^www.askapache.com$ [NC]
RewriteRule .* https://www.askapache.com%{REQUEST_URI} [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9} /(wp-admin/.*|wp-login.php.*) HTTP/ [NC]
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]
Advanced Windows Batch File Scripting
If for some reason you need to use windows .bat batch file scripting to do some task, or you just want to learn the most universal batch programming language on Windows machines, you lucked out and found the AskApache example. My expertise is the shell, in this article the shell is Windows cmd.exe. It has some pretty advanced windows shell usage, including pipes and redirection, but it's the modular linux-like coding approach that earns this script it's "advanced" title.
:SETPROMPT set PROMPT=$_[%USERNAME%@%USERDOMAIN%]$S[$P]$_$M$G && EXIT /B
My favorite tool (and I've tried sooo many) for editing most Windows files and especially .bat files is the free and open-source Notepad++. Set that up and you will have a color-syntax-highlighted editor for Batch Scripting that works very very well.
Automated Folder Backup Shell-Script
This simple unix shell script automatically creates backups of a specific folder at regular hourly, nightly, weekly, and monthly intervals. Instead of the usual method for copying directory trees using tar with fifo, pipes, rsync, or NFS methods this script uses cpio which is much much faster and has cool options like saving m/a/c times, symlinks, relative paths, and weird file names.
Search And Replace shell script helpful for Upgrades
Search all files in a directory, replacing all occurances of string with a replacement string.
PHP5 Custom Install Shell Script Example
Today I successfully learned how to compile and run multiple custom php installations for a DreamHost account, and to get it working I came upon a simple shell script that I made a couple changes to.
Custom bash_profile for Advanced Shell Users
Looking for some advanced uses for the shell? Here is some of my best. The shell is where 70% of my work takes place, and I have at least one terminal open almost 100% of the time, for viewing tailing color-coded logs, and of course for the SSH Tunnels that I use to route various networking through, like my email. So I decided that to standardize and create a bash_profile containing the most time-saving and helpful functions that I could use on all the various hosting environments would really be some sweet sugar, so here is my constant Work-in-progress.
It works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.
I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:
#!/bin/bash source ~/.bash_profile &>/dev/nulll pm "PM is a function to output nice messages with color" yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS" yn "Show Calendar" && aa_calendar yn "Show Fortune" && aa_fortune
THE Ultimate Htaccess
.htaccess is a very ancient configuration file for web servers, and is one of the most powerful configuration files most webmasters will ever come across. This htaccess guide shows off the very best of the best htaccess tricks and code snippets from hackers and server administrators.
You've come to the right place if you are looking to acquire mad skills for using .htaccess files!
On-Demand MySQL Backup Shell Script
On-Demand WordPress MySQL database Backup Shell Script for DreamHostShell Scripts for Dreamhost
Example shell scripts (bash or sh) that automate and easyate tasks from within the dreamhost ssh env
Serve External Javascript Files locally for Increased Speed
One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally. In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.
FastCGI on DreamHost
Using FastCGI on DreamHost and .htaccess
PHP and AJAX shell console
Ever wanted to execute commands on your server through php? Now you can. I'm calling this file (see below) shell.php and it allows you to run commands on your web server with the same permissions that your php executable has.
Fix Linux Users Home Permissions with a Cron Job
As a security nut myself, and also a Linux admin, one of my biggest pet peeves is when I've taken the time and care to segment all the users on a server into separate home directories, and then some developer comes along, logs in as root, and changes the ownership of files. Other things can cause this, like Apache, PHP, Mutt, etc.. So I've always used a cron job that executes daily (and on demand) which automatically fixes all the permissions back to what they should be.
Apache Environment Variables CGI Script
My improved version of the common printenv / test-cgi scripts for debugging Apache environment variables set in .htaccess files
Bash alternative to Reflector for Ranking Mirrors
A pure bash alternative to the python reflector, using curl, xargs, and sort for ranking Arch Linux mirrors.
Nice and simple. Short and sweet.
Custom Webmaster Shell Environment
Webmasters often use SSH to remotely admin web sites and servers, learn how to make your shell colorful and powerful using my sample .bash_profile and you can be up and running in 30 seconds!
Custom PHP.ini tips and tricks
- When php run as Apache Module (mod_php)
- When php run as CGI
- When cgi?d php is run with wrapper (for FastCGI)
MySQL Performance Tuning Scripts and Know-How
Unless you are a total linux-freak-guru like myself, and even if you are, it can be enormously challenging and somewhat overwhelming to locate and eliminate MySQL bottlenecks. While many DBAs focus on improving the performance of the queries themselves, this post will focus on the highest-impact items: MySQL Server Performance and OS Performance for MySQL. Specifically showing 4 awesome tools to auto-tune your MySQL and a few more tools just for monitoring!
Bash Script to Create index.html of Dir Listing
If you use Apache to auto-generate directory index listings of files/dirs, and you have a large number of files and directories in the root directory and/or slow IO speed, then generating the index could take Apache over a minute!
Crazy POWERFUL Bash Prompt
This amazing bash linux prompt does more than meets the eye. If you want to know how to become really good with technology, linux is the secret sauce behind the AskApache articles. Open Source is elixir of the web. Thanks to everyone who helped me for the past 20 years. I use linux/bsd because homey don't play, so this is geared to be as productive a prompt as I can make it.
Don't have much time.. or just don't care? Not a problem, here are the 3 lines to copy and paste - you can just paste them right in your shell to test it, or add to a startup script.
export AA_P="export PVE=\"\\033[m\\033[38;5;2m\"\$(( \`sed -n \"s/MemFree:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\` / 1024 ))\"\\033[38;5;22m/\"\$((\`sed -n \"s/MemTotal:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\`/ 1024 ))MB\"\\t\\033[m\\033[38;5;55m\$(< /proc/loadavg)\\033[m\";echo -en \"\""
export PROMPT_COMMAND="history -a;((\$SECONDS % 10==0 ))&&eval \"\$AA_P\";echo -en \"\$PVE\";"
export PS1="\\[\\e[m\\n\\e[1;30m\\][\$\$:\$PPID \\j:\\!\\[\\e[1;30m\\]]\\[\\e[0;36m\\] \\T \\d \\[\\e[1;30m\\][\\[\\e[1;34m\\]\\u@\\H\\[\\e[1;30m\\]:\\[\\e[0;37m\\]\${SSH_TTY} \\[\\e[0;32m\\]+\${SHLVL}\\[\\e[1;30m\\]] \\[\\e[1;37m\\]\\w\\[\\e[0;37m\\] \\n(\$SHLVL:\\!)\\\$ " && eval $AA_P
Enhanced printenv Script for Server Debugging
A souped-up version of the Apache printenv script for hard-core server environment debuggery.
#!/bin/sh
echo -e "Content-type: text/plainnn"
...
__T "CURRENT PROCESS CMDLINE"
{
for p in `echo /proc/[0-9]*/cmdline`;
do
pid=${p:6:$((${#p}-13))}
[[ $pid == $PPID || $pid == $$ ]] && continue;
__M "[ /proc/$pid ]";
sed 's/x00/ /g;G' $p 2>/dev/null
done
}
fi
HOWTO: Uninstall CPANEL over SSH
The following is just a quick but detailed reference of some of the commands I used to successfully uninstall cpanel permanently. This is for advanced users of the shell. If you run a single one of these commands without fully understanding it, you will probably kill your server, lose everything on it permanently, not have a website or email for weeks.. So backup everything. FIRST. You should also contact your hosting provider support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for web-hosting companies, and you are less than a drop in their bucket.
Encrypted WordPress / phpBB Backups
Enter your DOMAIN_ROOT and the location of your wp-config.php or config.php, and this script finds all the mysql settings by parsing the phpbb or wordpress config file, then creates GPG encrypted backups, and saves your settings for future automation.
Optimizing Servers and Processes for Speed with ionice, nice, ulimit
To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.
SUDO – sudoers Config File
If you can crash your daemon, you likely have a security problem
Tons of awesome tips and tricks using netcat. Port redirector, nessus wrapper, capture exploits being sent by vuln scanners, etc. This is very useful for doing stuff like redirecting traffic through your firewall out to other places like web servers and mail hubs, while posing no risk to the firewall machine itself.
COMPUTER SECURITY TOOLBOX
List of mainly obscure security software geared more for the master pentester. These are mostly for unix, bsd, and mac and many are difficult to install and setup (require custom servers, inside access points, obscure libraries). Only programs that output data are included, so no actual exploits or anything. Most of these output extremely useful albeit extremely technical information.Upgrading to DreamHost Private Servers
DreamHost PS gives you your own "virtual machine", protecting your CPU and RAM on your physical machine for faster websites. Here's what I like and dislike about DreamHostPS, and some of the issues and solutions for migrating.
PortaPutty Auto-Reconnecting SSH Tunnels on an Encrypted TrueCrypt Portable USB Key w GPG
Ok I just came back up to write the intro.. I'm trying to keep it short to avoid getting bogged down by the coolness of each step. Here is what goes on. When I logon to my XP machine at work, I bring my usb key and plug it in first. On logging a window pops up first and it's a password prompt to mount my encrypted drive leonardo. It also checks a keyfile that is located on my usb key, but all I do now is type in my password. That causes my encrypted folder to be accessible to me like a normal drive, and it autoruns a startup batch file.
The batch file causes Portable versions of Firefox (all my bookmarks, my settings) to load, and launches Portable Mozilla Thunderbird (IMAP makes this work well), which is my favorite program (great GPG features and open-source!). Also Some Adobe CS4 software is loaded from the hard drive, like DreamWeaver. In the background, a service we created executes a PortaPuttY plink command to create forwarded tunnels from various remote servers and accounts, all using key-based encryption. These tunnels are automatically reconnected if they are disconnected, meaning you can use a socks 5 if you want or even better!
Securing php.ini and php.cgi with .htaccess
If you have a php.cgi or php.ini file in your /cgi-bin/ directory or other pub directory, try requesting them from your web browser. If your php.ini shows up or worse you are able to execute your php cgi, you'll need to secure it ASAP. This shows several ways to secure these files, and other interpreters like perl, fastCGI, bash, csh, etc.
About AskApache
Speed Up Google Analytics with urchin.js
Apache Variable Fun in htaccess
Server and Environment Variables are used by The Apache HTTP Server by provides a mechanism for storing information. This information can be used to control various operations such as logging or access control.
Elite Log File Scrolling with Color Syntax
Scrolls the latest log entries for multiple log files to the current screen or to any other monitor or TTY in color using syntax highlighting, making debugging easier and saving a lot of time for multi-monitor workstations.
30x Faster Cache and Site Speed with TMPFS
NOT a typo.. 30x is measurable, well-documented, and easily tested. This is what open-source is about. I haven’t had time to post much the past year, I'm always working! So I wanted to make up for that by publishing an article on a topic that would blow your mind and be something that you could actually start using and really get some benefit out of it. This is one of those articles that the majority of web hosting companies would love to see in paperback, so they could burn it.
301 Redirect Cheatsheet
Ultimate Redirect Cheatsheet for multiple programming languages. Redirecting Users with Javascript redirect, meta refresh redirect, and php redirect, also htaccess methods, python, coldfusion, asp, perl, etc.