Looking for some advanced uses for the shell? Here is some of my best. The shell is where 70% of my work takes place, and I have at least one terminal open almost 100% of the time, for viewing tailing color-coded logs, and of course for the SSH Tunnels that I use to route various networking through, like my email. So I decided that to standardize and create a bash_profile containing the most time-saving and helpful functions that I could use on all the various hosting environments would really be some sweet sugar, so here is my constant Work-in-progress.
It works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.
I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:
#!/bin/bash
source ~/.bash_profile &>/dev/nulll
pm "PM is a function to output nice messages with color"
yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS"
yn "Show Calendar" && aa_calendar
yn "Show Fortune" && aa_fortune
Linux Arch Linux bash linux shell Unix shells
Example shell scripts (bash or sh) that automate and easyate tasks from within the dreamhost ssh env
Shell Scripting
While researching a unix/linux tool awk I came upon one of the most thorough and helpful tutorials I've ever seen devoted to a particular topic. It's old-school just the way I like it. I contacted the author, Bruce Barnett because I just HAD to have this article for my readers, who are predominantly running solaris/unix/bsd/linux and he kindly gave permission.
Linux
Ever wanted to execute commands on your server through php? Now you can. I'm calling this file (see below) shell.php and it allows you to run commands on your web server with the same permissions that your php executable has.
Hacking AJAX PHP
This simple unix shell script automatically creates backups of a specific folder at regular hourly, nightly, weekly, and monthly intervals. Instead of the usual method for copying directory trees using tar with fifo, pipes, rsync, or NFS methods this script uses cpio which is much much faster and has cool options like saving m/a/c times, symlinks, relative paths, and weird file names.
Shell Scripting
Today I successfully learned how to compile and run multiple custom php installations for a DreamHost account, and to get it working I came upon a simple shell script that I made a couple changes to.
Shell Scripting
As a security nut myself, and also a Linux admin, one of my biggest pet peeves is when I've taken the time and care to segment all the users on a server into separate home directories, and then some developer comes along, logs in as root, and changes the ownership of files. Other things can cause this, like Apache, PHP, Mutt, etc.. So I've always used a cron job that executes daily (and on demand) which automatically fixes all the permissions back to what they should be.
Linux bash chmod chown cron linux permissions shell
Unless you are a total linux-freak-guru like myself, and even if you are, it can be enormously challenging and somewhat overwhelming to locate and eliminate MySQL bottlenecks. While many DBAs focus on improving the performance of the queries themselves, this post will focus on the highest-impact items: MySQL Server Performance and OS Performance for MySQL. Specifically showing 4 awesome tools to auto-tune your MySQL and a few more tools just for monitoring!
MySQL InnoDB linux MariaDB my.cnf MyISAM MySQL Optimization Percona performance phpMyAdmin
The following is just a quick but detailed reference of some of the commands I used to successfully uninstall cpanel permanently. This is for advanced users of the shell. If you run a single one of these commands without fully understanding it, you will probably kill your server, lose everything on it permanently, not have a website or email for weeks.. So backup everything. FIRST. You should also contact your hosting provider support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for web-hosting companies, and you are less than a drop in their bucket.
Hacking bash cpanel linux whm
Hacking Cheat sheet linux
To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.
Optimization CFQ Idle linux Linux scheduler Load Nice RAM Scheduling algorithms
Using FastCGI on DreamHost and .htaccess
Shell Scripting
- When php run as Apache Module (mod_php)
- When php run as CGI
- When cgi?d php is run with wrapper (for FastCGI)
PHP CGI Htaccess PHP php.ini
If for some reason you need to use windows .bat batch file scripting to do some task, or you just want to learn the most universal batch programming language on Windows machines, you lucked out and found the AskApache example. My expertise is the shell, in this article the shell is Windows cmd.exe. It has some pretty advanced windows shell usage, including pipes and redirection, but it's the modular linux-like coding approach that earns this script it's "advanced" title.
:SETPROMPT
set PROMPT=$_[%USERNAME%@%USERDOMAIN%]$S[$P]$_$M$G && EXIT /B
My favorite tool (and I've tried sooo many) for editing most Windows files and especially .bat files is the free and open-source Notepad++. Set that up and you will have a color-syntax-highlighted editor for Batch Scripting that works very very well.
Windows Batch File Batch Scripting Command Prompt Microsoft Windows
Server and Environment Variables are used by The Apache HTTP Server by provides a mechanism for storing information. This information can be used to control various operations such as logging or access control.
Htaccess
One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally. In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.
Javascript
DreamHost PS gives you your own "virtual machine", protecting your CPU and RAM on your physical machine for faster websites. Here's what I like and dislike about DreamHostPS, and some of the issues and solutions for migrating.
Hosting DreamHost
.htaccess is a very ancient configuration file for web servers, and is one of the most powerful configuration files most webmasters will ever come across. This htaccess guide shows off the very best of the best htaccess tricks and code snippets from hackers and server administrators.
You've come to the right place if you are looking to acquire mad skills for using .htaccess files!
Apache, Hosting, Htaccess, Optimization, Security, SEO, Web Development, WordPress .htpasswd 301 Redirect Apache Apache HTTP Server Cache Hosting Htaccess Htaccess Software HTTP Headers httpd.conf HyperText Transfer Protocol mod_rewrite Redirect RewriteCond RewriteRule SSL
List of mainly obscure security software geared more for the master pentester. These are mostly for unix, bsd, and mac and many are difficult to install and setup (require custom servers, inside access points, obscure libraries). Only programs that output data are included, so no actual exploits or anything. Most of these output extremely useful albeit extremely technical information. Security
After many years of using any and all terminal emulators out there, from xterm to the Gnome terminal, to KDE Konsole to xfce4-terminal, lxterminal, vte, yakuake, rote, roxterm, sakura, terminator, and putty I finally I settled in for the long-haul with rxvt (rxvt-unicode).
My BOX: Slim -> Ratpoison -> URxvt -> Tmux -> Bash
Linux .Xdefaults .Xresources bash Gnome terminal GNU Screen ratpoison rxvt rxvt-unicode Terminal multiplexers tmux urxvt Xterm Yakuake
NOT a typo.. 30x is measurable, well-documented, and easily tested. This is what open-source is about. I haven’t had time to post much the past year, I'm always working! So I wanted to make up for that by publishing an article on a topic that would blow your mind and be something that you could actually start using and really get some benefit out of it. This is one of those articles that the majority of web hosting companies would love to see in paperback, so they could burn it.
Optimization tmpfs
Tons of awesome tips and tricks using netcat. Port redirector, nessus wrapper, capture exploits being sent by vuln scanners, etc. This is very useful for doing stuff like redirecting traffic through your firewall out to other places like web servers and mail hubs, while posing no risk to the firewall machine itself.
Security
If you have a php.cgi or php.ini file in your /cgi-bin/ directory or other pub directory, try requesting them from your web browser. If your php.ini shows up or worse you are able to execute your php cgi, you'll need to secure it ASAP. This shows several ways to secure these files, and other interpreters like perl, fastCGI, bash, csh, etc.
Htaccess Apache HTTP Server CGI Common Gateway Interface Htaccess HTTP 403 interpreter PHP php interpreter php.ini security SuEXEC
Unix file permissions are one of the more difficult subjects to grasp.. Well, ok maybe "grasp" isn't the word.. Master is the right word.. Unix file permissions is a hard topic to fully master, mainly I think because there aren't many instances when a computer user encounters them seriously, and bitwise is oldschool. This contains a listing of all possible permission masks and bits from a linux, php, and web hosting view.... cuz you guys AskApache Regs Rock!
Security chmod File Permissions umask
My improved version of the common printenv / test-cgi scripts for debugging Apache environment variables set in .htaccess files
Shell Scripting
Apache .htaccess Directives and Loaded Modules allowed on DreamHost Apache Server 2 Setups.
Hosting
Web Design
SetEnv, SetEnvIf, and SetEnvIfNoCase directives conditionally set environment variables accessible by scripts and apache based on HTTP Headers, Variables, and Request information.
Htaccess Environment variable FilesMatch Header Htaccess SetEnvIf SetEnvIfNoCase
While playing around with the Nmap Scripting Engine (NSE) we discovered an amazing number of open embedded devices on the Internet. Many of them are based on Linux and allow login to standard BusyBox with empty or default credentials. We used these devices to build a distributed port scanner to scan all IPv4 addresses. These scans include service probes for the most common ports, ICMP ping, reverse DNS and SYN scans. We analyzed some of the data to get an estimation of the IP address usage.
Hacking
Linux firewall nmap
Host Google Analytics ga.js file locally for increased speed! Makes web pages load faster.
Google
An image and technicacl achievement so profound, it will touch yoour heart.. So 3D... it'll hurt you eyes! LOL.. lol.. Man I am cracking up here. haha Ha definately the best intro ever, those really do look 3D for terminal though huh.. Sweet. Here is the little function I wrote to output that grey marble.
Linux ANSI escape code ANSI.SYS Control characters Escape sequence Stty Termcap Terminfo Tput
#### No https except to wp-admin -
# If the request is empty ( implies fopen or normal file access by a php script )
RewriteCond %{THE_REQUEST} ^$ [OR]
# OR if the request if for wp-admin or wp-login.php
RewriteCond %{REQUEST_URI} ^/(wp-admin|wp-login.php).*$ [NC,OR]
# OR if the Referer is https
RewriteCond %{HTTP_REFERER} ^https://www.askapache.com/.*$ [NC]
# THEN skip the following rule, basically all this does is force https or badhost to be redirected
# BUT because of the above 3 rewritecond's, this won't break poorly written admin scripts
RewriteRule .* - [S=1]
RewriteCond %{HTTPS} =on [OR]
RewriteCond %{HTTP_HOST} !^www.askapache.com$ [NC]
RewriteRule .* https://www.askapache.com%{REQUEST_URI} [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9} /(wp-admin/.*|wp-login.php.*) HTTP/ [NC]
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L] Htaccess RewriteCond RewriteEngine RewriteRule
If you ever wanted to know the best way to defragment and speed up your Windows-Based PC, I mean you really wanted to know, here is the 2nd part to my article on Windows Speed Optimizing that details the process I have found works really well. Definately not a quick process, and certainly not the best ever, just my best ever and one that you only have to do once to get the benefits.
This article has a lot of incredibly useful (and FREE) tools I recommend, which you can grab and use without reading the article..
Windows Defragmentation Defragmentation software Disk Defragmenter ERUNT JkDefrag Microsoft MyDefrag PageDefrag Registry cleaner UltraDefrag Windows Registry Windows XP
Links to htaccess tutorials and howtos in the htaccess forum
Htaccess
So my blog as been rather quiet for almost a year now, and very few updates if any have been released for my Password Protection PLugin, my Google 404 Plugin, and definately not for my AskApache CrazyCache plugin, which I will be releasing last... So for all of you who've helped me out by sending me suggestions and notifying me of errors and sticking with it... Just wanted to say sorry about that, and thanks for all the great ideas.. Well, I've been sticking with it as well believe it our not. I manage to get free days once in a while, and then its time to jam.
WordPress Apache HTTP Server CGI Debugging HTTP protocol Htaccess SuexecUserGroup
