Shell Scripting

Shell Scripting

A shell script is a script written for the shell, or command line interpreter, of an operating system. It is often considered a simple domain-specific programming language. Typical operations performed by shell scripts include file manipulation, program execution, and printing text. Usually, shell script refers to scripts written for a Unix shell, while COMMAND.COM (DOS) and cmd.exe (Windows) command line scripts are usually called batch files, but here properties of both are discussed.

Many shell script interpreters double as command line interface, such as the various Unix shells, Windows PowerShell or the MS-DOS COMMAND.COM. Others, such as AppleScript, add scripting capability to computing environments without requiring a command line interface. Other examples of programming languages primarily intended for shell scripting include DCL and JCL.

---

Optimizing Servers and Processes for Speed with ionice, nice, ulimit

To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.

· CPU  ·  RSS | 1:41 AM

---

Advanced WordPress wp-config.php Tweaks

The bottom line for this article is that I want to make WordPress as fast, secure, and easy to install, run, and manage because I am using it more and more for client production sites, I will work for days in order to solve an issue so that I never have to spend time on that issue again. Time is money in this industry and that is ultimately (time) what there is to gain by tweaking WordPress.

Note: I spent no time on readability, this is primarily a read the code and figure it out article.. This is for advanced users looking for a reference or discussion and for those of you looking to advance. Feedback would be great if you make it that far..

· Tweaks  ·  RSS | 3:23 AM

---

Custom bash_profile for Advanced Shell Users

Want to get started with my custom bash file without reading this page (recommended as page is not updated)? Execute the following in your shell and rock and roll.

curl -O http://z.askapache.com/askapache-bash-profile.txt && . askapache-bash-profile.txt

· unix  ·  RSS | 2:32 PM

---

PHP and AJAX shell console

Ever wanted to execute commands on your server through php? Now you can. I’m calling this file (see below) shell.php and it allows you to run commands on your web server with the same permissions that your php executable has.

· tools  ·  RSS | 9:01 PM

---

Mirroring an Entire Site using Rsync over SSH

Sometimes there is an urgent need for creating an exact duplicate or “mirror” of a web site on a separate server. This could be needed for creating Round Robin Setups, Load-Balancing, Failovers, or for just plain vanilla backups. In the past I have used a lot of different methods to copy data from one server to another, including creating an archive of the whole directory and then using scp to send the file over, creating an archive and then encrypting it and then sending that file over using ftp, curl, etc., and my persistence at learning new ways to do things has paid off because now I use rsync to keep an exact replica of the entire directory on an external server, without having to use all the CPU and resources of other mirroring methods.

· mirror  ·  RSS | 7:40 AM

---

Chmod, Umask, Stat, Fileperms, and File Permissions

Unix file permissions are one of the more difficult subjects to grasp.. Well, ok maybe “grasp” isn’t the word.. Master is the right word.. Unix file permissions is a hard topic to fully master, mainly I think because there aren’t many instances when a computer user encounters them. I’ve done a lot of research on it the past couple weeks… and now here’s everything I’ve learned so far.. cuz you guys AskApache Regs Rock!

· Permissions  ·  RSS | 6:16 AM

---

Optimize a Website for Speed, Security, and Easy Management

Learn how to setup, configure, secure, optimize, and create a low-maintenance website the AskApache way. I’m piecing together all the hacks, tricks, methods, and ideas discussed throughout this blog and all across Netdom and glueing them all together to show you how to have the most optimized, crazy fastest, and best website setup I can think of.

· compression  ·  RSS | 8:45 PM

---

Serve External Javascript Files locally for Increased Speed

One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally. In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.

· compression  ·  RSS | 2:55 AM

---

Pimp out your FeedBurner Count

I’ve had a lot of people ask about the FeedBurner FeedCount image on AskApache. Specifically how to set it up with custom messages and different colors each page view… It is pretty sweet..

· FeedBurner  ·  RSS | 4:50 AM

---

Elite Log File Scrolling with Color Syntax

Scrolls the latest log entries for multiple log files to the current screen or to any other monitor or TTY in color using syntax highlighting, making debugging easier and saving a lot of time for multi-monitor workstations.

· error  ·  RSS | 12:56 AM

---

Encrypted WordPress / phpBB Backups

Enter your DOMAIN_ROOT and the location of your wp-config.php or config.php, and this script finds all the mysql settings by parsing the phpbb or wordpress config file, then creates GPG encrypted backups, and saves your settings for future automation.

· bash  ·  RSS | 12:28 PM

---

Automated Folder Backup Shell-Script

This simple unix shell script automatically creates backups of a specific folder at regular hourly, nightly, weekly, and monthly intervals. Instead of the usual method for copying directory trees using tar with fifo, pipes, rsync, or NFS methods this script uses cpio which is much much faster and has cool options like saving m/a/c times, symlinks, relative paths, and weird file names.

· snapshot  ·  RSS | 1:22 AM

---

Search And Replace shell script helpful for Upgrades

Search all files in a directory, replacing all occurances of string with a replacement string.

· search and replace  ·  RSS | 1:09 AM

---

IP Abuse Detection for DreamHost

Scan Apache logs for IP address that are probably evil, then generates an .htaccess file to DENY them all.

RSS | 9:46 AM

---

Apache Environment Variables CGI Script

My improved version of the common printenv / test-cgi scripts for debugging Apache environment variables set in .htaccess files

RSS | 11:40 AM

---

PHP5 Custom Install Shell Script Example

Today I successfully learned how to compile and run multiple custom php installations for a DreamHost account, and to get it working I came upon a simple shell script that I made a couple changes to.

RSS | 11:11 PM

---

301 Redirect Cheatsheet

Ultimate Redirect Cheatsheet for multiple programming languages. Redirecting Users with Javascript redirect, meta refresh redirect, and php redirect, also htaccess methods, python, coldfusion, asp, perl, etc.

· headers  ·  RSS | 9:48 PM

---

Story by Author of PING for UNIX

Yes, it’s true! I’m the author of ping for UNIX. Ping is a little thousand-line hack that I wrote in an evening which practically everyone seems to know about. :-)

· Muuss  ·  RSS | 1:23 AM

---

Speed Up Google Analytics with urchin.js

Update: This article is a bit dated, to get the newest and latest Google Analytics Speed Tips, check out the 3 part update about Hosting, Cookie Domains, and JavaScript Loading and Failproofs

Ever notice that sometimes your sites take a while to load all the way because google’s urchin.js file is taking forever? You may recognize this problem when you see something similar to this in your browsers status bar “Transferring data from google-analytics.com...”

Time To Setup?
4 minutes I got tired of seeing that all the time and so I set up an automated cronjob that runs every 12 hours and downloads the latest version from google, and saves it to my website directory, then I reference /urchin.js instead of http://www.google-analytics.com/urchin.js.. and my site loads a lot faster!
Take a look at the source for this page if you want to see what is going on (look at the bottom)
There are 2 pretty major things that you accomplish by hosting urchin.js locally

You Enable persistant connections
You ensure that the correct 304 Not Modified header is sent back to your site visitors instead of reserving the entire file.

The urch.sh shell script
Create a shell script called urch.sh.. this will be executed every 1/day or 1/wk, whatever you want. The following script removes the local urchin.js, then it wgets the latests version into the local directory.

#!/bin/sh
rm /home/user/websites/askapache.com/z/j/urchin.js
cd /home/user/websites/askapache.com/z/j/
wget http://www.google-analytics.com/urchin.js
chmod 644 /home/user/websites/askapache.com/z/j/urchin.js
cd ${OLDPWD}
exit 0;

Improved shell script
I realized right away that a more modular shell script would be needed.. I admin like 50+ web-sites and it would be stupid to have to type the same block of code 50 times, wget the same file 50 times, etc.. So this version downloads the urchin.js file into a temporary directory, then it copies it OVER the old file for each directory. So 1 GET request …

RSS | 11:10 PM

---

Wget Trick to Download from Restrictive Sites

Before

After trick
I am often logged in to my servers via SSH, and I need to download a file like a WordPress plugin. I’ve noticed many sites now employ a means of blocking robots like wget from accessing their files. Most of the time they use .htaccess to do this. So a permanent workaround has wget mimick a normal browser.

RSS | 11:26 AM

---

Custom Webmaster Shell Environment

Webmasters often use SSH to remotely admin web sites and servers, learn how to make your shell colorful and powerful using my sample .bash_profile and you can be up and running in 30 seconds!

· ssh  ·  RSS | 2:17 PM

---

Install multiple OS Without Cds

I had a CD-RW drive but being a struggling computer security researcher I had no money for blank cd-recordables. What follows is how I managed to install various operating systems on my computer (1 hard drive) without having to burn to a CD the ISO and then boot from that.

· multi-boot  ·  RSS | 1:06 AM

---

On-Demand MySQL Backup Shell Script

On-Demand WordPress MySQL database Backup Shell Script for DreamHost

· shell-script  ·  RSS | 3:06 PM

---

Custom PHP.ini with FastCGI on DreamHost

I’m really sorry but I had to disable the sites you had running under
user $USER because that was the only way I could keep the associated
processes from restarting themselves and making the server unstable.

Please do not run these sites with a persistent php.cgi process with a
php.ini file as an argument.

RSS | 10:40 AM

---

Shell Scripts for Dreamhost

Example shell scripts (bash or sh) that automate and easyate tasks from within the dreamhost ssh env

· shellscript  ·  RSS | 11:46 AM

---

---