To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.
Optimization CFQ Idle linux Linux scheduler Load Nice RAM Scheduling algorithms
Hosting DreamHost Out of memory RAM virtual memory
This is a big update from the last time I looked into this, when I enumerated 57 Status Codes that Apache 2.x was capable of handling. This list contains 83 Status Codes recognized by Apache. I compiled the latest 2.4.4 Apache in order to view the actual codes sent by a live server.. very cool. You can read about the newest HTTP Status Codes in RFC 6585.
Net Apache cURL ErrorDocument Htaccess HTTP HTTP Headers Redirect Status Code
Its really nice to be able to print out a webpage you are reading using your browsers built-in print feature. Using CSS you can easily transform your site into a print-friendly site.
Today I received an email from a visitor to my site requesting that I add a way to print site articles on AskApache
CSS Cascading Style Sheets nicer printing Style sheet
Oh ya lets get it on! short but sweet
Hacking bash Screen ssh War Dialing
Advanced Web Development by AskApache is a Firefox Collection I created since I'm always trying new Addons out and using multiple computers and I wanted a quick and easy way to install my favorite's and keep a running list. Firebug, YSlow, LastPass, and Web Developer are the only ones I always use regularly.
I like the idea of the last.fm but it's not as powerful as the site, which is awesome. Lately listening to Kings of Leon Radio...
Software Addons Firefox Mozilla
The following is just a quick but detailed reference of some of the commands I used to successfully uninstall cpanel permanently. This is for advanced users of the shell. If you run a single one of these commands without fully understanding it, you will probably kill your server, lose everything on it permanently, not have a website or email for weeks.. So backup everything. FIRST. You should also contact your hosting provider support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for web-hosting companies, and you are less than a drop in their bucket.
Hacking bash cpanel linux whm
Ok I just came back up to write the intro.. I'm trying to keep it short to avoid getting bogged down by the coolness of each step. Here is what goes on. When I logon to my XP machine at work, I bring my usb key and plug it in first. On logging a window pops up first and it's a password prompt to mount my encrypted drive leonardo. It also checks a keyfile that is located on my usb key, but all I do now is type in my password. That causes my encrypted folder to be accessible to me like a normal drive, and it autoruns a startup batch file.
The batch file causes Portable versions of Firefox (all my bookmarks, my settings) to load, and launches Portable Mozilla Thunderbird (IMAP makes this work well), which is my favorite program (great GPG features and open-source!). Also Some Adobe CS4 software is loaded from the hard drive, like DreamWeaver. In the background, a service we created executes a PortaPuttY plink command to create forwarded tunnels from various remote servers and accounts, all using key-based encryption. These tunnels are automatically reconnected if they are disconnected, meaning you can use a socks 5 if you want or even better!
Hacking Backup encryption Microsoft Windows Secure Shell ssh TrueCrypt USB USB flash drive windows builtin command-line tools
Looking for some advanced uses for the shell? Here is some of my best. The shell is where 70% of my work takes place, and I have at least one terminal open almost 100% of the time, for viewing tailing color-coded logs, and of course for the SSH Tunnels that I use to route various networking through, like my email. So I decided that to standardize and create a bash_profile containing the most time-saving and helpful functions that I could use on all the various hosting environments would really be some sweet sugar, so here is my constant Work-in-progress.
It works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.
I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:
#!/bin/bash
source ~/.bash_profile &>/dev/nulll
pm "PM is a function to output nice messages with color"
yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS"
yn "Show Calendar" && aa_calendar
yn "Show Fortune" && aa_fortune
Linux Arch Linux bash linux shell Unix shells
NOT a typo.. 30x is measurable, well-documented, and easily tested. This is what open-source is about. I haven’t had time to post much the past year, I'm always working! So I wanted to make up for that by publishing an article on a topic that would blow your mind and be something that you could actually start using and really get some benefit out of it. This is one of those articles that the majority of web hosting companies would love to see in paperback, so they could burn it.
Optimization tmpfs
Want to block a bad robot or web scraper using .htaccess files? Here are 2 methods that illustrate blocking 436 various user-agents.
Htaccess
Enter your DOMAIN_ROOT and the location of your wp-config.php or config.php, and this script finds all the mysql settings by parsing the phpbb or wordpress config file, then creates GPG encrypted backups, and saves your settings for future automation.
Shell Scripting
Search all files in a directory, replacing all occurances of string with a replacement string.
Shell Scripting
DreamHost PS gives you your own "virtual machine", protecting your CPU and RAM on your physical machine for faster websites. Here's what I like and dislike about DreamHostPS, and some of the issues and solutions for migrating.
Hosting DreamHost
You've probably seen this.. one of THE coolest video games I've ever seen (though I haven't played).. Its tempting me back to MLG.. Games are games today, but in the future Gaming Skill will be highly sought after.. So I check in with the gaming world occasionally, I'd hate to miss something like a new Duke-Nukem (cancelled after 10 years in development and no releases).
I have around 10 really nice articles I've been working on for awhile now, my problem is that I am a perfectionist, so I keep finding improvements.. worth it.
Web Design Game YouTube
So my blog as been rather quiet for almost a year now, and very few updates if any have been released for my Password Protection PLugin, my Google 404 Plugin, and definately not for my AskApache CrazyCache plugin, which I will be releasing last... So for all of you who've helped me out by sending me suggestions and notifying me of errors and sticking with it... Just wanted to say sorry about that, and thanks for all the great ideas.. Well, I've been sticking with it as well believe it our not. I manage to get free days once in a while, and then its time to jam.
WordPress Apache HTTP Server CGI Debugging HTTP protocol Htaccess SuexecUserGroup
Scan Apache logs for IP address that are probably evil, then generates an .htaccess file to DENY them all.
Security
As an example, here is the boot line that I am using at the moment on an older Dell Desktop, just to illustrate module parameters and environment vars.
title Arch Linux X-256
linux /vmlinuz-linux root=UUID rw rootfstype=ext4 elevator=noop selinux=0 plymouth.enable=0 raid=noautodetect ipv6.disable=1 video=DP1:d rcutree.rcu_idle_gp_delay=1 i915.panel_ignore_lid=-1 i915.modeset=1 TERM=rxvt-256color
Linux Arch Linux Boot GRUB Kernel
FYI, using the Mod_Rewrite Variables Cheatsheet makes this example, and all advanced .htaccess code easier to understand. This demo lets you set a cookie with 1 of 3 values, then you just request the pdf file with a normal link click and get 1 of 3 different responses. This is accomplished with a nice bit of .htaccess code.
Htaccess Htaccess HTTP cookie JavaScript PDF pdf.js Portable Document Format
One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally. In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.
Javascript
List of mainly obscure security software geared more for the master pentester. These are mostly for unix, bsd, and mac and many are difficult to install and setup (require custom servers, inside access points, obscure libraries). Only programs that output data are included, so no actual exploits or anything. Most of these output extremely useful albeit extremely technical information. Security
A list of all DreamHost Servers and corresponding IP address. The dreamhost.com DNS zone is currently having troubles, so most hostnames within this zone are not resolving. Hosting
SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "google.com"
ErrorDocument 403 https://google.com
Some of the Ins and Outs of using SSL Connections with Apache.
Htaccess 301 Redirects Htaccess mod_rewrite SSL
I looked at a lot of different ways to display quotes and pullquotes and even though the javascript solutions are very nice, esp. the 456bereastreet.com solution, I decided to just use CSS (Keep It Simple Stupid).
CSS
Well, not this sites server, which has never been hacked, but a server hosted with hostgator.
Interestingly, the hacker sent me an email and left me an mp3, which I've uploaded for your listening leisure.
Do real hackers exist? Yes. Can we coexist? Preferably!
Hacking hack hacked p0wned
The Admin Bar in WordPress is always shown when in the backend administration area no matter what.
You are given the option to show/hide the admin bar when viewing the site, but there is no option to show/hide the admin bar when using the backend... So I created a plugin to provide that option on profile pages.
Why hide the admin bar in backend?
Normally I like the admin-bar and usually have it enabled, but it's nice to be able to switch it on/off from the "Edit Profile" page. The biggest reason for not loading/showing the admin_bar in the backend is SPEED. Here are 2 instances where I hide it-
· Sites with many authors/admins
· Moderating Comments on a Mobile Phone
WordPress PHP WordPress Plugin
Very nice tutorial dealing with the robots.txt file. Shows examples for google and other search engines. Wordpress robots.txt and phpBB robots.txt sample files. SEO
A hit-list of some of my favorite mod_rewrite code snippets for .htaccess files
Htaccess
Check out ReadWriteWeb's fantastic list of uniquely useful search engines, its a lot of fun!
Google
Hacking Cheat sheet linux
Htaccess Apache HTTP Server Htaccess Redirect rewrite RewriteBase RewriteCond RewriteEngine RewriteLock RewriteLog RewriteMap RewriteOptions RewriteRule
While reading up on gethostbyaddr on PHP.net, I saw a nice idea for using fsockopen to connect over UDP port 53 to any Public DNS server, like Google DNS 8.8.8.8, and sending the reverse addr lookup in oh about 100 bytes, then getting the response in oh about 150 bytes! All in less than a second. This is how/why to read/write data directly to the wire! This would be extremely valuable for use in things like my online header tool because it's faster than any other method. As usual, I went a bit overboard optimizing it to be lean and fast.
PHP DNS fsockopen gethostbyaddr Google networking OpenDNS PHP socket udp UltraDNS
I was over at Alex King's blog today checking out his mobile web plugin for wordpress and I noticed some fresh podcasts from some of the industry's finest. The feeds are .mp3 and are easy to find and browse on the search-this.com site. They have some pretty nice articles over there as well..
Web Design
PoshCSS.com has links to nice and new CSS Articles. Don't worry, they aren't snobs. ;)
CSS
This is a brief explanation of what is needed for an awesome theme, well awesome from my perspective. This is good stuff, memorize it.
Web Development, WordPress css web builder
This simple unix shell script automatically creates backups of a specific folder at regular hourly, nightly, weekly, and monthly intervals. Instead of the usual method for copying directory trees using tar with fifo, pipes, rsync, or NFS methods this script uses cpio which is much much faster and has cool options like saving m/a/c times, symlinks, relative paths, and weird file names.
Shell Scripting
While playing around with the Nmap Scripting Engine (NSE) we discovered an amazing number of open embedded devices on the Internet. Many of them are based on Linux and allow login to standard BusyBox with empty or default credentials. We used these devices to build a distributed port scanner to scan all IPv4 addresses. These scans include service probes for the most common ports, ICMP ping, reverse DNS and SYN scans. We analyzed some of the data to get an estimation of the IP address usage.
Hacking
While researching a unix/linux tool awk I came upon one of the most thorough and helpful tutorials I've ever seen devoted to a particular topic. It's old-school just the way I like it. I contacted the author, Bruce Barnett because I just HAD to have this article for my readers, who are predominantly running solaris/unix/bsd/linux and he kindly gave permission.
Linux
