FREE THOUGHT · FREE SOFTWARE · FREE WORLD

Creating an Offline Version of Page

I can do anythingSo, here's what I hacked together last night, that is being used today. It's essentially 2 files.

  1. A php file that scrapes uses curl to scrape all the urls for the page (favicon, css, images, pdfs, etc..)
  2. A simple bash shell script acting as a cgi that creates a zip file of all the urls, and a self-extracting exe file for those without a winzip tool

PHP

Optimizing Servers and Processes for Speed with ionice, nice, ulimit

To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.

Optimization

Rigging the DreamHost Site of the Month Contest

Since I was disqualified and accused of cheating, which is absolutely false, I decided to go ahead and do the crime I have been punished for. That's right, I'm going to show YOU how I could have rigged the DHSOTM contest to hopefully make the statement that I absolutely did not cheat and I absolutely want to be re-instated as the winner.

Hosting

Mirroring an Entire Site using Rsync over SSH

Sometimes there is an urgent need for creating an exact duplicate or "mirror" of a web site on a separate server. This could be needed for creating Round Robin Setups, Load-Balancing, Failovers, or for just plain vanilla backups. In the past I have used a lot of different methods to copy data from one server to another, including creating an archive of the whole directory and then using scp to send the file over, creating an archive and then encrypting it and then sending that file over using ftp, curl, etc., and my persistence at learning new ways to do things has paid off because now I use rsync to keep an exact replica of the entire directory on an external server, without having to use all the CPU and resources of other mirroring methods.

Security

DreamHost Server IP Address List

A list of all DreamHost Servers and corresponding IP address. The dreamhost.com DNS zone is currently having troubles, so most hostnames within this zone are not resolving.

Hosting

Actual Htaccess Files from My Server

#### No https except to wp-admin -
# If the request is empty ( implies fopen or normal file access by a php script )
RewriteCond %{THE_REQUEST} ^$ [OR]

# OR if the request if for wp-admin or wp-login.php
RewriteCond %{REQUEST_URI} ^/(wp-admin|wp-login.php).*$ [NC,OR]

# OR if the Referer is https
RewriteCond %{HTTP_REFERER} ^https://www.askapache.com/.*$ [NC]

# THEN skip the following rule, basically all this does is force https or badhost to be redirected
# BUT because of the above 3 rewritecond's, this won't break poorly written admin scripts
RewriteRule .* - [S=1]

RewriteCond %{HTTPS} =on [OR]
RewriteCond %{HTTP_HOST} !^www.askapache.com$ [NC]
RewriteRule .* https://www.askapache.com%{REQUEST_URI} [R=301,L]

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9} /(wp-admin/.*|wp-login.php.*) HTTP/ [NC]
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]

Htaccess

Custom bash_profile for Advanced Shell Users

Looking for some advanced uses for the shell? Here is some of my best. The shell is where 70% of my work takes place, and I have at least one terminal open almost 100% of the time, for viewing tailing color-coded logs, and of course for the SSH Tunnels that I use to route various networking through, like my email. So I decided that to standardize and create a bash_profile containing the most time-saving and helpful functions that I could use on all the various hosting environments would really be some sweet sugar, so here is my constant Work-in-progress.

It works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.

I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:

#!/bin/bash

source ~/.bash_profile &>/dev/nulll

pm "PM is a function to output nice messages with color"
yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS"
yn "Show Calendar" && aa_calendar
yn "Show Fortune" && aa_fortune

Linux

MariaDB with LZ4 Compression Howto for CentOS

I really really wanted to run the latest MariaDB with LZ4 Page Compression.. it is a game changer for many types of large databases I deal with. So I compiled it manually in a way that is repeatable and follows best-practices and it's now powering this site. Now I can use InnoDB Page Compression with lzo, lzma, bzip2, snappy, or my favorite algorithm: LZ4.

Thought this would be a good chance to post a howto, and show that there is a lot you can do by compiling software yourself and breaking the package management one-click install shackles.

Linux, MySQL

Faster Form Submission and Processing with fsockopen

Speedy Forms using background-requestsPart II: Example illustrating how to speed up GET/POST form submissions. Uses fsockopen to initiate a server-side background request to process the submitted data, so that the result page of the form is displayed to the client lightningly quick.

PHP

SEO Redirects without mod_rewrite

Web Professionals use mod_rewrite to issue 301 and 302 Redirects for Search Engines. Sometimes you may not have mod_rewrite.c or you want an alternative redirect method. Using mod_alias RedirectMatch you can use REGEX in Redirect commands!

Htaccess

Wget Trick to Download from Restrictive Sites

Before
wget 403 Forbidden After trick
wget bypassing restrictionsI am often logged in to my servers via SSH, and I need to download a file like a WordPress plugin. I've noticed many sites now employ a means of blocking robots like wget from accessing their files. Most of the time they use .htaccess to do this. So a permanent workaround has wget mimick a normal browser.

Linux

Password Protection Plugin Status

Enumerating Permissions can be Annoying

Don’t ask me how because I won’t tell you, but on one of the hosts I was testing on that did not allow direct access I was able to get the Apache server running as dhapache to erroneously write a file into my users blog directory. This is a big security no-no and I now have my .htaccess file written into the blog directory where it should go, but instead of my php script’s user having write access to the file so I can modify it, its owned by dhapache! Because the file is owned by dhapache I shouldn’t even be allowed to know it exists, but there it is. So the next step was to try and take ownership of the .htaccess file so that I could modify it. I tried and tried but was unsuccessful, I couldn’t modify it so that was another dead end. Actually it took me awhile to figure out how to remove the file from my directory. Being that it was owned by dhapache I couldn’t delete or modify it using my php process or even through ftp/ssh! Sysadmins regularly run find commands that search the servers for any files owned by dhapache that should not be there as this is a big red flag that someone has found a way to manipulate dhapache which could potentially lead to modifying dhapache-owned server config files, which sometimes is all it takes to hack your website and server.. Luckily I was able to delete it by basically running the hack again to overwrite the file.

WordPress