Unless you are a total linux-freak-guru like myself, and even if you are, it can be enormously challenging and somewhat overwhelming to locate and eliminate MySQL bottlenecks. While many DBAs focus on improving the performance of the queries themselves, this post will focus on the highest-impact items: MySQL Server Performance and OS Performance for MySQL. Specifically showing 4 awesome tools to auto-tune your MySQL and a few more tools just for monitoring!
MySQL InnoDB linux MariaDB my.cnf MyISAM MySQL Optimization Percona performance phpMyAdmin
Example shell scripts (bash or sh) that automate and easyate tasks from within the dreamhost ssh env
Shell Scripting
The following is just a quick but detailed reference of some of the commands I used to successfully uninstall cpanel permanently. This is for advanced users of the shell. If you run a single one of these commands without fully understanding it, you will probably kill your server, lose everything on it permanently, not have a website or email for weeks.. So backup everything. FIRST. You should also contact your hosting provider support - but be prepared for some MAJOR negativity.. cpanel makes things very easy for web-hosting companies, and you are less than a drop in their bucket.
Hacking bash cpanel linux whm
If you examine the preferences dialog of any modern Web browser (like Internet Explorer, Safari or Mozilla), you'll probably notice a 'cache' setting. This lets you set aside a section of your computer's hard disk to store representations that you've seen, just for you. The browser cache works according to fairly simple rules. It will check to make sure that the representations are fresh, usually once a session (that is, the once in the current invocation of the browser).
Optimization
Using FastCGI on DreamHost and .htaccess
Shell Scripting
Server and Environment Variables are used by The Apache HTTP Server by provides a mechanism for storing information. This information can be used to control various operations such as logging or access control.
Htaccess
While researching a unix/linux tool awk I came upon one of the most thorough and helpful tutorials I've ever seen devoted to a particular topic. It's old-school just the way I like it. I contacted the author, Bruce Barnett because I just HAD to have this article for my readers, who are predominantly running solaris/unix/bsd/linux and he kindly gave permission.
Linux
- When php run as Apache Module (mod_php)
- When php run as CGI
- When cgi?d php is run with wrapper (for FastCGI)
PHP CGI Htaccess PHP php.ini
.htaccess is a very ancient configuration file for web servers, and is one of the most powerful configuration files most webmasters will ever come across. This htaccess guide shows off the very best of the best htaccess tricks and code snippets from hackers and server administrators.
You've come to the right place if you are looking to acquire mad skills for using .htaccess files!
Apache, Hosting, Htaccess, Optimization, Security, SEO, Web Development, WordPress .htpasswd 301 Redirect Apache Apache HTTP Server Cache Hosting Htaccess Htaccess Software HTTP Headers httpd.conf HyperText Transfer Protocol mod_rewrite Redirect RewriteCond RewriteRule SSL
If you have a php.cgi or php.ini file in your /cgi-bin/ directory or other pub directory, try requesting them from your web browser. If your php.ini shows up or worse you are able to execute your php cgi, you'll need to secure it ASAP. This shows several ways to secure these files, and other interpreters like perl, fastCGI, bash, csh, etc.
Htaccess Apache HTTP Server CGI Common Gateway Interface Htaccess HTTP 403 interpreter PHP php interpreter php.ini security SuEXEC
Tons of awesome tips and tricks using netcat. Port redirector, nessus wrapper, capture exploits being sent by vuln scanners, etc. This is very useful for doing stuff like redirecting traffic through your firewall out to other places like web servers and mail hubs, while posing no risk to the firewall machine itself.
Security
If for some reason you need to use windows .bat batch file scripting to do some task, or you just want to learn the most universal batch programming language on Windows machines, you lucked out and found the AskApache example. My expertise is the shell, in this article the shell is Windows cmd.exe. It has some pretty advanced windows shell usage, including pipes and redirection, but it's the modular linux-like coding approach that earns this script it's "advanced" title.
:SETPROMPT
set PROMPT=$_[%USERNAME%@%USERDOMAIN%]$S[$P]$_$M$G && EXIT /B
My favorite tool (and I've tried sooo many) for editing most Windows files and especially .bat files is the free and open-source Notepad++. Set that up and you will have a color-syntax-highlighted editor for Batch Scripting that works very very well.
Windows Batch File Batch Scripting Command Prompt Microsoft Windows
Unix file permissions are one of the more difficult subjects to grasp.. Well, ok maybe "grasp" isn't the word.. Master is the right word.. Unix file permissions is a hard topic to fully master, mainly I think because there aren't many instances when a computer user encounters them seriously, and bitwise is oldschool. This contains a listing of all possible permission masks and bits from a linux, php, and web hosting view.... cuz you guys AskApache Regs Rock!
Security chmod File Permissions umask
One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally. In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.
Javascript
List of mainly obscure security software geared more for the master pentester. These are mostly for unix, bsd, and mac and many are difficult to install and setup (require custom servers, inside access points, obscure libraries). Only programs that output data are included, so no actual exploits or anything. Most of these output extremely useful albeit extremely technical information. Security
This simple unix shell script automatically creates backups of a specific folder at regular hourly, nightly, weekly, and monthly intervals. Instead of the usual method for copying directory trees using tar with fifo, pipes, rsync, or NFS methods this script uses cpio which is much much faster and has cool options like saving m/a/c times, symlinks, relative paths, and weird file names.
Shell Scripting
Web Design
SetEnv, SetEnvIf, and SetEnvIfNoCase directives conditionally set environment variables accessible by scripts and apache based on HTTP Headers, Variables, and Request information.
Htaccess Environment variable FilesMatch Header Htaccess SetEnvIf SetEnvIfNoCase
Apache .htaccess Directives and Loaded Modules allowed on DreamHost Apache Server 2 Setups.
Hosting
Apache, Hosting 2.4 Apache Error
Use these standards best-practices to achieve more powerful links in terms of SEO, EM, STRONG, DFN, CODE, SAMP, KBD, VAR, CITE, ABBR, and ACRONYM SEO
Securing Subdirectories using unique apache htaccess solutions.
Htaccess
Apache Security tips and tricks for securing Apache Web Servers using htaccess, httpd.conf, and other built-in techniques to thwart attackers. This really should be required reading for any Apache admin or user because these little tricks are so easy to do.
Htaccess
These are a few of the mod_rewrite uses that BlogSecurity declared pushed the boundaries of Mod_Rewrite! Some of these snippets are quite exotic and unlike anything you may have seen before, also only for those who understand them as they can kill a website pretty quick.
Htaccess Htaccess mod_rewrite RewriteCond RewriteRule wordpress
To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.
Optimization CFQ Idle linux Linux scheduler Load Nice RAM Scheduling algorithms
AskApache Password Protect adds some serious password protection to your WordPress Blog. Not only does it protect your wp-admin directory, but also your wp-includes, wp-content, plugins, etc. plugins as well. Imagine a HUGE brick wall protecting your frail .php scripts from the endless attacks of automated web robots and password-guessing exploit-serving scripts.
WordPress .htpasswd Htaccess Password wordpress
DreamHost PS gives you your own "virtual machine", protecting your CPU and RAM on your physical machine for faster websites. Here's what I like and dislike about DreamHostPS, and some of the issues and solutions for migrating.
Hosting DreamHost
Hosting
Enumerating Permissions can be Annoying
Don’t ask me how because I won’t tell you, but on one of the hosts I was testing on that did not allow direct access I was able to get the Apache server running as dhapache to erroneously write a file into my users blog directory. This is a big security no-no and I now have my .htaccess file written into the blog directory where it should go, but instead of my php script’s user having write access to the file so I can modify it, its owned by dhapache! Because the file is owned by dhapache I shouldn’t even be allowed to know it exists, but there it is. So the next step was to try and take ownership of the .htaccess file so that I could modify it. I tried and tried but was unsuccessful, I couldn’t modify it so that was another dead end. Actually it took me awhile to figure out how to remove the file from my directory. Being that it was owned by dhapache I couldn’t delete or modify it using my php process or even through ftp/ssh! Sysadmins regularly run find commands that search the servers for any files owned by dhapache that should not be there as this is a big red flag that someone has found a way to manipulate dhapache which could potentially lead to modifying dhapache-owned server config files, which sometimes is all it takes to hack your website and server.. Luckily I was able to delete it by basically running the hack again to overwrite the file.
WordPress
#### No https except to wp-admin -
# If the request is empty ( implies fopen or normal file access by a php script )
RewriteCond %{THE_REQUEST} ^$ [OR]
# OR if the request if for wp-admin or wp-login.php
RewriteCond %{REQUEST_URI} ^/(wp-admin|wp-login.php).*$ [NC,OR]
# OR if the Referer is https
RewriteCond %{HTTP_REFERER} ^https://www.askapache.com/.*$ [NC]
# THEN skip the following rule, basically all this does is force https or badhost to be redirected
# BUT because of the above 3 rewritecond's, this won't break poorly written admin scripts
RewriteRule .* - [S=1]
RewriteCond %{HTTPS} =on [OR]
RewriteCond %{HTTP_HOST} !^www.askapache.com$ [NC]
RewriteRule .* https://www.askapache.com%{REQUEST_URI} [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9} /(wp-admin/.*|wp-login.php.*) HTTP/ [NC]
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L] Htaccess RewriteCond RewriteEngine RewriteRule
An image and technicacl achievement so profound, it will touch yoour heart.. So 3D... it'll hurt you eyes! LOL.. lol.. Man I am cracking up here. haha Ha definately the best intro ever, those really do look 3D for terminal though huh.. Sweet. Here is the little function I wrote to output that grey marble.
Linux ANSI escape code ANSI.SYS Control characters Escape sequence Stty Termcap Terminfo Tput
I have written more than a few articles detailing how to improve the speed of a website, and due to popular demand, this post will detail all the latest and greatest caching tricks and techniques that I utilize and love. The AskApache Best-Practices.
Optimization
Advanced Web Development by AskApache is a Firefox Collection I created since I'm always trying new Addons out and using multiple computers and I wanted a quick and easy way to install my favorite's and keep a running list. Firebug, YSlow, LastPass, and Web Developer are the only ones I always use regularly.
I like the idea of the last.fm but it's not as powerful as the site, which is awesome. Lately listening to Kings of Leon Radio...
Software Addons Firefox Mozilla
Looking for some advanced uses for the shell? Here is some of my best. The shell is where 70% of my work takes place, and I have at least one terminal open almost 100% of the time, for viewing tailing color-coded logs, and of course for the SSH Tunnels that I use to route various networking through, like my email. So I decided that to standardize and create a bash_profile containing the most time-saving and helpful functions that I could use on all the various hosting environments would really be some sweet sugar, so here is my constant Work-in-progress.
It works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.
I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:
#!/bin/bash
source ~/.bash_profile &>/dev/nulll
pm "PM is a function to output nice messages with color"
yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS"
yn "Show Calendar" && aa_calendar
yn "Show Fortune" && aa_fortune
Linux Arch Linux bash linux shell Unix shells
Linux firewall nmap
If you ever wanted to know the best way to defragment and speed up your Windows-Based PC, I mean you really wanted to know, here is the 2nd part to my article on Windows Speed Optimizing that details the process I have found works really well. Definately not a quick process, and certainly not the best ever, just my best ever and one that you only have to do once to get the benefits.
This article has a lot of incredibly useful (and FREE) tools I recommend, which you can grab and use without reading the article..
Windows Defragmentation Defragmentation software Disk Defragmenter ERUNT JkDefrag Microsoft MyDefrag PageDefrag Registry cleaner UltraDefrag Windows Registry Windows XP
Today I successfully learned how to compile and run multiple custom php installations for a DreamHost account, and to get it working I came upon a simple shell script that I made a couple changes to.
Shell Scripting
SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "google.com"
ErrorDocument 403 https://google.com
Some of the Ins and Outs of using SSL Connections with Apache.
Htaccess 301 Redirects Htaccess mod_rewrite SSL
Apache server performance can be improved by adding additional hardware resources such as RAM, faster CPU, etc. Optimization
While playing around with the Nmap Scripting Engine (NSE) we discovered an amazing number of open embedded devices on the Internet. Many of them are based on Linux and allow login to standard BusyBox with empty or default credentials. We used these devices to build a distributed port scanner to scan all IPv4 addresses. These scans include service probes for the most common ports, ICMP ping, reverse DNS and SYN scans. We analyzed some of the data to get an estimation of the IP address usage.
Hacking
