Saturday, October 10th, 2009
To prepare for several upcoming articles on AskApache that are focused on optimizing Servers and Sites from a server admin level, here is an article to introduce the main tools that we will be using. These tools are used to optimize CPU time for each process using nice and renice, and other tools like ionice are used to optimize the Disk IO, or Disk speed / Disk traffic for each process. Then you can make sure your mysqld and httpd processes are always fast and prioritized.
Tags: 503, Advanced, Apache, askapache, Backups, Bandwidth, bash, Blocking, Boot, Bottleneck, caching, compression, CPU, CSS, curl, Disk IO, DNS, Examples, feed, fifo, GET, httpd, ionice, iostat, Javascript, Linux, mysql, Nice, Optimization, pagefile, Performance, Perl, PHP, Port, ram, Renice, Round Robin, rsync, Scripts, Security, server, servers, Shell, shell script, Shell Scripting, Socket, SPEED, SSH, SSI, stat, taskset, trick, Ulimit, Web Hosting, WordPress, wp-config.php
Posted in Apache, Cache, Featured, Linux Unix BSD, Review, Security, Server Administration, Shell Scripting, Web Hosting, Webmaster | 3 Comments »
Saturday, January 10th, 2009
Skip this – still under edit
I discovered these tips and tricks mostly while working as a network security penetration specialist hired to find security holes in web hosting environments. Shared hosting is the most common and cheapest form of web-hosting where multiple customers are placed on a single machine and “share” the resources (CPU/RAM/SPACE). The machines are configured to basically ONLY do HTTP and FTP. No shells or any interactive logins, no ssh, just FTP access. That is when I started examining htaccess files in great detail and learned about the incredible untapped power of htaccess. For 99% of the worlds best Apache admins, they don’t use .htaccess much, if AT ALL. It’s much easier, safer, and faster to configure Apache using the httpd.conf file instead. However, this file is almost never readable on shared-hosts, and I’ve never seen it writable. So the only avenue left for those on shared-hosting was and is the .htaccess file, and holy freaking fiber-optics.. it’s almost as powerful as httpd.conf itself!
Most all .htaccess code works in the httpd.conf file, but not all httpd.conf code works in .htaccess files, around 50%. So all the best Apache admins and programmers never used .htaccess files. There was no incentive for those with access to httpd.conf to use htaccess, and the gap grew. It’s common to see “computer gurus” on forums and mailing lists rail against all uses and users of .htaccess files, smugly announcing the well known problems with .htaccess files compared with httpd.conf – I wonder if these “gurus” know the history of the htaccess file, like it’s use in the earliest versions of the HTTP Server- NCSA’s HTTPd, which BTW, became known as Apache HTTP. So you could easily say that htaccess files predates Apache itself.
Once I discovered what .htaccess files could do towards helping me enumerate and exploit security vulnerabilities even on big shared-hosts I focused all my research into .htaccess files, meaning I was reading the venerable Apache HTTP Source code 24/7! I compiled every released version of the Apache Web Server, ever, even NCSA’s, and focused on enumerating the most powerful htaccess directives. Good times! Because my focus was on protocol/file/network vulnerabilites instead of web dev I built up a nice toolbox of htaccess tricks to do unusual things. When I switched over to webdev in 2005 I started using htaccess for websites, not research. I documented most of my favorites and rewrote the htaccess guide for webdevelopers. After some great encouragement on various forums and nets I decided to start a blog to share my work with everyone, AskApache.com was registered, I published my guide, and it was quickly plagiarized and scraped all over the net. Information is freedom, and freedom is information, so this blog has the least restrictive copyright for you. Feel free to modify, copy, republish, sell, or use anything on this site ;)
Tags: .htaccess examples, 301 Redirect, 302 Redirect, 401, 403 Forbidden, 404 Not Found, 500, 503, admin, Advanced, Apache, Apache Htaccess, apache ssl, askapache, ASP, authorization, Backups, Bandwidth, bash, Blocking, Boot, Cache, Cache-Control, caching, cheatsheet, chmod, code snippets, compression, Cookies, CSS, debugging, DreamHost, Email, error log, errordocument, Etags, Examples, experiments, feed, FeedBurner, File System, FilesMatch, filesystem, Firefox, Flash, Forms, GET, Google, Hacking, hotlinking, HowTo, Htaccess, htaccess files, htaccess guide, htaccess rewrite, htaccess tricks, htaccess tutorial, Htpasswd, HTTP Error, HTTP Headers, HTTP-EQUIV, httpd, httpd.conf, HTTPS SSL, hyper text transfer protocol, If-Modified-Since, Javascript, Last-Modified, Linux, Login, Logs, mad skills, mod_include, mod_python, Mod_Rewrite, Mod_Rewrite examples, Mod_Security, Mod_Setenvif, mysql, Nice, nsa, password, password protection, PDF, Performance, Perl, PHP, php.ini, phpinfo, Port, post, Powweb, Prompt, Python, ram, Redirect, Redirection, Request Method, Rewrite Tricks, rewritecond, rewriterule, Robot, robots, Sample .htaccess, Scripts, Security, SEO, seo secrets, server, server config, servers, SetEnvIf, Shell, Socket, Source Code, SPEED, SSH, SSI, stat, SymLinks, trick, tutorial, ultimate htaccess, Username, Web Hosting, WordPress
Posted in Apache, Apache Modules, Cache, DreamHost, Featured, Google, Hacking, Htaccess, Linux Unix BSD, Mod_Rewrite, SEO, Security, Web Design, Web Hosting, Web Tools, Webmaster, WordPress | 71 Comments »
Wednesday, July 2nd, 2008
PHP’s fsockopen function lets you open an Internet or Unix domain socket connection for connecting to a resource, and is one of the most powerful functions available in the php language.
Tags: 500, Advanced, Apache, askapache, AskApache Crazy Cache, ASP, Bandwidth, Blocking, Cache, Cookies, CSS, curl, debugging, DNS, Examples, feed, File System, Forms, Fsockopen, GET, Google, Hacking, Htaccess, HTTP Headers, httpd, HTTPS SSL, Linux, Login, Networking, PDF, Performance, PHP, Pipelining, Port, post, ram, server, servers, Snoopy, Socket, SPEED, SSI, stat, trick, WordPress
Posted in Cache, Featured, Hacking, Linux Unix BSD, PHP, Security, Webmaster | 4 Comments »
Tuesday, April 8th, 2008
Want to block a bad robot or web scraper using .htaccess files? Here are 2 methods that illustrate blocking 436 various user-agents.
Tags: 403 Forbidden, Apache, askapache, Blocking, curl, Dig, Email, errordocument, feed, Flash, GET, Htaccess, htaccess files, Linux, Logs, Mod_Rewrite, Nice, Perl, PHP, Port, Python, ram, Rewrite Tricks, rewritecond, rewriterule, Robot, robots, rsync, Security, server, SetEnvIf, Snoopy, SPEED, SSH, SSH Tunnels, SSI, Wget
Posted in Apache, Featured, Hacking, Htaccess, Linux Unix BSD, Mod_Rewrite, Security, Server Administration, Web Hosting, Webmaster | 18 Comments »
Sunday, March 16th, 2008
Scan Apache logs for IP address that are probably evil, then generates an .htaccess file to DENY them all.
Tags: 403 Forbidden, Apache, askapache, bash, Blocking, debugging, DNS, DreamHost, error log, grep, Htaccess, htaccess files, ip abuse, Logs, Nice, Port, server, servers, Shell, shell script, SSI
Posted in Apache, DreamHost, Security, Shell Scripting | 5 Comments »
Saturday, October 20th, 2007
Very nice tutorial dealing with the robots.txt file. Shows examples for google and other search engines. Wordpress robots.txt and phpBB robots.txt sample files.
Tags: 401, 403 Forbidden, 404 Not Found, admin, Advanced, Analytics, Apache, askapache, ASP, Bandwidth, Blocking, Boot, Cache, CSS, Elite, Email, Examples, feed, GET, Google, Htaccess, HTTP Headers, HTTP Status Codes, httpd, HTTPS SSL, Linux, Login, Logs, Nice, Optimization, Perl, PHP, phpBB, Port, post, Python, ram, Redirect, Robot, robots, robots.txt, rsync, SEO, server, SSI, stat, Web Hosting, Wget, Windows XP, WordPress
Posted in Google, SEO, WordPress, phpBB | 18 Comments »
Sunday, September 23rd, 2007
Tons of awesome tips and tricks using netcat. Port redirector, nessus wrapper, capture exploits being sent by vuln scanners, etc. This is very useful for doing stuff like redirecting traffic through your firewall out to other places like web servers and mail hubs, while posing no risk to the firewall machine itself.
Tags: admin, Backups, Blocking, Boot, debugging, DNS, Ethernet, Examples, feed, GET, Login, Performance, Port, ram, Redirect, Scripts, Security, server, servers, Shell, Socket, SSI, stat, Username
Posted in Linux Unix BSD, Security | No Comments »
Thursday, May 10th, 2007
WordPress robots.txt file can make a huge impact on your WordPress blogs traffic and search engine rank. This is an SEO optimized robots.txt file.
Tags: adsense, Blocking, Examples, feed, Google, Logs, mediapartners, phpBB, Robot, robots, robots.txt, SEO, WordPress
Posted in Google, SEO, WordPress | 32 Comments »
Wednesday, January 3rd, 2007
In 2003, Nick Kew released a new module that complements Apache’s
mod_proxy and is essential for reverse-proxying. Since then he gets
regular questions and requests for help on proxying with Apache. In
this article he…
Tags: 302 Redirect, Accessibility, admin, Apache, askapache, Bandwidth, Blocking, Cache, caching, compression, Cookies, CSS, debugging, DNS, Email, error log, feed, Forms, GET, HTTP Headers, httpd, httpd.conf, HTTPS SSL, Javascript, Performance, Perl, PHP, Port, Redirect, Redirection, Rewrite Tricks, Scripts, Security, server, servers, SPEED, SSH Tunnels, SSI, stat
Posted in Apache, Htaccess | 12 Comments »
Tuesday, November 7th, 2006
mod_rewrite is very useful in many situations. Yet some behaviors were not so obvious when I started to mess with it. After many testings, I understand it much better, now. Having said that, I do not pretend to know it perfectly. I also make mistakes.
Tags: 302 Redirect, 403 Forbidden, 500, Apache, authorization, Bandwidth, Blocking, Cache, CSS, debugging, Elite, Examples, experiments, GET, Htaccess, HTTP Headers, httpd, httpd.conf, Mod_Rewrite, Nice, Perl, PHP, Port, post, Powweb, ram, Redirect, Rewrite Tricks, rewritecond, rewriterule, Robot, robots, Scripts, SEO, server, servers, SSI, stat, trick
Posted in Apache, Htaccess | 1 Comment »
Just a very brief look at speeding up form submission by delegating the processing and bandwidth to your server, not your client.
I am often logged in to my servers via SSH, and I need to download a file like a WordPress plugin. I’ve noticed many sites now employ a means of blocking robots like wget from accessing their files. Most of the time they use .htaccess to do this. So a permanent workaround has wget mimick a normal browser.