Download Multiple URL’s FAST with cURL
Wget Trick to Download from Restrictive Sites
Before
After trick
I am often logged in to my servers via SSH, and I need to download a file like a WordPress plugin. I've noticed many sites now employ a means of blocking robots like wget from accessing their files. Most of the time they use .htaccess to do this. So a permanent workaround has wget mimick a normal browser.
Downloading Multiple Files with Curl Simultaneously
Wouldn't it be great if you could use php and curl to download multiple files simultaneously using built-in curl functions? You can!
Blocking Bad Bots and Scrapers with .htaccess
Want to block a bad robot or web scraper using .htaccess files? Here are 2 methods that illustrate blocking 436 various user-agents.
Firefox Add-ons for Web Developers
Advanced Web Development by AskApache is a Firefox Collection I created since I'm always trying new Addons out and using multiple computers and I wanted a quick and easy way to install my favorite's and keep a running list. Firebug, YSlow, LastPass, and Web Developer are the only ones I always use regularly.
I like the idea of the last.fm but it's not as powerful as the site, which is awesome. Lately listening to Kings of Leon Radio...
Serve External Javascript Files locally for Increased Speed
One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally. In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.
Fight Blog Spam with Apache
Fighting Blog Spam with Apache htaccess and other methods.
SEO with Robots.txt
Very nice tutorial dealing with the robots.txt file. Shows examples for google and other search engines. Wordpress robots.txt and phpBB robots.txt sample files.PHP5 Custom Install Shell Script Example
Today I successfully learned how to compile and run multiple custom php installations for a DreamHost account, and to get it working I came upon a simple shell script that I made a couple changes to.
3 Ways to Serve PDF Files using Htaccess Cookies, Headers, Rewrites
FYI, using the Mod_Rewrite Variables Cheatsheet makes this example, and all advanced .htaccess code easier to understand. This demo lets you set a cookie with 1 of 3 values, then you just request the pdf file with a normal link click and get 1 of 3 different responses. This is accomplished with a nice bit of .htaccess code.
Set PDF Viewing Mode - Make a selection, then click the view pdf button.
Inline Download Save As View PDF using selected mode »
Windows Optimization – Intense Part II
If you ever wanted to know the best way to defragment and speed up your Windows-Based PC, I mean you really wanted to know, here is the 2nd part to my article on Windows Speed Optimizing that details the process I have found works really well. Definately not a quick process, and certainly not the best ever, just my best ever and one that you only have to do once to get the benefits.
This article has a lot of incredibly useful (and FREE) tools I recommend, which you can grab and use without reading the article..
Speed Up Sites with htaccess Caching
2 awesome ways to implement caching on your website using Apache .htaccess or httpd.conf. Both methods are extremely simple to set up and will dramatically speed up your site!Internet Census 2012 – Port scanning using insecure embedded devices – Carna Botnet
While playing around with the Nmap Scripting Engine (NSE) we discovered an amazing number of open embedded devices on the Internet. Many of them are based on Linux and allow login to standard BusyBox with empty or default credentials. We used these devices to build a distributed port scanner to scan all IPv4 addresses. These scans include service probes for the most common ports, ICMP ping, reverse DNS and SYN scans. We analyzed some of the data to get an estimation of the IP address usage.
Install multiple OS Without Cds
Orig published 2006. I had a CD-RW drive but being a computer security researcher I had no money for blank cd-recordables. What follows is how I managed to install various operating systems on my computer (1 hard drive) without having to burn to a CD the ISO and then boot from that.
And also:
==Phrack Inc.==
Volume 0x0b, Issue 0x3f, Phile #0x0a of 0x14
|=-----------------=[ Hacking Grub for fun and profit ]=-----------------=|
Libcurl Error Codes
Freshen your Anti-virus, Anti-Rootkits, and Anti-Spyware
I've tried 100's of Anti-virus, Anti-Rootkit, and Anti-Spyware tools over the last 10 years, but it's always good to re-examine your system's security every couple of months.
Convert Image Base64 Encoder
Preload flash .flv files into browser cache
How I was able to preload many flash flv and swf files on one of my clients sites that has a lot of online video and relatively small traffic. Their site visitors would usually watch 3-10 videos per visit and so to make the videos load almost instantly on every page I came up with a way to preload the top 10 .flv files and the swf flv player files as soon as the visitor successfully started watching the 1st video. Of course I also setup .htaccess caching on the server so that once they downloaded the files into their cache they would never request them from the server again. I was having fun with this so its pretty funky and uses some really cool combinations of javascript, swf preloader from xml, css classes to help automate it all..
Fsockopen Magic
PHP's fsockopen function lets you open an Internet or Unix domain socket connection for connecting to a resource, and is one of the most powerful functions available in the php language.
Firefox Adsense WordPress Plugin
If your WordPress blog uses AdSense, and you love Firefox you will love this plugin. When a user downloads and installs Firefox through your referral, we'll credit your account with up to $1.00 (more details).
Bash alternative to Reflector for Ranking Mirrors
A pure bash alternative to the python reflector, using curl, xargs, and sort for ranking Arch Linux mirrors.
Nice and simple. Short and sweet.
Debugging Tools for Windows
I am almost ready to publish the final tutorial on using ssh tunnels from a truecrypt partition on a Windows machine. Of course I choose to go way overboard in my research and the tutorial is full of some pretty awesome windows tricks.. I will come back to this post soon and post all the other advanced tools I use for debugging windows, for now though you MUST know how to debug the kernel and use these basic debugging tools.
Caching Tutorial for Webmasters
If you examine the preferences dialog of any modern Web browser (like Internet Explorer, Safari or Mozilla), you'll probably notice a 'cache' setting. This lets you set aside a section of your computer's hard disk to store representations that you've seen, just for you. The browser cache works according to fairly simple rules. It will check to make sure that the representations are fresh, usually once a session (that is, the once in the current invocation of the browser).
Htaccess Mod_Rewrite – Guidedddd
Running a Reverse Proxy in Apache
DreamHost’s Private Servers Finally Rooted
THE Ultimate Htaccess
.htaccess is a very ancient configuration file for web servers, and is one of the most powerful configuration files most webmasters will ever come across. This htaccess guide shows off the very best of the best htaccess tricks and code snippets from hackers and server administrators.
You've come to the right place if you are looking to acquire mad skills for using .htaccess files!
Update: AskApache Password Protect Plugin
WordPress plugin gives you control over HTTP Basic Authentication for your WordPress blog which among other things, stops most automated hacking attempts and exploits being attempted, cutting down on the number of requests, connections, and mysql queries for all WordPress blogs on the Internet.
Chmod, Umask, Stat, Fileperms, and File Permissions
Unix file permissions are one of the more difficult subjects to grasp.. Well, ok maybe "grasp" isn't the word.. Master is the right word.. Unix file permissions is a hard topic to fully master, mainly I think because there aren't many instances when a computer user encounters them seriously, and bitwise is oldschool. This contains a listing of all possible permission masks and bits from a linux, php, and web hosting view.... cuz you guys AskApache Regs Rock!
Surf the net anonymously with Tor
Tor is a network of virtual tunnels that give you a very good level of anonymity, using Tor with privoxy and FoxTor for firefox gives you privacy at the touch of a button.
Google: Let’s Make the Web Faster
WordPress What Is This Plugin
It shows a semi-transparent box on the document you are viewing that tells you if its a page, single, feed, category, etc..
Creating an Offline Version of Page
So, here's what I hacked together last night, that is being used today. It's essentially 2 files.
- A php file that scrapes uses curl to scrape all the urls for the page (favicon, css, images, pdfs, etc..)
- A simple bash shell script acting as a cgi that creates a zip file of all the urls, and a self-extracting exe file for those without a winzip tool
Links to htaccess tutorials and articles
Links to htaccess tutorials and howtos in the htaccess forum
Hack WP-Cache for Maximum Speed
If you desire SPEED from your WordPress blog, the #1 speed improvement comes from using the WP-Cache Plugin. If you still desire SPEED after installing the Plugin, you can modify the WP-Cache Plugin code to make your blog even faster!
.htaccess mod_rewrite rewrite examples
A hit-list of some of my favorite mod_rewrite code snippets for .htaccess files
Firefox, Firebug, and yslow are REQUIRED
A lot of the past and upcoming articles on AskApache are focused on speeding up a server or website, or just making things more efficient. I'm starting several speed tip articles that require the following 3 programs if you want to follow along, that is... if you can keep up! ;)
Optimized Vimrc with 256 Colorscheme
This is not your everyday vimrc. This is my personalized vimrc.. works especially great on remote servers, in tmux, screen, etc.
This doesn't require a git checkout, a vim update, a bash update, an OS update, a plugin. No. 1 vimrc, 1 colorscheme, just use curl to download and it's on. I use this on Arch Linux, NetBSD, FreeBSD, OpenBSD, Debian, Ubuntu, Cent OS, Red Hat, and that's mostly it so far.
If you aren't already using vim as your primary editor in a powerful way, you are gonna owe me for this.