Blocking Bad Bots and Scrapers with .htaccess
Want to block a bad robot or web scraper using .htaccess files? Here are 2 methods that illustrate blocking 436 various user-agents.
Home » Search results for "ii"
Search For ii
Hacking VLAN switched networks
There isn't much vlan info on the net in terms of specifics and I had to learn all about it because I needed to log in to a switch that was on a different vlan. With the help of the Ettercap developers NaGA and ALoR I figured it out.
AskApache Password Protection, For WordPress
AskApache Password Protect adds some serious password protection to your WordPress Blog. Not only does it protect your wp-admin directory, but also your wp-includes, wp-content, plugins, etc. plugins as well. Imagine a HUGE brick wall protecting your frail .php scripts from the endless attacks of automated web robots and password-guessing exploit-serving scripts.
Updated robots.txt for WordPress
Implementing an effective SEO robots.txt file for WordPress will help your blog to rank higher in Search Engines, receive higher paying relevant Ads, and increase your blog traffic. Get a search robots point of view... Sweet!
Faster Form Submission and Processing with fsockopen
Part II: Example illustrating how to speed up GET/POST form submissions. Uses fsockopen to initiate a server-side background request to process the submitted data, so that the result page of the form is displayed to the client lightningly quick.
Apache Directives and Modules on DreamHost
Apache .htaccess Directives and Loaded Modules allowed on DreamHost Apache Server 2 Setups.
Awk Tutorial and Introduction
While researching a unix/linux tool awk I came upon one of the most thorough and helpful tutorials I've ever seen devoted to a particular topic. It's old-school just the way I like it. I contacted the author, Bruce Barnett because I just HAD to have this article for my readers, who are predominantly running solaris/unix/bsd/linux and he kindly gave permission.
100 Alternative Search Engines
Check out ReadWriteWeb's fantastic list of uniquely useful search engines, its a lot of fun!
SEO with Robots.txt
Very nice tutorial dealing with the robots.txt file. Shows examples for google and other search engines. Wordpress robots.txt and phpBB robots.txt sample files.Freshen your Anti-virus, Anti-Rootkits, and Anti-Spyware
I've tried 100's of Anti-virus, Anti-Rootkit, and Anti-Spyware tools over the last 10 years, but it's always good to re-examine your system's security every couple of months.
Speed Tips: Turn Off ETags
By removing the ETag header, you disable caches and browsers from being able to validate files, so they are forced to rely on your Cache-Control and Expires header.
DreamHost Site of The Month is AskApache!
AskApache.com won the contest for May! Thanks to all of you who voted for my site! Even though AskApache won the contest according to the rules, somehow they said I cheated by giving DreamHost too much free publicity and advertising. I love DreamHost!
The Final Countdown – Mario Bros Style
Tupac here is using the new youtube-nocookie option, which is a slight improvement.. but flash from youtube is always gonna be slow as they are obsessed with user-metrics and pushing content at us..
Like those behind Vista, they seem to believe we all have state-of-the-art supercomputers and datacenters like they do.
I'm staying old school, resist the gui!
Htaccess File Sample
Comprehensive .htaccess example file with advanced examples in 1 htaccess sample skeleton .htaccess file with the very best apache htaccess examples... Updated frequently based on detailed info from the Apache htaccess tutorial.
Sending POST form data with php CURL
CURL Guide for sending POST data form request with PHP and CURL
Custom PHP.ini tips and tricks
- When php run as Apache Module (mod_php)
- When php run as CGI
- When cgi?d php is run with wrapper (for FastCGI)
Fight Blog Spam with Apache
Fighting Blog Spam with Apache htaccess and other methods.
Top 5 Vulnerability Port Scanners
Asked some hackers and Computer Security gurus to list their 5 favorite Vulnerability/Port Scanners. Here are the results.My Picks
- Mod_Rewrite Tips and Tricks
- Crazy Advanced Mod_Rewrite
- THE Ultimate Htaccess
- THE Mod_Rewrite Cheatsheet
- Log Viewing in Console
- Custom php.ini Tips
- DNS: Round Robin Speed
- Fsockopen: PHP Socket-Level
- Rsync and SSH Transfer
Live Free or Die
Hacking and Hackers
The use of "hacker" to mean "security breaker" is a confusion on the part of the mass media. We hackers refuse to recognize that meaning, and continue using the word to mean someone who loves to program, someone who enjoys playful cleverness, or the combination of the two. See my article, On Hacking.
-- Richard M. Stallman
Newest Posts
- Show or Hide WordPress Admin Bar Plugin
- Optimized Vimrc with 256 Colorscheme
- Mod_Rewrite Security
- Chmod, Umask, Stat, Fileperms, and File Permissions
- PirateBay and Anonymous SOPA Press Releases
I'm Reading
- Go Null YourSelf
- AntiLimit
- BackTrack
- newestindustry.org
- Metasploit
- DoxPara Research
- Honeynet Project
- phpied
- MIT World
htaccess Guide
- htaccess tricks for Webmasters
- HTTP Header control with htaccess
- PHP on Apache tips and tricks
- SEO Redirects without mod_rewrite
- mod_rewrite examples, tips, and tricks
- HTTP Caching and Site Speedups
- Authentication on Apache
- htaccess Security Tricks and Tips
- SSL tips and examples
- Variable Fun (mod_env) Section
- .htaccess Security with MOD_SECURITY
- SetEnvIf and SetEnvIfNoCase Examples
Website Speed Tips Series
- Turn On Compression
- Add Future Expires Header
- Add Cache-Control Headers
- Turn Off ETags
- Remove Last-Modified Header
- Use Multiple SubDomains
The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect. -Tim Berners-Lee
A strong free software movement focused on the principled issues of software freedom and a strong FSF in particular will determine what freedoms the next generation of computer users enjoy. At stake is no less than the next generation's autonomy. -Benjamin Mako Hill
It's very simple - you read the protocol and write the code. -Bill Joy
HTML | DCMI | GRDDL | OGP | XOXO | XDMP | XFN | DOM | XML | XHTML 1.1 Strict | CSS 2.1 | W3C
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 3.0 License, just credit with a link.
This site is not supported or endorsed by The Apache Software Foundation (ASF). All software and documentation produced by The ASF is licensed. "Apache" is a trademark of The ASF. NCSA HTTPd.
UNIX ® is a registered Trademark of The Open Group. POSIX ® is a registered Trademark of The IEEE.