Comments on: Instruct Search Engines to come back to site after you finish working on it

By: tom3k

tom3k — Sun, 04 Oct 2009 22:10:12 +0000

awesome bud!

+1 for you :P

By: Marc

Marc — Wed, 23 Sep 2009 19:34:41 +0000

Hi, for what is ob_start(); needed? In php.net is a comment with this ending:

$g=ob_get_clean();
echo $g;
exit;
exit();

This seems to be deleted in this article. Do you missed to delete ob_start(); as well?

By: yetyety

yetyety — Sun, 08 Feb 2009 11:04:52 +0000

Options +FollowSymLinks
RewriteEngine On
RewriteBase /
 
RewriteCond %{HTTP_USER_AGENT} ^.*(Googlebot|Googlebot|Mediapartners|Adsbot|Feedfetcher)-?(Google|Image)? [NC]
# or RewriteCond %{HTTP_USER_AGENT} ^.*google.* [NC]
RewriteRule .* /cgi-bin/error/503.php

By: Steve

Steve — Sun, 11 Jan 2009 13:01:50 +0000

Thank you, you saved me!

By: AskApache

AskApache — Fri, 21 Nov 2008 23:14:46 +0000

@ Judebert

That would result in a 500 Error for every version of Apache because mod_rewrite cannot use the R flag to redirect for anything other than 300 status codes. Redirect and RedirectMatch on the other hand, can redirect with all 57 status codes.

By: Judebert

Judebert — Wed, 19 Nov 2008 15:01:43 +0000

I tried to redirect with [R=503], but my server (shared hosting) considers that a 500 server error. I guess I'll have to go with the PHP route.

By: AskApache

AskApache — Mon, 11 Aug 2008 17:19:33 +0000

@ Mark

VERY COOL! That is probably way to advanced for most people.. I especially love your implementation of both the Header directive and Environment Variables.

By: Mark Reidenbach

Mark Reidenbach — Thu, 05 Jun 2008 23:15:20 +0000

I was curious if there was a way of doing this strictly in apache2 using mod_rewrite and mod_headers. I ended up with the following which is similar to your developer example:

ErrorDocument 503 /maintenance.htm   # Custom 503 error page
 
# Developer Address
RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.1$
 
# Only rewrite dynamic pages
RewriteCond %{REQUEST_URI} ^(\/.*\.php|\/.*\.html|\/)$
 
# Set environment variable
RewriteRule .* - [E=maintain:1]
 
# Send Retry-After header for clients that match above rewrite section
Header always set Retry-After "7200" env=maintain
 
# Send 503 error to matching clients after setting Retry-After header
RewriteCond %{ENV:maintain} 1
 
# Again, only send 503 for dynamic pages (php, html, and directory index pages)
RewriteRule ^(\/.*\.php|\/.*\.html|\/)$ - [R=503]

Hope someone finds that useful.

By: Oliver

Oliver — Thu, 10 Jan 2008 14:18:50 +0000

Thanks for this Information…

Oliver

By: Kim

Kim — Mon, 08 Oct 2007 15:14:08 +0000

Good morning,

I have question about these bots in general. Comparing the data from Google Analytics, which I run myself, and from AWStats which the company that does my SEO sends me, I’ve noticed some very large discrepancies. I’m wondering if the SEO company is using autosurf or other kinds of bots to drive up the numbers. Is there any way to detect this? I’d appreciate any advice or direction anyone could provide. My email is grace618@aol.com.

Thanks,
Kim