Comments for AskApache searching on robots.txt http://www.askapache.com Advanced Web Development Wed, 18 Nov 2009 23:28:48 -0500 hourly 1 Comment on Updated robots.txt for WordPress by Robertino http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-109510 Robertino Sat, 14 Nov 2009 09:05:47 +0000 http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-109510 My previous posts are from October 1st/2nd 2009. You shouldn't worry about, this information is very static. It's not like they are changing the WordPress directory names, nor what's in them frequently. If it would change frequently, the theme designers, plugin coders, blog owners, and search engines wouldn't bother. What's most likely to change (frequently) are your own directory names and their content. If you don't update that part of the robots.txt file, then it will be old information. Off Topic 1: Tip : Be careful with robots.txt. It's a simple text file, that anyone can show in their browser. Don't giveaway your whole site structure, use meaningless names for the directories that you wish to protect. Off Topic 2: Bloody Hell, I see that Mehmet of GabfireThemes posted in here. How could I have missed that before ? My previous posts are from October 1st/2nd 2009.
You shouldn’t worry about, this information is very static.
It’s not like they are changing the WordPress directory names, nor what’s in them frequently.
If it would change frequently, the theme designers, plugin coders, blog owners, and search engines wouldn’t bother.
What’s most likely to change (frequently) are your own directory names and their content.
If you don’t update that part of the robots.txt file, then it will be old information.

Off Topic 1:
Tip : Be careful with robots.txt. It’s a simple text file, that anyone can show in their browser.
Don’t giveaway your whole site structure, use meaningless names for the directories that you wish to protect.

Off Topic 2:
Bloody Hell, I see that Mehmet of GabfireThemes posted in here. How could I have missed that before ?

]]> Comment on Updated robots.txt for WordPress by Dick Raney http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-109395 Dick Raney Fri, 13 Nov 2009 22:48:03 +0000 http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-109395 How old is this information? Why don't you show dates on the post or the comments? How old is this information? Why don’t you show dates on the post or the comments?

]]> Comment on WordPress robots.txt SEO by MakeMoney http://www.askapache.com/seo/wordpress-robotstxt-seo.html#comment-106446 MakeMoney Tue, 27 Oct 2009 05:35:26 +0000 http://www.askapache.com.com/seo/wordpress-robotstxt-optimized-for-seo.html#comment-106446 We try this robots for our blog. Hope get more value for SEO. We try this robots for our blog. Hope get more value for SEO.

]]> Comment on Updated robots.txt for WordPress by hpnugroho http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-105154 hpnugroho Mon, 12 Oct 2009 14:46:56 +0000 http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-105154 hi .. I have question about this file, can we use on free blog on wordpress.com or blogspot.com ? Thanks for your information hi ..
I have question about this file, can we use on free blog on wordpress.com or blogspot.com ?
Thanks for your information

]]> Comment on Redirect index.php to root by mktanny http://www.askapache.com/htaccess/redirect-index-blog-root.html#comment-105097 mktanny Sun, 11 Oct 2009 16:54:25 +0000 http://www.askapache.com/htaccess/redirect-index-blog-root.html#comment-105097 hey i am having the same problem as Dave Regards hey i am having the same problem as Dave

Regards

]]> Comment on Protecting Files with Advanced Mod_Rewrite Anti-Hotlinking by Haldun Bayrak http://www.askapache.com/htaccess/anti-hotlinking-mod_rewrite.html#comment-104907 Haldun Bayrak Mon, 05 Oct 2009 00:15:29 +0000 http://www.askapache.com/?p=3287#comment-104907 Hi, Thank for the nice prof. information shared with us! I was looking find a way to mask folder and file name seen in the URL after de domain address/ (<code>/Folder/subfolder/index.php?modname=xxxx&op=yyyyy</code>). Just to fix and show the static address (<code>/</code>) ......what ever the user is browsing. Can you help? Best regards Hi, Thank for the nice prof. information shared with us! I was looking find a way to mask folder and file name seen in the URL after de domain address/ (/Folder/subfolder/index.php?modname=xxxx&op=yyyyy). Just to fix and show the static address (/) ……what ever the user is browsing.

Can you help?

Best regards

]]> Comment on Instruct Search Engines to come back to site after you finish working on it by tom3k http://www.askapache.com/htaccess/503-service-temporarily-unavailable.html#comment-104904 tom3k Sun, 04 Oct 2009 22:10:12 +0000 http://www.askapache.com.com/htaccess/instruct-search-engines-to-come-back-to-site-after-you-finish-working-on-it.html#comment-104904 awesome bud! +1 for you :P awesome bud!

+1 for you :P

]]> Comment on Updated robots.txt for WordPress by Ujjwol http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-104836 Ujjwol Fri, 02 Oct 2009 04:44:57 +0000 http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-104836 Thanks Guys Thanks Guys

]]> Comment on Updated robots.txt for WordPress by Robertino http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-104834 Robertino Fri, 02 Oct 2009 03:13:32 +0000 http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-104834 <strong>@Ujjwol</strong> If you really want archive.org to duplicate your content remove the following. <blockquote> <pre>User-agent: ia_archiver-web.archive.org Disallow: /</pre> </blockquote> <blockquote> <pre>User-agent: ia_archiver Disallow: /</pre> </blockquote> But archive.org is a reasonably good way to proof that your site's content existed at a certain date and was not parked with a an all ads page, or had the content first (in case of plagiarism). @Ujjwol
If you really want archive.org to duplicate your content remove the following.

User-agent: ia_archiver-web.archive.org
Disallow: /
User-agent: ia_archiver
Disallow: /

But archive.org is a reasonably good way to proof that your site’s content existed at a certain date and was not parked with a an all ads page, or had the content first (in case of plagiarism).

]]> Comment on Updated robots.txt for WordPress by Ujjwol http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-104803 Ujjwol Thu, 01 Oct 2009 05:14:55 +0000 http://www.askapache.com/seo/updated-robotstxt-for-wordpress.html#comment-104803 Archive.org says it cannot crawl my website due to this <code>robots.txt</code> ? How to fix this ? Archive.org says it cannot crawl my website due to this robots.txt ?
How to fix this ?

]]>