Comments on: WordPress robots.txt SEO

By: AskApache

AskApache — Tue, 28 Feb 2012 13:43:40 +0000

@Chinekop ~ It's just a difference where one has a more inclusive regex. The reason is that while Google's robot may correctly interpret either, some robots from other search engines may need the other. Always go with Google!

By: Chinekop

Chinekop — Thu, 01 Dec 2011 00:58:15 +0000

Hello, thanks for this great explanation. On google's webmaster help area, they mentioned that if you want to disallow any url which has aword in it, you should add the line : Disallow: /*aword, but here you said it should be like Disallow: /*aword*, so my question is what is the difference between /*aword and /*aword*, thanks!

By: Ganeshbabu

Ganeshbabu — Sat, 02 Apr 2011 07:44:22 +0000

Wonderful Tutorials. now i am clear about these robots concept in clearly. but still have some doubts. can you explain some more details about wordpress site map and robots concept. is any plugin is availble to generate robots.txt file. if you find solution means mail me.. thanks

By: Connected

Connected — Sun, 06 Mar 2011 07:58:29 +0000

Where the latest? I need for my newly web.

By: jimmy

jimmy — Tue, 22 Feb 2011 13:05:51 +0000

very nice information...thanks

By: jimmy choo

jimmy choo — Mon, 05 Jul 2010 09:02:08 +0000

thanks a lot for this useful article

By: HDxxx

HDxxx — Sun, 04 Jul 2010 15:19:05 +0000

Thanks, This is exactly what I needed.

By: evilripper

evilripper — Mon, 10 May 2010 09:47:08 +0000

thanks!
i had some problem with comment-page-1 title duplicates and maybe I solved with these lines:

Disallow: /comment-page-*/
Disallow: /blog/comment-page/

By: it's my story

it's my story — Wed, 14 Apr 2010 13:57:54 +0000

thanks very much.. i am very confused about my duplicate content...

By: tetembak

tetembak — Wed, 14 Apr 2010 13:56:41 +0000

oh.. it's very great... there's not article as well as it.. I will learn slowly about it.. thanks...