我的wordpress所使用的robots.txt

Written by 标点符 on 2009年08月9日 in 程序测试

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /page
Disallow: /date
Disallow: /author
Disallow: /about
Disallow: /advertise

User-agent: Googlebot
# disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /*?*
Sitemap: http://www.biaodianfu.com/sitemap.xml
Sitemap: http://www.biaodianfu.com/sitemap.xml.gz