http://www.taobao.com/robots.txt5 L+ Y1 b2 ^9 q' v
# A- }2 I6 M. Y9 i" tUser-agent: Baiduspider X- W7 a* ^4 z' ^+ o1 z& Y
Disallow: /" J% G- z' h+ O3 _ ?* }" n
7 b u0 h% a" {: W( b/ V
User-agent: baiduspider1 X7 W$ X! K- r% b
Disallow: /+ c+ @ o4 u6 P' l0 G
# ]5 L1 b, j& F
貌似還是disallow0 N* x$ Y6 v$ E; E& c; |; p
|
發表於 2013-6-1 18:14:41