19 Oct
2008
19 Oct
'08
9:36 a.m.
On Mon, Oct 20, 2008 at 4:31 PM, Murray Fox
This was news to me ('till international traffic on some of my sites began to climb); The National Library is this month ripping the entire NZ Internet ..... from an international host ..... blatantly ignoring robots.txt
http://www.natlib.govt.nz/about-us/current-initiatives/web-harvest-2008
Fail. I think that covers it. Cheers - N