Hi All, This was news to me ('till international traffic on some of my sites began to climb); The National Library is this month ripping the entire NZ Internet ..... from an international host ..... blatantly ignoring robots.txt http://www.natlib.govt.nz/about-us/current-initiatives/web-harvest-2008 International traffic in NZ often costs end customers real money (especially in colo situations). A site I'm involved with, entirely NZ oriented content, served up 2GB to these guys in one day (that's half a normal month of international). Nevermind the sovereignty question; presumably the goal is that by the end of Oct the entire NZ Internet will be mirrored at nlnzdata000.us.archive.org :-/ Anyone else raging about this? Ignoring the robots.txt issue, why couldn't this have been done over national links? Or damn, at the very least they could have pinged off an email to webmaster(a)target.co.nz ahead of ripping the site to let admins get prepared? M.