As a reasonably large provider of NZ-based hosting services, we've certainly noticed NatLib's mirroring activity as a non-negligible increase on our international bandwidth utilisation over the last few weeks. Yes, there are a million and one things that'll have an impact on your bandwidth consumption. However, I echo Murray's frustration that a New Zealand organisation either wouldn't think about this, or have completely ignored it.
Also there is the problem of "NZ Only Content" which the international spiders will not see/get such as the tvnzondemand and the citylink NZ Anycast content (+ others I am sure) They also note that they will "stop" at a certain point so some large websites will not get mirrored 100% .. So whats the point of that? you either want all or nothing. I am 100% there is some content people don't want to be spidered. a quick .htaccess to deny their spider. (I think this will work) RewriteEngine On Options +FollowSymLinks RewriteCond %{HTTP_USER_AGENT} ^NLNZHarvester2008 RewriteRule ^.* - [F,L] (And the spider will get a 403 - hopefully) Thanks Craig