Re: [nznog] NLNZHarvester2008

On 22/10/2008, at 1:17 PM, Don Gould wrote:

Gordon,

Are you guys going to tell us how much data was collected in the end?

I also note that the crawler stepped off .nz and went after links that were in .nz pages.

eg. This page has links in it to pointclark.net which were also crawled. http://www.crra.org.nz/content/view/17/7/

Why this happens is explained in the FAQ. -- Nathan Ward