I think what we need to do is have two types of pointers to the servers, those that will be changed in event of failure and those that won't. We already have s[1-3].ntp.net.nz and p[1-3].ntp.net.nz , which has turned out to be blurred distinction, so we may look at changing the meaning of those. As it happens we ordered two new Sonoma D12 boxes on Friday, one to replace ntp2 and one to become ntp4, probably based in Chch. ntp2 has a blown PSU of an extraordinary type that is very expensive to get replaced, much to our surprise. cheers Jay On 2/08/2014, at 7:45 am, Mark Foster wrote:

For systems supporting multiple ntp servers you have a very good point but what about systems that only support a single ntp server? A few appliances I've dealt with fall into this area. (Perhaps a separate DNS entry and/or IP address for this so as not to wreck the "average" ?)

Or what about simply setting up any ntp client able to cope with multiple servers, with ip addresses instead of host names? DNS changes in this sort of case wouldn't matter so much?

(I confess to some holes in my understanding of the way NTP deals with multiple time servers)

Regards

-- Mark.

Sent from a mobile device.

On 2/08/2014, at 0:48, Nathan Ward wrote:

...

I’m catching up on old mail and I notice this hasn’t been fixed yet.

The NTP protocol is designed to cope with failures without this sort of re-mapping. By re-mapping ntp2 on to ntp3, you have doubled the influence that ntp3 has, so that if someone has both ntp2 and ntp3 configured (which is a reasonable assumption) and ntp3 starts giving out bad data, things are more likely to break in some situations.

We have had discussion on this list before about the architecture of this, and so I hope others have taken my advice and are running at least 4 NTP servers (ie. the ntp.net.nz 3 + 1(+) others, perhaps). If people are running with only the ntp.net.nz time servers, the impact of bad data is reduced if it bad data on ntp1, but increased if it is on ntp3.

I have just done some quick analysis, and ntpd treats named peers that resolve to the same IP addresses are distinct peers - I have a process running here that has selected the ntp2 entry as the system peer, and ntp1 and ntp3 are candidates. This means that it is averaging the three of them, and of course ntp2 and ntp3 are going to give me the same data so the average is weighted in their favour.

In any case, let’s not futs with the protocol and try outsmart it, it’s got redundancy built in, so lets use it - please remove the ntp2 -> ntp3 mapping, or, point ntp2 to somewhere other than ntp3.. though, if you point it to somewhere that people rely on as their 4th(+) server, you’re only slightly improving things.

On 14/07/2014, at 5:04 pm, Josh Simpson wrote:

...

Hi all,

As some of you may have noticed ntp2.ntp.net.nz and its aliases p2.ntp.net.nz and s2.ntp.net.nz stopped responding late on 08-07-2014.

This is related to a hardware fault with the NTP appliance, we are currently investigating both repair and replacement options.

As a temporary measure the DNS records for ntp2.ntp.net.nz, p2.ntp.net.nz and s2.ntp.net.nz have been pointed to ntp3.ntp.net.nz.

Information about these appliances and the services they provide can be found at https://ntp.net.nz.

If you need any further information, please contact us at support(a)nzrs.net.nz

Josh

-- Josh Simpson Systems Administrator .nz Registry Services M: +64 21 783 399 P: +64 4 555 0124 GPG: 6516 B4EA 413B CAED 57B5 0956 124C 8AC3 A362 8080

www.nzrs.net.nz _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Nathan Ward

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

---

NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Jay Daley Chief Executive .nz Registry Services (New Zealand Domain Name Registry Limited) desk: +64 4 931 6977 mobile: +64 21 678840 linkedin: www.linkedin.com/in/jaydaley

On 2/08/2014, at 3:41 pm, Nathan Ward wrote:

I agree re. s/p - the meaning here isn’t immediately obvious from the name, I have to look them up when I want to figure out which to use. I note that both s2 and p2 point to ntp3 - is that intentional, or should only s2 point to ntp3? Out of interest, what do s/p actually stand for?

Perhaps “pool.ntp.net.nz" could have A records for the current working servers, and ask that people with clients that don’t support more than one peer/want to do one-shot sync etc. use that.

I thought about that originally but I was concerned about being added to the global pool list. We'll look at this again though.

Great news re. ntp4! Those D12 boxes look like they’re CDMA synced - will that work in NZ? If it does, does that mean the servers will be stratum 2?

We're getting the GPS versions. With rubidium oscillators. http://www.endruntechnologies.com/pdf/SonomaD12_GPS.pdf We did have the option of PTP but didn't get it though we did consider offering it over a lan for those in the same colo as the timeservers. Of interest?

Another thought I had since last night was that maybe ntp2 could be pointed at a stratum 3 or higher server in the interim, to allow Mark’s use case to work, while not significantly impacting accuracy for those just using all 3 ntp.net.nz servers. I haven’t thought through all the complexities of this, but, it seems like an option. It would be interesting to understand how many people use ntp2 and not the other servers, I would imagine most people with single-peer clients would use s1.ntp.net.nz.

We should have delivery date early next week and if it's too far off then we'll discuss that. cheers Jay

On 2/08/2014, at 3:09 pm, Jay Daley wrote:

...

I think what we need to do is have two types of pointers to the servers, those that will be changed in event of failure and those that won't. We already have s[1-3].ntp.net.nz and p[1-3].ntp.net.nz , which has turned out to be blurred distinction, so we may look at changing the meaning of those.

As it happens we ordered two new Sonoma D12 boxes on Friday, one to replace ntp2 and one to become ntp4, probably based in Chch. ntp2 has a blown PSU of an extraordinary type that is very expensive to get replaced, much to our surprise.

cheers Jay

On 2/08/2014, at 7:45 am, Mark Foster wrote:

...

For systems supporting multiple ntp servers you have a very good point but what about systems that only support a single ntp server? A few appliances I've dealt with fall into this area. (Perhaps a separate DNS entry and/or IP address for this so as not to wreck the "average" ?)

Or what about simply setting up any ntp client able to cope with multiple servers, with ip addresses instead of host names? DNS changes in this sort of case wouldn't matter so much?

(I confess to some holes in my understanding of the way NTP deals with multiple time servers)

Regards

-- Mark.

Sent from a mobile device.

On 2/08/2014, at 0:48, Nathan Ward wrote:

...

I’m catching up on old mail and I notice this hasn’t been fixed yet.

The NTP protocol is designed to cope with failures without this sort of re-mapping. By re-mapping ntp2 on to ntp3, you have doubled the influence that ntp3 has, so that if someone has both ntp2 and ntp3 configured (which is a reasonable assumption) and ntp3 starts giving out bad data, things are more likely to break in some situations.

We have had discussion on this list before about the architecture of this, and so I hope others have taken my advice and are running at least 4 NTP servers (ie. the ntp.net.nz 3 + 1(+) others, perhaps). If people are running with only the ntp.net.nz time servers, the impact of bad data is reduced if it bad data on ntp1, but increased if it is on ntp3.

I have just done some quick analysis, and ntpd treats named peers that resolve to the same IP addresses are distinct peers - I have a process running here that has selected the ntp2 entry as the system peer, and ntp1 and ntp3 are candidates. This means that it is averaging the three of them, and of course ntp2 and ntp3 are going to give me the same data so the average is weighted in their favour.

In any case, let’s not futs with the protocol and try outsmart it, it’s got redundancy built in, so lets use it - please remove the ntp2 -> ntp3 mapping, or, point ntp2 to somewhere other than ntp3.. though, if you point it to somewhere that people rely on as their 4th(+) server, you’re only slightly improving things.

On 14/07/2014, at 5:04 pm, Josh Simpson wrote:

...

Hi all,

As some of you may have noticed ntp2.ntp.net.nz and its aliases p2.ntp.net.nz and s2.ntp.net.nz stopped responding late on 08-07-2014.

This is related to a hardware fault with the NTP appliance, we are currently investigating both repair and replacement options.

As a temporary measure the DNS records for ntp2.ntp.net.nz, p2.ntp.net.nz and s2.ntp.net.nz have been pointed to ntp3.ntp.net.nz.

Information about these appliances and the services they provide can be found at https://ntp.net.nz.

If you need any further information, please contact us at support(a)nzrs.net.nz

Josh

-- Josh Simpson Systems Administrator .nz Registry Services M: +64 21 783 399 P: +64 4 555 0124 GPG: 6516 B4EA 413B CAED 57B5 0956 124C 8AC3 A362 8080

www.nzrs.net.nz _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Nathan Ward

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

---

NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Jay Daley Chief Executive .nz Registry Services (New Zealand Domain Name Registry Limited) desk: +64 4 931 6977 mobile: +64 21 678840 linkedin: www.linkedin.com/in/jaydaley

-- Nathan Ward

-- Jay Daley Chief Executive .nz Registry Services (New Zealand Domain Name Registry Limited) desk: +64 4 931 6977 mobile: +64 21 678840 linkedin: www.linkedin.com/in/jaydaley

...

I agree re. s/p - the meaning here isn’t immediately obvious from the name, I have to look them up when I want to figure out which to use. I note that both s2 and p2 point to ntp3 - is that intentional, or should only s2 point to ntp3? Out of interest, what do s/p actually stand for?

Looking at the AUP I always presumed that the differentiation between s2/3/4 and p1/2/3 was to allow future differentiation between 'classes' of client. This doesn't seem like a bad idea, even if (currently) they wind up going to the same place(s).

Looking at the AUP, I'd always through the p* and s* referred to primary and secondary. Re NTP, primary is 'important gear' (- needed for billing\ security\ logs), secondary is a 'nice to have' (- doesn't strictly matter if NTP not functional). There's also mention of server load in the AUP, so if the load being received got out of hand, then any requests to the 'secondary' hosts could be null routed, to ensure availability\access for 'primary' systems. Paul Adshead