On Mon, 5 Nov 2007, Nathan Ward wrote:
Do Telecom manage this service for the Fire Service, or is it managed by a third party, or even the Fire Service itself?
It's a bit complicated. i/CAD is used by both NZFS and NZ Police, who also colocate their communications centres in the same places (in Auckland they're in a big, open-plan room together, for example). However, the outage removed their ability to even pass messages between services by way of i/CAD, which strongly suggests that i/CAD itself is hosted elsewhere. If it were in the same building, even if the links with the other centres failed (which they did) it should still have been possible to pass messages between services within the same centre.
It seems like the blame for the Fire Service outage is proposed to be Telecom by some people - but if there's a third party involved I'd put the blame on them for either: a) Not planning for an outage (which might be because of b, or incompetence). b) Deciding that an outage wasn't likely to happen.
Where it gets more complicated is the necessarily very close relationship between Telecom and the emergency services. 111 is a Telecom service, which needs to link tightly with the ComCens themselves for obvious reasons. Provider diversity becomes far more difficult in that situation. 111 calls are answered at a centre in Palmy (with a backup in CHC), and then routed regionally - Auckland for Taupo north, Wellington for the rest of the North Island, Christchurch for the South Island - but with the ability to overflow calls to another centre if the primary destination is busy. Once a call-taker has entered the information into i/CAD, the despatcher for the relevant geographical area gets presented with it and begins the turnout process. Without knowing exactly, all I can do is engage in informed speculation. Certainly the consequence of the Mayoral Drive outage was that connections to i/CAD from the Fire Service communications centres were completely down.
In short though - yep, their fallback appeared to work pretty well, and it's good that they were prepared for it.
Not just prepared, practiced. Apparently the Fire Service practices for a computer system failure on a monthly basis. That's the very best kind of business continuity planning.
Does anyone have info about how this power outage happened? The analysis of 365 Main's outage was interesting - I wonder if Telecom will provide the same level of info?
Unlikely. Very, very unlikely. Xtra's website denied any kind of outage, as is par for the course. Getting outage information from Telecom is like getting blood from a stone, all the more so if it's something that makes them look bad. -- Matthew Poole "Don't use force. Get a bigger hammer."