We're back up. It appears a rectifier (1 of 8) failed and its failure tripped the other units. The battery backup doesn't appear to have done its job hence voltage drop to 32V and thus your equipment powering down. On Sep 13, 2015 16:39, "Craig Whitmore" wrote:

There seems to be a power outage in the exchange affecting all chorus circuits. UFF seems to be broken as well .

Sent from my iPhone

On 13/09/2015, at 3:49 PM, Jesse Archer wrote:

We've lost power in the colo room, to both A + B feeds.

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

So the real story (as I understand so far anyway) appears to be a rectifier failed, and that caused some sort of surge upstream tripping a(n) AC breaker(s) that feeds both the A + B power feeds. This appeared to be local to do the colocation room (UFF + various ISPs). On top of that, it was found that the battery bank couldn't perform to the required level. Tauranga is a "silver" exchange and is supposed to have redundant A + B with each feed including a separate chain of UPS and generator. I believe a serious investigation needs to be made as this event took out the UFF network in Tauranga as well as virtually every other provider's gear in the colocation room. *Jesse Archer* *General Manager*Full Flavour *p. *07 577 0099 *ddi*. 07 281 1391 *s*. Skype "myfullflavour" *e*. jesse(a)fullflavour.nz *w*. fullflavour.nz http://www.fullflavourmedia.co.nz/ *a. *Basestation, 148 Durham Street, Tauranga *a*. PO Box 16306, Bethlehem 3176, New Zealand On Sun, Sep 13, 2015 at 7:01 PM, Lindsay Druett wrote:

That seems very odd Jesse. Especially considering that in all major exchanges they have a separate rectifier rack and battery bank on the A side to the B side, so if a rectifier failed and took the other rectifiers down, we should have seen one of the -48VDC feeds stay up (or at least that is what I would have expected).

All the same, I'm keen to know what actually happened. Sounds like some more major has happened.

On Sun, Sep 13, 2015 at 4:44 PM, Jesse Archer wrote:

...

We're back up.

It appears a rectifier (1 of 8) failed and its failure tripped the other units.

The battery backup doesn't appear to have done its job hence voltage drop to 32V and thus your equipment powering down. On Sep 13, 2015 16:39, "Craig Whitmore" wrote:

...

There seems to be a power outage in the exchange affecting all chorus circuits. UFF seems to be broken as well .

Sent from my iPhone

On 13/09/2015, at 3:49 PM, Jesse Archer wrote:

We've lost power in the colo room, to both A + B feeds.

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Lindsay Druett.

lindsay(a)wired.net.nz +64-7-974 9300 +64-21-92 90 92