I think a rubber band just fell off.
any one else seeing a *large* number of Telecom NZ outages? FR/PON/DSL/Dial etc etc jamie
They appear to have fallen off Citylink in Wgtn as well. Anybody from Telecom got an ETA? Cheers Si On Mon, Jun 20, 2005 at 11:15:05AM +1200, jamie.baddeley(a)vpc.co.nz said:
any one else seeing a *large* number of Telecom NZ outages?
FR/PON/DSL/Dial etc etc
jamie
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
I have just had a call from our Telecom account manager and he has advised
that the east and west links have both been cut, he also advised that over
5000 links are down, with no ETA.
Drew Collins
Group Communications Manager
Group Technology Services
APN Holdings NZ Ltd
DDI: +64 9 373 9573
Mobile +64 21 823268
Fax: +64 9 373 6411
Ph: +64 9 379 5050
eMail: drew.collins(a)apn.co.nz
Website: www.apn.co.nz
Simon Blake
They appear to have fallen off Citylink in Wgtn as well. Anybody from Telecom got an ETA?
Cheers Si
On Mon, Jun 20, 2005 at 11:15:05AM +1200, jamie.baddeley(a)vpc.co.nz said:
any one else seeing a *large* number of Telecom NZ outages?
FR/PON/DSL/Dial etc etc
jamie
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
=================================================================== This message has been scanned for viruses. However this message may contain executable files or other program code.
Open these types of file at your own risk. ===================================================================
******************************************************************************** NOTICE This email and any attachments are confidential. They may contain privileged information or copyright material. If you are not an intended recipient, you should not read, copy, use or disclose the contents without authorisation as we request you contact us as once by return email. Please then delete the email and any attachments from your system. We do not accept liability in connection with computer viruses, data corruption, delay, interruption, unauthorised access or unauthorised amendment. Any views expressed in this email and any attachments do not necessarily reflect the views of the company. ********************************************************************************
This Just In ... from the horse's mouth: SIGNIFICANT OUTAGE ON TELECOM NETWORK An outage on the Telecom network has been affecting a large number of customers since late this morning. Telecom is working to repair the outage as soon as possible. Telecom regrets the inconvenience to customers and will provide more information as soon as possible. The outage, which occurred at 10.48am today, has been caused by two separate incidents affecting different parts of the network. Internet, mobile and landline calls are affected. Technicians are currently working on repairing one of the fibre breaks which is in the Rimutaka area. Once it has been repaired the majority of services will be restored. Technicians are investigating the other fault and will repair it as soon as possible. The outage is affecting calls to Telecom’s own call centres, causing overloading. Telecom is asking customers to be patient. We will provide more information as soon as possible. For more information, please contact: Sarah Berry Media Relations Executive 04 498 9481 027 470 7900 Phil Love Senior Media Relations Executive 04 498 9155 027 277 8496 Melanie Marshall 04 498 9272 027 452 6231 John Goulter Public Affairs & Government Relations Manager 04 498 9369 027 232 4303 This message was sent to you because your email address is on the Telecom media releases mailing list. To be removed from this mailing list please visit http://www.telecom-media.co.nz/lists/?id=941&e=ma IMPORTANT: Security is limited with current Internet email technology. If you have doubts about the contents or source of this message, please verify that this release comes from Telecom New Zealand by checking the Telecom website at: http://www.telecom-media.co.nz/releases.asp
and all the phone numbers listed in the press release are in the affected "areas" ie 04 and 027! On Mon, 20 Jun 2005 Paul_Brislen(a)computerworld.co.nz wrote:
This Just In ... from the horse's mouth: So could you contact the horses?
lin
SIGNIFICANT OUTAGE ON TELECOM NETWORK
Internet, mobile and landline calls are affected.
04 498 9481 027 470 7900 04 498 9155 027 277 8496 04 498 9272 027 452 6231 04 498 9369 027 232 4303
Yes, have just been talking to Telecom about the DSL. Something to do with fibre links being down. Seems to be affecting a large chunk of the country, but is very scattered. I guess we just wait for something official to come through... Jeremy jamie.baddeley(a)vpc.co.nz wrote:
any one else seeing a *large* number of Telecom NZ outages?
FR/PON/DSL/Dial etc etc
jamie
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
Anyone from Telecom online? please post more information an ETA would be
great?
Regards
Adam Fenech
www.dreamnet.co.nz
0800 484 393
WARNING This email contains information which is CONFIDENTIAL and may be
subject to LEGAL PRIVILEGE. If you are not the intended recipient you must
not peruse, use, disseminate, distribute or copy this email or attachments.
If you have received this in error please notify us immediately by return
email, 0800484393 and delete this email. Thank you.
----- Original Message -----
From: "Jeremy Brake"
Yes, have just been talking to Telecom about the DSL. Something to do with fibre links being down. Seems to be affecting a large chunk of the country, but is very scattered.
I guess we just wait for something official to come through...
Jeremy
jamie.baddeley(a)vpc.co.nz wrote:
any one else seeing a *large* number of Telecom NZ outages?
FR/PON/DSL/Dial etc etc
jamie
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
-- No virus found in this incoming message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.7.8/22 - Release Date: 17/06/2005
-- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.7.8/22 - Release Date: 17/06/2005
Just spoken to several people in the Sth Island, all with no connections and unable to dial North od Wellington. Calls are terminating at a Telecom "Overloaded" message. -----Original Message----- From: jamie.baddeley(a)vpc.co.nz [mailto:jamie.baddeley(a)vpc.co.nz] Sent: Monday, 20 June 2005 11:15 To: nznog(a)list.waikato.ac.nz Subject: [nznog] I think a rubber band just fell off. any one else seeing a *large* number of Telecom NZ outages? FR/PON/DSL/Dial etc etc jamie _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
On Mon, 20 Jun 2005, jamie.baddeley(a)vpc.co.nz wrote:
any one else seeing a *large* number of Telecom NZ outages? FR/PON/DSL/Dial etc etc
Yeah, ICONZ as well. Last time I saw something like this, it was a major bearer physical discontinuity[1], which is about all you'd expect would be able to knock out big chunks of the country. JSR [1] Think "Backhoe".
Anyone got an ETA on this yet?
Regards
Adam Fenech
www.dreamnet.co.nz
0800 484 393
WARNING This email contains information which is CONFIDENTIAL and may be
subject to LEGAL PRIVILEGE. If you are not the intended recipient you must
not peruse, use, disseminate, distribute or copy this email or attachments.
If you have received this in error please notify us immediately by return
email, 0800484393 and delete this email. Thank you.
----- Original Message -----
From: "John S Russell"
On Mon, 20 Jun 2005, jamie.baddeley(a)vpc.co.nz wrote:
any one else seeing a *large* number of Telecom NZ outages? FR/PON/DSL/Dial etc etc
Yeah, ICONZ as well.
Last time I saw something like this, it was a major bearer physical discontinuity[1], which is about all you'd expect would be able to knock out big chunks of the country.
JSR
[1] Think "Backhoe".
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
-- No virus found in this incoming message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.7.8/22 - Release Date: 17/06/2005
-- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.7.8/22 - Release Date: 17/06/2005
The Media seems to have noticed. http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 http://stuff.co.nz/stuff/0,2106,3319672a10,00.html Dream Net Internet wrote:
Anyone got an ETA on this yet?
Regards Adam Fenech www.dreamnet.co.nz 0800 484 393
On 20/06/05, Tony Wicks
The Media seems to have noticed.
http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 http://stuff.co.nz/stuff/0,2106,3319672a10,00.html
And a slightly updated one: http://stuff.co.nz/stuff/0,2106,3319699a10,00.html
The story at NZ Herald has been updated: http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 Sounds like Telecom had a ring cut yesterday which made it non-redundant and Murphy's law struck somewhere else in the ring before it was repaired....
Tony Wicks wrote:
The Media seems to have noticed.
http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 http://stuff.co.nz/stuff/0,2106,3319672a10,00.html
"The fault, which happened around 10am, also crashed the New Zealand Stock Exchange..." Heh. -- Juha
Juha Saarinen wrote:
Tony Wicks wrote:
The Media seems to have noticed.
http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 http://stuff.co.nz/stuff/0,2106,3319672a10,00.html
"The fault, which happened around 10am, also crashed the New Zealand Stock Exchange..."
Heh.
Bugger, so i cant sell my telecom stock
The day InfoTech reports Milners resignation, coincidence? Post hoc ergo propter hoc. Hamish. -- http://del.icio.us/Hamish.MacEwan
On Mon, 20 Jun 2005, Juha Saarinen wrote:
"The fault, which happened around 10am, also crashed the New Zealand Stock Exchange..."
This is hyperbole, surely? I'd be fairly shocked if the NZSE internal systems themselves could be "crashed" by a fault in an external vendor. Access to the system from external points may be impacted, of course, but even then I'd have to do the Dance of Stupid[1] if the NZSE public and secure networks only used a single network provider for everything. JSR [1] You know. It's got moves like smacking yourself on the forehead, shutting your eyes while you shake your head slowly, and of course the old classic: Dragging the palms of your hands down your face while saying "Gahhhhhhh!". The Dance of Stupid. Learn it. Love it. Use it.
I think one of the articles pointed out that the NZSE was no longer able to sync between AKL and WGN, so the closed it (rather than crashing it). And what are the odds that Telecom is (a) a major share holder in NZSE and/or (b) a major vendor and/or (c) a major client (well we know this) - and either by force or political persuasion... ("... You don't need another Telco... We have five 9's uptime on our core network... We'll go somewhere else, if you go somewhere else...) got NZSE to use a single Telco for it's networking... Later'ish Craig BTW I see the list posting time is down to a few minutes now... [Disclaimer: While this is hauntingly familiar, it in no way represents the view of my employer...]
-----Original Message----- From: John S Russell [mailto:jsr(a)jsr.com] Sent: Monday, June 20, 2005 2:27 PM To: Juha Saarinen Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] I think a rubber band just fell off.
On Mon, 20 Jun 2005, Juha Saarinen wrote:
"The fault, which happened around 10am, also crashed the New Zealand Stock Exchange..."
This is hyperbole, surely? I'd be fairly shocked if the NZSE internal systems themselves could be "crashed" by a fault in an external vendor. Access to the system from external points may be impacted, of course, but even then I'd have to do the Dance of Stupid[1] if the NZSE public and secure networks only used a single network provider for everything.
JSR
lol @ the rats chewing threw the cable!!
----- Original Message -----
From: "Juha Saarinen"
Tony Wicks wrote:
The Media seems to have noticed.
http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 http://stuff.co.nz/stuff/0,2106,3319672a10,00.html
"The fault, which happened around 10am, also crashed the New Zealand Stock Exchange..."
Heh.
-- Juha
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
At 12:46 20/06/2005, Tony Wicks wrote:
The Media seems to have noticed.
http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 http://stuff.co.nz/stuff/0,2106,3319672a10,00.html
Can't even reach the stuff site.... Nz Herald is ok though.. Regards, Simon
I would guess that AT&T only has an Auckland-Wellington link via Telecom (and is down) as WIX attached networks get there fine, but GGIS or APE only networks don't - host8:~# traceroute www.stuff.co.nz traceroute to www.stuff.co.nz (202.135.111.130), 30 hops max, 38 byte packets 1 vl206-sw1-asc.wxnz.net (202.182.192.33) 0.856 ms 0.414 ms 0.752 ms 2 ip-58-28-8-2.wxnz.net (58.28.8.2) 1.436 ms 0.951 ms 1.311 ms 3 vlan-1144-aksw7.global-gateway.net.nz (203.96.116.53) 2.219 ms 2.285 ms 2.071 ms 4 f4-0-1-30.akcr5.global-gateway.net.nz (210.55.202.100) 1.873 ms 2.243 ms 2.086 ms 5 att.ape.net.nz (192.203.154.74) 1.747 ms 1.861 ms 2.527 ms 6 * * * 7 * * * 8 * * * 9 * * * Simon Byrnand wrote:
At 12:46 20/06/2005, Tony Wicks wrote:
The Media seems to have noticed.
http://www.nzherald.co.nz/index.cfm?c_id=1&ObjectID=10331696 http://stuff.co.nz/stuff/0,2106,3319672a10,00.html
Can't even reach the stuff site.... Nz Herald is ok though..
Regards, Simon
Update from Telecom. Two Exchanges are out (Masterton and Wanganui), there is also a fibre cut somewhere (they don't know where yet) and the main lines to the South Island are out. This is affecting Data and Mobile networks in the North Island, while the loss to the South Island means no Mobile/Data connectivity between the Islands. ETA at this stage is 4pm, though they of course hope to have it back before then. I asked about using Frame Relay, apparently other customers have tried this, with very little success, due to that system now being overloaded... [And emails to this list are taking 30 minutes to show up]
Major core network outage at Masterton apparently (do much for redundant routes...) It's taken out our connectivity between AKL, WGN and CHC. (Telecom POP product, aka NGN) No ETA when I called about 10:45am. So much for 5 nines.
-----Original Message----- From: jamie.baddeley(a)vpc.co.nz [mailto:jamie.baddeley(a)vpc.co.nz] Sent: Monday, June 20, 2005 11:15 AM To: nznog(a)list.waikato.ac.nz Subject: [nznog] I think a rubber band just fell off.
any one else seeing a *large* number of Telecom NZ outages?
FR/PON/DSL/Dial etc etc
Ihug's web site has an ETA of 4pm! This is really unacceptable. http://www.ihug.co.nz/cgi-bin/motd.pl?182125.motd Craig Humphrey wrote:
Major core network outage at Masterton apparently (do much for redundant routes...) It's taken out our connectivity between AKL, WGN and CHC. (Telecom POP product, aka NGN)
No ETA when I called about 10:45am.
So much for 5 nines.
-----Original Message----- From: jamie.baddeley(a)vpc.co.nz [mailto:jamie.baddeley(a)vpc.co.nz] Sent: Monday, June 20, 2005 11:15 AM To: nznog(a)list.waikato.ac.nz Subject: [nznog] I think a rubber band just fell off.
any one else seeing a *large* number of Telecom NZ outages?
FR/PON/DSL/Dial etc etc
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
participants (19)
-
Bjorn Nilsen -
Craig Humphrey -
Dream Net Internet -
Drew.Collins＠apn.co.nz -
Hamish MacEwan -
Ian McDonald -
jamie.baddeley＠vpc.co.nz -
Jeremy Brake -
John S Russell -
Juha Saarinen -
Lin Nah -
Matthew Poole -
Paul Shingles -
Paul_Brislen＠computerworld.co.nz -
Simon Blake -
Simon Byrnand -
Tony Wicks -
Tristram Cheer -
VeNoMouS