We are still having issues today with
the TCL cache server, is anyone other then us have issues?
Drew Collins
Group Communications Manager
Group IT Services NZ
APN Holdings NZ Ltd
My DDI: +64 9 373 9573
My Mobile +64 21 823268
My Fax: +64 9 373 6411
Ph: +64 9 379 5050
My eMail: drew_collins@apn.co.nz
Website: www.apn.co.nz
Gavin Legge <glegge@InvincibleTechnologies.co.nz>
06/07/2004 04:20 p.m.
To
nznog@list.waikato.ac.nz
cc
Subject
Re: [nznog] (no subject)
wow it would have been nice to have been told this
during the outage so
at least we could have known we where 'in line' for the switch we are
attached to to be rolled back/whatever.
oh well. - appears to be problem free today - roll on tomorrow...
On Tue, 2004-07-06 at 16:08, Drew_Collins@apn.co.nz wrote:
>
>
>
> This is the formal reply from TCL on a email I sent them two days
ago.
>
>
>
>
>
>
>
Last Sunday morning between
0100-0600
> during the regular
> planned maintanance window we upgraded 90% of the IP Networks' access
> switches to a new version of code.
>
>
The main reason for this
upgrade was
> to introduce software
> into the core switches that would support 'hitless upgrades'
upon
> the next
> release of processor cards. This feature added resiliance to the IP
> Network
> in that we could upgrade software on the core switches and customers
> would
> not notice any outage.
>
>
During the course of Sunday
day/night
> there were a number of
> customers who appeared to have unrelated issues and engineers worked
> through
> the evening to resolve these.
>
>
On Monday morning, when
most
> customers started work, other
> network faults were reported. Along with some unrelated issues, it
> also
> appeared that there were issues with regard to the software upgrade
on
> Sunday morning. About 12% of IP customers were affected with software
> issues
> related to the upgrade.
>
>
After further diagnosis,
it was
> decided to roll back to the
> previous, known good, software version of code on the core access
> switches
> in Auckland, Wellington and Christchurch. This was achieved by around
> 11am
> Monday morning and appeared to help restore service to affected
> customers.
> >From that point on, it was decided to work through all known affected
> customers with issues and rollback the software on the affected
> switches
> that they are connected to. This work carried on through the day on
a
> case
> by case basis.
>
>
This process will continue,
as
> required, and engineers will
> work with any further affected customer.
>
>
While the above is occuring,
> engineers are working with
> suppliers to resolve the issues and then a plan can be devised to
> complete
> the upgrade.
>
>
We regret the impact this
has had on
> customers. Significant
> steps were taken prior to the software change to avoid this happening.
> It is
> important to note that this version of GA (or General availability)
> code
> had gone through extensive testing in the TelstraClear IP Lab and
had
> been
> running successfully online in parts of the Network for several weeks.
>
>
> Russell Prince
> TelstraClear Business Consultant
>
>
>
> Drew Collins
> Group Communications Manager
> Group IT Services NZ
> APN Holdings NZ Ltd
>
>
> My DDI: +64 9 373 9573
> My Mobile +64 21 823268
> My Fax: +64 9 373 6411
> Ph: +64 9
379 5050
> My eMail: drew_collins@apn.co.nz
> Website: www.apn.co.nz
>
>
> ***************************************************************************
> This eMail may contain privileged and confidential information
> intended
> only for the use of the intended recipient. If you are not the
> intended
> recipient of this message, any use, dissemination, distribution or
> reproduction of this message is prohibited. Any views expressed in
> this
> message are those of the individual sender and may not necessarily
> reflect
> the views of APN News & Media NZ Ltd.
> For more information on APN News & Media NZ Ltd please visit our
> web site at http://www.apn.co.nz
> ***************************************************************************
>
>
> ______________________________________________________________________
>
> _______________________________________________
> NZNOG mailing list
> NZNOG@list.waikato.ac.nz
> http://list.waikato.ac.nz/mailman/listinfo/nznog
--
Gavin Legge <glegge@InvincibleTechnologies.co.nz>
Invincible Technologies Limited
_______________________________________________
NZNOG mailing list
NZNOG@list.waikato.ac.nz
http://list.waikato.ac.nz/mailman/listinfo/nznog
***************************************************************************
This eMail may contain privileged and confidential information intended
only for the use of the intended recipient. If you are not the intended
recipient of this message, any use, dissemination, distribution or
reproduction of this message is prohibited. Any views expressed in this
message are those of the individual sender and may not necessarily reflect
the views of APN News & Media NZ Ltd.
For more information on APN News & Media NZ Ltd please visit our
web site at http://www.apn.co.nz
***************************************************************************