*there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful* I don't necessarily disagree that a migration over time may be useful, I disagree with the end state of an inconsistent behavior... The key here is having a date by which we enforce a consistent behavior.... Having said that, fair point on RFC2119 sir :) On Sun, Feb 26, 2017 at 8:45 PM, Nathan Ward wrote:

Hi,

Given the importance of RFCs in the community, I feel it is valuable to notify this communication from me to you to the entire list.

I would note that the interpretation of RFC7947 in the following email (sorry for top posting) is in direct violation of RFC2119; 4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.

I would like to encourage (publicly) that RFCs be read consistently, rather than in a haphazard way that varies reader by reader. Consistency is the key to supporting the NZ Internet effectively *(Note, RFC2119 doesn’t say “read it this way when you can be bothered”)*. Additionally, I would like to note that I share your preference that the route server not prepend its own AS number to the AS_PATH, but that improper interpretations of RFCs could have an operational impact on the Internet in NZ as a whole.

<serious> Given exchange participants may need to implement configuration changes, I think it’s reasonable that this is opt-in. I think the Citylink approach here is reasonable, even if it doesn’t get everyone to where you want immediately. Let’s work on lobbying those participants who have not opted in, rather than flogging an almost-dead horse. I’ll help! </serious>

Cheers, Nathan

####### FOREWARDED [sic] MESSAGE BELOW #######

On 27/02/2017, at 4:57 PM, Tim Hoffman wrote:

Greetings,

Given the importance of Citylink's IX services in the community, I feel that it is valuable to notify this communication from them to the entire list.

I would note that they are explicitly stating in this communication they will not ever forcibly require consistent behavior. I would note that this is in direct violation of RFC7947;

*2.2.2.1. Route Server AS_PATH Management*

* As a route server does not participate in the process of forwarding* * data between client routers, and because modification of the AS_PATH* * attribute could affect the route server client BGP Decision Process,* * the route server SHOULD NOT prepend its own AS number to the AS_PATH* * segment nor modify the AS_PATH segment in any other way.*

I would like to encourage (publicly) that Citylink implement this consistently, rather than in a haphazard way that varies customer by customer. Consistency is the key to supporting the NZ internet effectively *(Note, RFC7947 doesn't say "do this when your customer can be bothered")*. Additionally, I would like to encourage Citylink to communicate with the NZNOG community when publishing updates that could have an operational impact on the internet in NZ as a whole.

Cheers, Tim

####### FOREWARDED MESSAGE BELOW #######

Dear REDACTED

CityLink would like to inform you of some exciting NZIX developments that concern your existing ExchangeNET connection(s). We’re introducing some changes to the way we operate the NZIX route servers in order to improve our compliance with recent proposed standards (RFC7947 and RFC7948), and to minimise the disruption caused by routine administrative changes going forward.

*Background*

CityLink provides two route servers at each of the NZIX exchanges - currently Auckland (APE), Hamilton (HIX), Wellington (WIX), Christchurch (CHIX) and Dunedin (DPE). The route servers facilitate multilateral interconnection between the participants at each exchange by simplifying the BGP configuration and admin overhead otherwise needed to exchange routing information between multiple eBGP speakers. The NZIX route servers have traditionally prepended the exchange’s Autonomous System (AS) number to the AS path of the prefixes they advertise back out to peers.

*What is changing?*

A number of ExchangeNET customers have expressed a preference that the NZIX route servers no longer prepend the exchange AS number to the AS path of advertisements sent to them. This preference is consistent with recent proposed standards (RFC7947 and RFC7948), and is the recommended best current practice for Internet exchange route server operation. However, in order to put this change into effect, ExchangeNET customers who peer with the NZIX route servers may need to make a (potentially disruptive) change to their existing BGP configuration.

While CityLink would prefer all ExchangeNET customers peering with the NZIX route servers to support this alignment with best current practice, we recognise that the timing to implement such changes may differ based on your individual business needs. Therefore, we have decided to implement the changes on a per-participant basis, and we invite those ExchangeNET customers who would like to “opt-in” to the changes (that is, for the NZIX route servers to no longer prepend the exchange AS number to the AS path of the advertisements sent to them), to contact us. We will endeavor to accommodate requests for configuration changes to be made at times convenient for each ExchangeNET customer, commencing from *13th March 2017* onwards.

*Note:* We will *not* be changing the configuration of any existing sessions without that participant’s agreement.

*What are the implications of the change?*

· ExchangeNET customers who peer with the NZIX route servers will see the AS path for prefixes received shortened by one hop (the exchange AS number). This may alter the current choice of best path for those prefixes.

· If you had been using the presence of an exchange AS in the AS path to influence your organisation’s routing policy, you may need to amend your configuration in order to achieve your routing policy objectives. It may not be quite so obvious that a given route has been learnt from a particular NZIX exchange point going forward.

· Your router may need to be configured with its equivalent of *no bgp enforce-first-as* to restore its BGP session(s) with the NZIX route servers after the change is made.

*I’d like to “opt-in” to the change. What do I need to do?*

If you’d like us to remove the exchange AS number from the AS path of the advertisements you receive from the NZIX route servers, please email us at *peering(a)citylink.co.nz * with your AS number, the contact details for the person(s) from your organisation that will be involved in making the changes, and an initial indication of when you’d like to implement the change (if known). Changes will only be implemented at a mutually-agreed time to minimise any disruption, commencing from *13th March 2017* onwards.

*I’d prefer not to “opt-in” to the change right now. Do I need to do anything?*

If you’d prefer that the NZIX route servers continue prepending the exchange AS number to the AS path of advertisements sent to you, you don’t need to take any action right now. However, we would appreciate a return email confirming the contact details for the most suitable person(s) from your organisation whom we should engage with on these matters going forward.

*I’d prefer not to “opt-in” to the change at all. Will I be encouraged to eventually?*

It is hoped that over time all ExchangeNET customers will choose to “opt-in” and thus support CityLink’s efforts to align the NZIX exchanges with best current practices, to the benefit of the wider NZ Internet community.

*What will be the “default” configuration for new ExchangeNET customers?*

Going forward, all new BGP sessions configured on the NZIX route servers will conform to the recommend best current practice (that is, the route servers will *not* prepend the exchange AS number to the AS path of prefixes sent to peers).

*Is anything else changing?*

Yes! The method by which configuration changes have historically been “pushed” to the NZIX route servers has necessitated a “hard reset” of the BGP session(s) with all existing peers. Although the two route servers at each exchange are reloaded independently (ensuring that each peer has an established session with at least one NZIX route server at all times), the overheads associated with performing the hard reset are higher than they need to be. In order to minimise the impact of routine administrative changes going forward, we are changing the way in which configuration changes are pushed to the route servers. Note that changes will continue to be applied to each route server independently, however, thereby ensuring optimal availability for NZIX.

*I have a question about these changes. Who should I contact?*

Please email us at *peering(a)citylink.co.nz * in the first instance. We’ll be happy to help!

Kind Regards,

http://www.citylink.co.nz/

*Toll Free* 0800 424 895

Level 6, 25 Cambridge Terrace

*www.citylink.co.nz http://www.citylink.co.nz/*

Wellington 6011

This mail message contains information that is confidential and which may be subject to legal privilege. If you are not the intended recipient, you must not use, distribute or copy this message. If you have received this message in error, please notify the sender immediately and erase this mail and its attachments.

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog

On 27/02/2017, at 5:50 PM, Tim Hoffman wrote:

there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful I don't necessarily disagree that a migration over time may be useful, I disagree with the end state of an inconsistent behavior... The key here is having a date by which we enforce a consistent behavior….

So let’s make up a date and push people towards it. No reason we can’t get to a consistent state, right? The Citylink IXes started life as community IXes, no reason we can’t make them community IXes again. How about your birthday next year? Perhaps we could talk about ways to track who is opting-in and who isn’t, do you have thoughts on how to achieve that? I’m not sure I can think of anything technically. Does Citylink intend to publish this information? Perhaps we can encourage people to post on the NZNOG list when they change their “mode”? With that information, you could channel your energies in to an email to a handful of operators every couple of months. That would be totally reasonable to copy to the list.

Having said that, fair point on RFC2119 sir :)

Others who thought of it first know who they are :) -- Nathan Ward

Maybe the Citylink product should just die a natural death... We have AKL-IX and Megaport now :)

On 27/02/2017, at 9:07 PM, Tim Price wrote:

How does not opting in to this change negatively affect anyone but the people who choose to abstain? (And their customers). The free market will surely sort this one out.

I'm assuming that NZIX's standard deployment will be from now on sans-ix-asn-injection so standard attrition will eventually get us to a semi-consistent state and as to the outliers... my first point applies.

On 27/02/2017 6:01 PM, "Nathan Ward" mailto:nznog(a)daork.net> wrote:

...

On 27/02/2017, at 5:50 PM, Tim Hoffman mailto:tim(a)hoffman.net.nz> wrote:

there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful I don't necessarily disagree that a migration over time may be useful, I disagree with the end state of an inconsistent behavior... The key here is having a date by which we enforce a consistent behavior….

So let’s make up a date and push people towards it. No reason we can’t get to a consistent state, right? The Citylink IXes started life as community IXes, no reason we can’t make them community IXes again. How about your birthday next year?

Perhaps we could talk about ways to track who is opting-in and who isn’t, do you have thoughts on how to achieve that? I’m not sure I can think of anything technically. Does Citylink intend to publish this information? Perhaps we can encourage people to post on the NZNOG list when they change their “mode”?

With that information, you could channel your energies in to an email to a handful of operators every couple of months. That would be totally reasonable to copy to the list.

...

Having said that, fair point on RFC2119 sir :)

Others who thought of it first know who they are :)

-- Nathan Ward

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz mailto:NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog https://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog

Generally speaking most of the internet lives in Auckland, being that thats where most of the content and our two international links live (for now). I work with other regional providers (not in Orkland) and apart from some local data for things like VPN’s and other on-net content like netlfix / google caches that can be shifted locally, the majority (over 50%) is backhauled to Auckland anyway. I imagine for most providers the majority of their data is exchanged in AKL or transited via it. (If this isn’t the case feel free to correct me). In terms of “Citylink hate” I think they reap what they sow, taking a $195 / a month service and upping it to $895 didn’t win them any favours, especially when its double what the other exchange providers are now charging, but on the positive side it did create an opportunity in the market for the likes of AKL-IX and Megaport to enter the market. (I have also personally had less than pleasant experiences with Citylink sales in the past, and would quite happily see them go by the wayside). IMHO AKL-IX and Megaport have both been doing a much better job of engaging their customers, and have been a lot easier to work with thank Citylink has been historically. Based on the customer service levels and their can-do attitude I am a fan of AKL-IX in particular. As someone who has previously had multiple Citylink IX ports (and since cancelled them) that’s my 2c worth anyway.

On 27/02/2017, at 9:17 PM, Tim Price wrote:

Right, that nicely covers Orkland but not the rest of the country where the rest of us live. Why the Citylink hate anyway? This general hate for Citylink has been becoming more and more obvious and isn’t conducive to them being forthcoming with information and otherwise participating in this community.

From: on behalf of Liam Farr Date: Monday, 27 February 2017 at 9:10 PM To: NZNOG Subject: Re: [nznog] Citylink IX communication

Maybe the Citylink product should just die a natural death...

We have AKL-IX and Megaport now :)

...

On 27/02/2017, at 9:07 PM, Tim Price mailto:tim(a)initech.co.nz> wrote:

How does not opting in to this change negatively affect anyone but the people who choose to abstain? (And their customers). The free market will surely sort this one out.

I'm assuming that NZIX's standard deployment will be from now on sans-ix-asn-injection so standard attrition will eventually get us to a semi-consistent state and as to the outliers... my first point applies.

On 27/02/2017 6:01 PM, "Nathan Ward" mailto:nznog(a)daork.net> wrote:

...

...

On 27/02/2017, at 5:50 PM, Tim Hoffman mailto:tim(a)hoffman.net.nz> wrote:

there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful I don't necessarily disagree that a migration over time may be useful, I disagree with the end state of an inconsistent behavior... The key here is having a date by which we enforce a consistent behavior….

So let’s make up a date and push people towards it. No reason we can’t get to a consistent state, right? The Citylink IXes started life as community IXes, no reason we can’t make them community IXes again. How about your birthday next year?

Perhaps we could talk about ways to track who is opting-in and who isn’t, do you have thoughts on how to achieve that? I’m not sure I can think of anything technically. Does Citylink intend to publish this information? Perhaps we can encourage people to post on the NZNOG list when they change their “mode”?

With that information, you could channel your energies in to an email to a handful of operators every couple of months. That would be totally reasonable to copy to the list.

...

Having said that, fair point on RFC2119 sir :)

Others who thought of it first know who they are :)

-- Nathan Ward

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz mailto:NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog https://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz mailto:NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz mailto:NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog https://list.waikato.ac.nz/mailman/listinfo/nznog

Disclaimer: I worked for CityLink a while back so feel free to consider the following biased or well informed as you see fit :) I think one aspect of this discussion that is often overlooked is who participates in the peering exchanges. It's not just a handful of large ISP's and content providers. Take a look at: http://nzix.net/ape-peers.html http://nzix.net/wix-peers.html We have a huge range of participants: universities, schools, government, data centres, finance, researchers, broadcasters, etc. Very few of these organisations operate networks as their core business. If it's so important to convince them all to opt-in to the change what are we doing to explain why they should make that change? Pointing at an RFC and yelling loudly hardly seems likely to be accepted as a convincing argument outside of the networking community. Personally, I think the approach CityLink is taking seems entirely reasonable. I would be keen to have regular updates from CityLink regarding the amount of uptake and any issues encountered by those making the change that might help others considering it. Thanks, Dylan On 27 February 2017 at 18:01, Nathan Ward wrote:

On 27/02/2017, at 5:50 PM, Tim Hoffman wrote:

*there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful* I don't necessarily disagree that a migration over time may be useful, I disagree with the end state of an inconsistent behavior... The key here is having a date by which we enforce a consistent behavior….

So let’s make up a date and push people towards it. No reason we can’t get to a consistent state, right? The Citylink IXes started life as community IXes, no reason we can’t make them community IXes again. How about your birthday next year?

Perhaps we could talk about ways to track who is opting-in and who isn’t, do you have thoughts on how to achieve that? I’m not sure I can think of anything technically. Does Citylink intend to publish this information? Perhaps we can encourage people to post on the NZNOG list when they change their “mode”?

With that information, you could channel your energies in to an email to a handful of operators every couple of months. That would be totally reasonable to copy to the list.

Having said that, fair point on RFC2119 sir :)

Others who thought of it first know who they are :)

-- Nathan Ward

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog