I believe (from memory) that while the APE route servers insert 9560 into the as path of routes they are distributing, AKL-IX and Megaport do not insert theirs into the path, so you may find that traffic errs toward the new IX players anyway all things being the same given the shorter ASpath. Sent from my iPhone

On Dec 12, 2014, at 12:30 PM, Ben wrote:

I think the most pragmatic solution on your end is to have local preference the same for akl-ix and ape and use bgp weight so that for the same as-length it'll prefer akl-ix. the hops with vocus in them should have an extra asn in the path, and if it was just a prepend it'd be even between ape and vocus/akl-ix.

i think it's semi-reasonable that things behave like they do. "it's just the way it is".

Ben.

...

On Fri, Dec 12, 2014 at 12:22:13PM +1300, Dave Mill wrote: Hi all (Note, I'm not trying to place blame at any party here. Just interested in people's thoughts on this situation) We, Inspire/AS17705, recently peered at AKL-IX. I was interested in using this peering exchange as much as possible so we have tended to prefer AKL-IX routes over any other peering exchange (though we still prefer bi-lats, private interconnects, etc, over AKL-IX). Simple stuff.. We learnt a lot of routes off AKL-IX. We started sending traffic out via. AKL-IX. Cue customer complaints. In general, we were sending all traffic to AS4826/Vocus out via AKL-IX. I believe if another ISP had Vocus as an international provider the traffic would ingress that ISP via their international pipe. This led to one customer having packet loss over a VPN (I'd assume the far end had congested international) and another customer getting billed for this traffic as international. We ceased learning Vocus routes from the AKL-IX. Problem solved... Except, this doesn't really sound ideal to me. If Vocus are announcing routes to AKL-IX (their choice) I would like to be free to use them. What should we be doing in a situation like this? What should Vocus be doing? Am I just making a mountain out of a molehill here and should I just shut-up and leave my current fix in place? Cheers Dave

...

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

I have seen this problem more than once, Vocus will advertise an International only customer on APE despite you not wanting them to. As a Vocus International customer I will always advertise to them with the community tag “4826:2073” so they prepend networks onto APE to stop this carry on. http://tools.vocus.com.au/additionals/communities2.0.html From: nznog-bounces(a)list.waikato.ac.nz [mailto:nznog-bounces(a)list.waikato.ac.nz] On Behalf Of Scott Pettit Sent: Friday, 12 December 2014 12:39 p.m. To: Dave Mill Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] AKL-IX and domestic vs. international Shouldn’t Vocus be adding a community to let their customers prepend/withhold their routes on to AKL-IX if they’re peering there (ditto for Megaport if they go there)? As per APE -Scott From: Dave Mill mailto:dave(a)mill.net.nz > Date: Friday, 12 December 2014 12:22 pm To: "nznog(a)list.waikato.ac.nz mailto:nznog(a)list.waikato.ac.nz " mailto:nznog(a)list.waikato.ac.nz > Subject: [nznog] AKL-IX and domestic vs. international What should Vocus be doing? -- https://s3-ap-southeast-2.amazonaws.com/e2emailsig/unnamed.png Scott Pettit | Director +64 9 9502222 | SPettit(a)end2end.co.nz mailto:SPettit(a)end2end.co.nz End 2 End Limited +64 9 9502220 205/100 Parnell Road, Parnell, Auckland 1052, New Zealand http://www.end2end.co.nz The content of this message and any attachments may be privileged, confidential or sensitive and is intended only for the use of the intended recipient(s). Any unauthorised used is prohibited. Views expressed in this message are those of the individual sender, except where stated otherwise with appropriate authority. All pricing provided is valid at the time of writing only and due to factors such as the exchange rate, may change without notice. Sales are made subject to our Terms & Conditions, available on our website or on request. _____ The content of this message and any attachments may be privileged, confidential or sensitive. Any unauthorised used is prohibited. Views expressed in this message are those of the individual sender, except where stated otherwise with appropriate authority. All pricing provided is valid at the time of writing only and due to factors such as the exchange rate, may change without notice. Sales are made subject to our Terms & Conditions, available on our website or on request. _____

I have seen this problem more than once, Vocus will advertise an International only customer on APE despite you not wanting them to. As a Vocus International customer I will always advertise to them with the community tag "4826:2073" so they prepend networks onto APE to stop

carry on.

http://tools.vocus.com.au/additionals/communities2.0.html

From: nznog-bounces(a)list.waikato.ac.nz [mailto:nznog-bounces(a)list.waikato.ac.nz] On Behalf Of Scott Pettit Sent: Friday, 12 December 2014 12:39 p.m. To: Dave Mill Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] AKL-IX and domestic vs. international

Shouldn't Vocus be adding a community to let their customers prepend/withhold their routes on to AKL-IX if they're peering there (ditto for Megaport if they go there)? As per APE

-Scott

From: Dave Mill Date: Friday, 12 December 2014 12:22 pm To: "nznog(a)list.waikato.ac.nz" Subject: [nznog] AKL-IX and domestic vs. international

What should Vocus be doing?

--

Scott Pettit | Director +64 9 9502222 | SPettit(a)end2end.co.nz

End 2 End Limited +64 9 9502220 205/100 Parnell Road, Parnell, Auckland 1052, New Zealand http://www.end2end.co.nz

The content of this message and any attachments may be privileged, confidential or sensitive and is intended only for the use of the intended recipient(s). Any unauthorised used is prohibited. Views expressed in

To be clear, this is not for the ISP receiving the routes on APE and setting their Local Pref as appropriate, it's for the Advertising to Vocus via International transit ISP. -----Original Message----- From: Ben [mailto:ben(a)meh.net.nz] Sent: Friday, 12 December 2014 12:54 p.m. To: Tony Wicks Cc: 'Scott Pettit'; 'Dave Mill'; nznog(a)list.waikato.ac.nz Subject: Re: [nznog] AKL-IX and domestic vs. international that doesn't help if people use local preference. one time coresite/any2ix california had an incident with massive packet loss, vocus withdrew advertisement whilst he.net continued to advertise for a while, then both withdrew, then vocus were the first to readvertise. people do use local preference and it can hurt especially in the case of high levels of packet loss rather than an outright outage. Ben. On Fri, Dec 12, 2014 at 12:46:31PM +1300, Tony Wicks wrote: this this

message are those of the individual sender, except where stated otherwise with appropriate authority. All pricing provided is valid at the time of writing only and due to factors such as the exchange rate, may change without notice. Sales are made subject to our Terms & Conditions, available on our website or on request.

---------------------------------------------------------------------- ----

The content of this message and any attachments may be privileged, confidential or sensitive. Any unauthorised used is prohibited. Views expressed in this message are those of the individual sender, except where stated otherwise with appropriate authority. All pricing provided is valid at the time of writing only and due to factors such as the exchange rate, may change without notice. Sales are made subject to our Terms & Conditions, available on our website or on request.

---------------------------------------------------------------------- ----

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

i only see prepend for aus/nz networks, no "do not export". and really there should be a community for don't advertise to nz / don't advertise to au if wanting to preempt issues. but you still want to advertise in au to asia etc, so it gets complicated.. Ben. On Fri, Dec 12, 2014 at 12:47:56PM +1300, Tim Hoffman wrote:

Have a look at the first hit on Google for "Vocus bgp communities" ;)

Sent from my iPhone On Dec 12, 2014, at 12:38 PM, Scott Pettit wrote:

Shouldn't Vocus be adding a community to let their customers prepend/withhold their routes on to AKL-IX if they're peering there (ditto for Megaport if they go there)? As per APE -Scott From: Dave Mill Date: Friday, 12 December 2014 12:22 pm To: "nznog(a)list.waikato.ac.nz" Subject: [nznog] AKL-IX and domestic vs. international What should Vocus be doing? --

Scott Pettit | Director +64 9 9502222 | SPettit(a)end2end.co.nz

End 2 End Limited +64 9 9502220 205/100 Parnell Road, Parnell, Auckland 1052, New Zealand http://www.end2end.co.nz

The content of this message and any attachments may be privileged, confidential or sensitive and is intended only for the use of the intended recipient(s). Any unauthorised used is prohibited. Views expressed in this message are those of the individual sender, except where stated otherwise with appropriate authority. All pricing provided is valid at the time of writing only and due to factors such as the exchange rate, may change without notice. Sales are made subject to our Terms & Conditions, available on our website or on request.

----------------------------------------------------------------------

The content of this message and any attachments may be privileged, confidential or sensitive. Any unauthorised used is prohibited. Views expressed in this message are those of the individual sender, except where stated otherwise with appropriate authority. All pricing provided is valid at the time of writing only and due to factors such as the exchange rate, may change without notice. Sales are made subject to our Terms & Conditions, available on our website or on request.

----------------------------------------------------------------------

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

So, as an attempt to summarise the replies somewhat. We should: -Treat all IXes as equal LocalPref wise -Optional: prepend an AS once on routes learnt from AKL-IX so it matches path length with Citylink run IXes -Do further ASPath length mangling if you wish to prefer one IX over another (but probably only vary the path by +/- 1 AS) Vocus international customers should: -Use an appropriate Community to achieve AS path prepending on their advertisements by Vocus to the various IXes so as to not attract domestic traffic in their international tubez. Does that all seem sensible? Cheers Dave

Vocus really, really need to offer do-not-advertise on their combined product.

I think this needs to be reiterated yet again. That and people shouldn't use local preference to decide on peering exchange within the same region. If both sides work together, then things are less likely to break. I want to go off on a slight tangent though. If APE goes down, and a provider has APE and Vocus, should traffic travel via Sydney, or is it preferable for it to go via AKL-IX? If a provider is on APE, WIX, and Vocus is it better for it to go via WIX, or via AKL-IX over Vocus transit? Valve/Steam network has some odd routing as they have peering via Equinix and transit via Highlands CDN's network. And Highlands CDN's network is advertised at Megaport. So they're in a slightly similar situation, as some people definitely are preferring Megaport over Equinix in Sydney. The carryover effect is that if local preferencing traffic to go over Megaport/Highlands rather than Equninix, the return path will still be via Equninix so an outage on either of the networks will create issues. They've had quite a lot of stability issues, and some complications like that aren't obvious at a first glance. And would be helped if more people had looking glasses. And preference wise, who's to say which is better? Cheaper or less congested or closer network wise. But as soon as both parties don't choose the same thing you increase your chances of issues - and so in a way it would be nice if more of these exchanges would fail completely rather than partially. Given that people in New Zealand really want to keep traffic domestic when possible it makes sense to make it obvious when domestic peering fails, and route via Australia or such. But to remote locations like Australia, it makes more sense to go over peering links when possible falling back on transit, before going via another country. If a provider is on Equinix Sydney, paid transit in Sydney and ANY2IX/Coresite peering in California it's better go via paid transit than California if Equinix fails. On a pure performance basis, if a provider has combined international/national from Vocus with APE and WIX peering if APE goes down latency should be reduced going via international on Vocus at AKL-IX rather than WIX. And cost wise it's cheaper than transitting to WIX. So it's actually quite a good thing for NZ domestic stability that there are becoming more peering exchanges. Ben.