Hello Hurricane Electric, doubling the size of our domestic table?

Hi all, So my domestic tables now have some 76,000 prefixes in them, as of ~ 6:30 last night. I guess hello HE and welcome to AKL-IX? Domestic table for NZ was more like ~ 10,000 prefixes not that long ago! -- Michael

They moved into 220q a few weeks back, routes went live on Megaport
Auckland yesterday.
They’re also selling transit now in Auckland, (great to see more options
arriving locally).
It does however seem that Auckland is just an extension of their Sydney POP
with traffic going AKL>SYD>USA.
On Wed, 19 Sep 2018 at 10:42 AM, Michael Fincham
-- Kind Regards Liam Farr Maxum Data +64-9-950-5302

One would assume however that being based at 220q they will also pickup
AKL-IX in due course as the other main Auckland IX.
On Wed, 19 Sep 2018 at 10:47 AM, Michael Fincham
-- Kind Regards Liam Farr Maxum Data +64-9-950-5302

HE peers everywhere they can, so yes I assume you'll see them on AKL-IX
too... APE might depend if they see value, last I checked between MegaIX
and AKL-IX you collected most of the routes on APE direct or indirect.
To be honest none of the IX's in AKL have been truely domestic for a long
time, there's been chunks of AU and other parts of the world, CDNs, global
OTTs (Microsoft etc) presenting non NZ routes. This has then flowed into
the concept of domestic transit, I've certainly seen bits of international
in there a long time.. as did AU when we used to talk about domestic table
too (and NZ was always in it!)
Ultimately it's probably a good thing, new names in a market is new
competition and even if you don't shop with them you'll probably gain from
the market getting a shake.
As a reminder, HE will give you even MORE routes if you bring up a
bilateral with them email the peering email @ https://peeringdb.com/net/291
or talk to Walt from Hurricane.. I'll throw him on the CC as I'm not sure
if he's on the list. Maybe he'll visit NZNOG next year.
Gav
On Wed, 19 Sep 2018 at 06:57, Liam Farr
-- Global Interconnection Director Megaport https://www.megaport.com +61 498 498 458

Eek, I wonder if this is why my NZ domestic table size had a blow out at around 0100 this morning and tipped one of my (smaller) boxes over the edge... I've spent all morning trying to triage the damn thing, knowing that the BGP table had blown out and taken all of the TCAM memory... We're still on the old style internet ISP plan from FX days; 1x International feed with a default route only 1x Domestic feed with All NZ Domestic routes, (and now a shedload more in this table) Cheers, Mike On 19/09/18 11:33, Gavin Tweedie wrote:

Yes,
This is exactly why this happened. Vocus made a change around 0050 this morning. We dropped some peerings when the prefix limits breached.
If your box isn't coping you could consider rejecting routes received from Megaport with HE in the path.
Cam
-----Original Message-----
From: nznog-bounces(a)list.waikato.ac.nz
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog

Thanks Cam, good idea Turns out that the node in question only has one exit point to the internet anyway (now), so it only really needs a default route for IPv4 and the same for IPv6 Problem solved! Regards, Mike Mike Taylor The Total Team XMPP: mike.totalteam(a)gmail.com 0800 888 326 / +64 3 3779050 On 19/09/18 12:26, Cameron Bradley wrote:

OK, so the routes are on AKL-IX now [cue a bunch more peoples import limits breaking again]. Did we not learn anything from the last time this broke a whole bunch of stuff? HE needs to be giving us some heads up before they turn up here and increase the size of NZ's domestic route table by an order of magnitude... Hello? -- Michael

Megaport AKL emailed their customers multiple times about HE turning up
their services.
NZIX also emailed.
So I assume its just 'domestic' transit providers that maybe aren't
notifying us?
Dave
On Wed, Sep 19, 2018 at 4:17 PM, Michael Fincham

On Wed, 19 Sep 2018 16:23:56 +1200
Dave Mill
I don't think this is the right way to look at it. Those routes are going to end up in everyone's "domestic" feeds. HE should have been here on NZNOG months ago making sure everyone was OK with this and ready. Nobody is running a 10x safety margin on their BGP limits... -- Michael

On Wed, 19 Sep 2018 16:25:35 +1200
Michael Fincham
I don't think this is the right way to look at it. Those routes are going to end up in everyone's "domestic" feeds. HE should have been here on NZNOG months ago making sure everyone was OK with this and ready.
And to elaborate on this: the Internet is a community resource and right now HE are being bad neighbors by not participating appropriately. -- Michael

They did notify, "months ago" back in June... (as below) and again
yesterday.
-------------------------------
Date: Tue, 18 Sep 2018 at 11:30
Subject: Re: [Megaport Auckland IX] Upcoming large increase in prefixes
from MegaIX Auckland Route Servers
To:
On Wed, 19 Sep 2018 at 16:25, Michael Fincham
-- Kind Regards Liam Farr Maxum Data +64-9-950-5302

On Wed, 19 Sep 2018 16:27:16 +1200
Liam Farr
They did notify, "months ago" back in June... (as below) and again yesterday.
Yes, but how is this meant to filter out to all the other operators not on that exchange exactly, who have spent much of the day fixing things and helping other people fix things? -- Michael

If your not on the exchange directly, you only buy (domestic) transit from
an upstream, and your upstream didn't notify you... perhaps its time to
find a new upstream or have a long hard chat with your account manager /
their ops team?
On Wed, 19 Sep 2018 at 16:29, Michael Fincham
-- Kind Regards Liam Farr Maxum Data +64-9-950-5302

Through whoever you buy domestic transit from - communication for these sorts of changes, and if required protecting customers who want to run such a service on c2610s, is a core part of such a service. If your transit providers are unable to do this I would suggest looking towards other operators. -- Nathan Ward

On Wed, 19 Sep 2018 16:32:09 +1200
Nathan Ward
Through whoever you buy domestic transit from - communication for these sorts of changes, and if required protecting customers who want to run such a service on c2610s, is a core part of such a service. If your transit providers are unable to do this I would suggest looking towards other operators.
This is a bad attitude IMO. An operator turns up in NZ, breaks a bunch of stuff, doesn't even bother to post to NZNOG letting us know they're going to break things, and you're just blaming my transit provider? Also, none of the IXes posted on NZNOG? -- Michael

The IX’s mailed out to their customers, I guess they assumed it would be unwelcome noise and that members of the exchanges would get notified that way
Nathan Brookfield
Chief Executive Officer
Simtronic Technologies Pty Ltd
http://www.simtronic.com.au
On 19 Sep 2018, at 14:36, Michael Fincham
Through whoever you buy domestic transit from - communication for these sorts of changes, and if required protecting customers who want to run such a service on c2610s, is a core part of such a service. If your transit providers are unable to do this I would suggest looking towards other operators.
This is a bad attitude IMO. An operator turns up in NZ, breaks a bunch of stuff, doesn't even bother to post to NZNOG letting us know they're going to break things, and you're just blaming my transit provider? Also, none of the IXes posted on NZNOG? -- Michael _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz https://list.waikato.ac.nz/mailman/listinfo/nznog

IMO I think its great that a large carrier has come to our neck of the
woods, and they are openly peering on both our major IX's. This is a good
thing for internet in NZ.
Notifications were sent to the affected parties who peer, months in advance.
I don't believe HE had any malicious intent to break your internet.
If your stuff broke because you have a shit upstream, fix it and move on.
On Wed, 19 Sep 2018 at 16:35, Michael Fincham
-- Kind Regards Liam Farr Maxum Data +64-9-950-5302

On Wed, 19 Sep 2018 16:39:50 +1200
Liam Farr
No contest. And I'm sure HE isn't here intentionally breaking the Internet. It is great they're here. I am just unhappy that nobody thought that maybe increasing the size of the table we all carry as "national" by this much wasn't worth mentioning to the operator community. I really don't think this is a "your transit provider" is bad moment. Who actually got a message about this from their transit provider? -- Michael

Don't forget that the NZ/AU concept of 'domestic table' is a fairly unique
concept that isn't seen in much of the rest of the world- generally any
feed is going to be default, a peering feed (customer routes), or a full
table.... Doubt anyone thought of it ;).
Congrats to Mike & Walt on extending to NZ... pretty cool to see a
worldwide carrier finally making it there - fantastic to see how far the NZ
internet has progressed!
On Tue, Sep 18, 2018 at 9:47 PM Bill

If I can ask, what does a "domestic" table genuinely bring you. If you
carry routes to your edge(s) that have both peering and transit, do you
need to extend the DFZ all the way through your network? If you've got a
single exit path (or a few) do you need to extend the domestic routes any
further?
Another thing to note: BGP Sanity. Apply AS_PATH filters to all your peers.
This doubly so if you're extending the routes throughout your network. Much
of HE's routes might not be relevant to your network and could have easily
been sanitized with some nice filters.
Also congrats to HE/Team!
-Tom
On Tue, Sep 18, 2018 at 9:50 PM Tim Hoffman

Hi, It’s been around here for a long time. Perhaps it’s a bit dated these days, sure. Getting through cables was expensive, so people offered connectivity to other NZ networks at a significantly cheaper price (think at least an order of magnitude). It wasn’t always delivered as seperate transit - often it would be a single connection but “domestic” was zero rated, and “international” was billed.. or, similar thing but shaping rather than billing. You get the idea. A large number of regular business Internet connections had this sort of thing on them I dunno, 10 years ago maybe? Few have it now, but it won’t have been too long ago that it got like that. I’m pretty sure my home internet had some different “international” component on it at some point, too. It’s meant we’ve always cared a lot about local content, which has meant our peering and the community around it has been strong, even when the economics aren’t quite as glaring anymore. It’s also a hard thing to take away - providers offering both “domestic" and “international” transit have to either take a stab at their costs (like they do for US vs. AU for the “international” part) and risk a loss, or they have to effectively say “domestic is the same price as international”. That’s a fast way to have your transit customers move elsewhere, I’d say. Worth noting, most “international” products also include “domestic” routes and transit. It’s really “domestic” and “global”.

Hey All, We have only opened the filters for HE up on one of our route servers (RS2). We chose RS2 because unfortunately only 50% of our peers have actually bothered to configure their session to this route server. The reason we chose to do it this way is to reduce the impact on participants. If you hit the limit the session drops, but RS1 will still hum along. We plan to open up RS1 when we see all the peers that did drop resume normal operation on RS2. We only just sent out an email about this today. So sorry for the lack of notice on this one, my bad. I want to also use this opportunity to remind people to bring up their sessions with our second route server if they have not done so already! Feel free to ping me off list if you need help and/or the details! Cheers Joe

Did the AKL-IX people send an email ahead of this?
I ask because we didn't see anything so I'm wondering if we just lost it in
a spam folder somewhere or if others got missed also.
Thanks,
Dylan
On 19 September 2018 at 16:23, Dave Mill

We didn’t get an email from akl IX either
On Wed, 19 Sep 2018 at 4:27 PM, Dylan Hall
--
*Jesse Archer*
*General Manager*Full Flavour
*p. *07 577 0099 *ddi*. 07 281 1391
*s*. Skype "myfullflavour"
*e*. jesse(a)fullflavour.nz
participants (14)
-
Bill
-
Cameron Bradley
-
Dave Mill
-
Dylan Hall
-
Gavin Tweedie
-
Jesse Archer
-
Joe Wooller
-
Liam Farr
-
Michael Fincham
-
Mike Taylor
-
Nathan Brookfield
-
Nathan Ward
-
Tim Hoffman
-
Tom Paseka