Anyone know what the rational is around the "official policy around APE" of only having a single MAC address per port? I would have thought that there was some technical merit in allowing at least 2 to cover rotuer replacement (without MAC address spoofing) and offer some peer router diversity options (e.g. either via separate peering sessions or VRRP implementation.) Regards Mark Frater
Well, I recall a certain 7507 many years ago that randomly flooded the peering exchange necessitating the MAC lock down. The actual number, perhaps Dylan can answer that. -----Original Message----- From: Mark Frater Sent: Wednesday, October 03, 2012 4:09 PM To: nznog(a)list.waikato.ac.nz Subject: [nznog] FW: APE MAC addresses Anyone know what the rational is around the "official policy around APE" of only having a single MAC address per port? I would have thought that there was some technical merit in allowing at least 2 to cover rotuer replacement (without MAC address spoofing) and offer some peer router diversity options (e.g. either via separate peering sessions or VRRP implementation.) Regards Mark Frater _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
Anyone know what the rational is around the "official policy around APE" of only having a single MAC address per port?
I would have thought that there was some technical merit in allowing at least 2 to cover rotuer replacement (without MAC address spoofing) and offer some peer router diversity options (e.g. either via separate peering sessions or VRRP implementation.) FFS, please don't do VRRP on an exchange. Just don't. If you want redundancy, do it properly and run two independent peers, on two independent exchange ports. VRRP is a nasty hack for having two (or more) routers facing gear that is only expecting to see one. It has absolutely no place in core or exchange routing where proper routing
On 03/10/12 16:09, Mark Frater wrote: protocols are in play. A single dynamic MAC address filter should reset on interface transition, so router replacement (which will drop the port) is covered unless you're doing something "clever" with an extra layer 2 devices between the APE port and your router. If you are, please remove it and plug your router directly into your APE port. If we all had a dollar for every outage caused by people trying to be clever on exchange nets, we'd have quite a lot of dollars. Just don't. -- don
On 10/2/2012 8:29 PM, Don Stokes wrote:
A single dynamic MAC address filter should reset on interface transition, so router replacement (which will drop the port) is covered unless you're doing something "clever" with an extra layer 2 devices between the APE port and your router. If you are, please remove it and plug your router directly into your APE port.
Operators using metro ethernet services that don't provide LLF might find this difficult.
On Tue, Oct 02, 2012 at 09:43:47PM -0700, Alastair Johnson said:
On 10/2/2012 8:29 PM, Don Stokes wrote:
A single dynamic MAC address filter should reset on interface transition, so router replacement (which will drop the port) is covered unless you're doing something "clever" with an extra layer 2 devices between the APE port and your router. If you are, please remove it and plug your router directly into your APE port.
Operators using metro ethernet services that don't provide LLF might find this difficult.
On a "normal" APE port, AFAIK, the single MAC is learnt, with a five minute timeout. So even if a port bounce can't be achieved, if the MAC is idle for 300 seconds the APE switch will forget about it, and take the next MAC that speaks into the port as the new one true MAC. Hopefully, that's your new router, and not some random metro-switch-along-the-path. Cheers Simon -- Simon Blake simon(a)katipo.co.nz Geek for hire +64 22 402 0044
On Wed, Oct 03, 2012 at 04:09:52PM +1300, Mark Frater said:
Anyone know what the rational is around the "official policy around APE" of only having a single MAC address per port?
It's a pretty common caveat, if you look at the T&C's of other IX's (LINX, AMIX, et al) they've certainly had such restrictions in the past - I don't know if they still do, but that's coz it's no longer my problem to know :-). The historic rationale was based on the observation that it's very hard to prevent loops in a shared ethernet (where you don't control the edge devices) - either you a) expose your internal loop prevention process (xSTP or EAPS or whatever) to the edge devices, in which case you get into all sorts of ugly cross vendor interactions, edge devices trying to be the root of the STP, and so forth, or you b) turn off your internal loop process in ports facing the devices of the IX participants, in which case your internal fabric should be pretty stable, but you have no mechanism to prevent loops between two edge ports. Most IX operators plump for option (B), on the basis that one should at least try and keep one's own house in order. So the MAC limit is a way of reducing the impact of loops when they do happen - if ISP A has two connections to the exchange, and loops them together, then the only traffic that can get round the loop will the two MAC addresses associated with ISP A's two ports. That changes the loop event from being a fullon broadcast/unicast flooding extravaganza to one where the only packets getting looped are packets from the ISP doing the looping. This is much easier/quicker to track down and fix, and with the right broadcast/multicast storm controls in place, often isn't particularly impactful on other IX participants.
I would have thought that there was some technical merit in allowing at least 2 to cover rotuer replacement (without MAC address spoofing)
I think you'll find that the limit is implemented as "one unique MAC address", rather than a specific MAC - if you flap the copper, or wait five minutes, then you can start over with a new MAC. No MAC spoofing required. That said, if you've got a chatty layer 2 device in front of your peering router, your MAC may have had to have been fixed to a specific device, so that the switch in front didn't keep getting the slot.
and offer some peer router diversity options (e.g. either via separate peering sessions or VRRP implementation.)
For the limit to be useful, it doesn't have to be one MAC, it has to match how many MAC's the port will see under normal conditions, so that when the port goes abnormal, it can't suddenly add a bunch of MAC's from the exchange and reannounce the looped traffic. VRRP? Surely you jest. It's a peering exchange, not a microsoft exchange :-). Cheers Simon -- Simon Blake simon(a)katipo.co.nz Geek for hire +64 22 402 0044
Simon Blake wrote:
VRRP? Surely you jest. It's a peering exchange, not a microsoft exchange :-).
Yes, we're not contemplating this...agreed it's an ugly hack. I just threw it in there to gauge reaction. ;-) Two separate IP APE addresses with two separate BGP peering sessions over one APE interface is what we were contemplating, but this of course would need to come with a (how did you put it) "random metro-switch-along-the-path" in between. Consensus seems to be two ports, two routers, two IPs, two BGP peering sessions - nothing else. Mark
On Wed, Oct 03, 2012 at 06:17:16PM +1300, Mark Frater said:
Two separate IP APE addresses with two separate BGP peering sessions over one APE interface is what we were contemplating, but this of course would need to come with a (how did you put it) "random metro-switch-along-the-path" in between.
Consensus seems to be two ports, two routers, two IPs, two BGP peering sessions - nothing else.
I think that every exchange operator the world over would prefer there to be an absolute minimum of cleverness between the port on their exchange fabric and your BGP talker - in an ideal world, there wouldn't even be media converters, just copper and fibre ties terminating directly on kit. Happily, the world doesn't revolve around exchange operators, much as they might wish it did. Back in the real world, they've learnt to cope with people mashing random chunks of layer two infrastructure on the side of their exchange. Ultimately, exchanges exist to save money for the ISP's/carriers involved, and exchange operators are aware that if the costs associated with complying with exchange T&C's get too onerous then they may lose participants (or at least the alert IX operators are aware). Assuming you can guarantee that you won't leak frames from intermediate devices, and that under normal circumstances you will have two and only two devices sending frames, then off the top of my head I can't think of a technical reason why that would present more of a risk to the exchange fabric. OTOH, I haven't been involved with IX management for a couple of years, so my views may be old hat now. So if you've got your heart set on two MAC's on a single port, ring up Citylink and try it on. You'll be asking them to forgo the revenue associated with an exchange port, so be prepared for some pushback unless you offer something up in return. Alternatively, if you've got some glass from L48 back to some location and you want to put two routers on it, then consider asking Citylink if they'll put an IX switch on that glass at your location, and give you two ports on it. Cheers Simon -- Simon Blake simon(a)katipo.co.nz Geek for hire +64 22 402 0044
participants (5)
-
Alastair Johnson -
Don Stokes -
Mark Frater -
Simon Blake -
Tony Wicks