On 13/08/14 4:15 pm, Brian E Carpenter wrote:

The blogosphere is saying the limit has already been exceeded in some parts of the world, and that some black holes are out there as a result.

Time to sharpen up those /24 filters, I suspect.

This article in the BGPmon.net blog appeared today. http://www.bgpmon.net/what-caused-todays-internet-hiccup/ It seems there was a straw of announcements that broke some camels back...

Brian

On 13/08/2014 15:38, Dave Mill wrote:

...

Hi all

Just as a bit of a heads-up in case anyone else is hitting this issue (and not aware).

We tend to be learning a total of about 511k or so unique IPv4 routes nowadays from external peers. Combine this with various internal routes and the actual number of IPv4 routes on a couple of our border routers is a few short of 512k.

If any of these routers have a RIB or FIB limit of 512k IPv4 routes then expect some strange issues.

Like random routes not showing up in your routing table..

*cough* brocade xmr default cam partition *cough*

Have a nice day!

Cheers Dave

------------------------------------------------------------------------

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

---

NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Sebastian Castro Technical Research Manager .nz Registry Services (New Zealand Domain Name Registry Limited) desk: +64 4 495 2337 mobile: +64 21 400535

We've moved to 764k v4 and 64k v6 on our two 'problem' routers. Again, a reboot was required to change the CAM around. Cheers Dave On Thu, Aug 14, 2014 at 5:56 AM, Paul Gear wrote:

On 13/08/14 13:38, Dave Mill wrote:

Hi all

Just as a bit of a heads-up in case anyone else is hitting this issue (and not aware).

We tend to be learning a total of about 511k or so unique IPv4 routes nowadays from external peers. Combine this with various internal routes and the actual number of IPv4 routes on a couple of our border routers is a few short of 512k.

If any of these routers have a RIB or FIB limit of 512k IPv4 routes then expect some strange issues.

Like random routes not showing up in your routing table..

*cough* brocade xmr default cam partition *cough*

Saw this on a Cisco 7604 + RSP720 3CXL, which have the same default TCAM partition settings. The overall limit is 1M IPv4 or 512K IPv6 (don't try anything smaller than a /64 at home, kids!), but the default is 50% to each, and it requires a reboot to fix. Fun times.

Paul

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

Good question. On our Brocade XMRs we have the choice of about 18 different CAM partition profiles. We have chosen one named ipv4-ipv6-2. Whilst this gives us 768k ipv4 routes and 64k ipv6 routes it also gives us 64k MAC or VPLS MAC entries. We don't actually use the later on these routers but its the best CAM profile for our needs. The other two we could have picked were multi-service-4 (764k ipv4, but only 32k ipv6 (and other types of routes)) or ipv4 profile (1m ipv4...) Short story we should have picked ipv4-ipv6-2 a long time ago but mistakes and assumptions were made. If anyone is interested all of this is on page 2626 of the Brocade NetIron manual :) Cheers Dave On Thu, Aug 14, 2014 at 12:25 PM, Geoff Huston wrote:

On 14 Aug 2014, at 6:59 am, Dave Mill wrote:

...

We've moved to 764k v4 and 64k v6 on our two 'problem' routers.

Again, a reboot was required to change the CAM around.

If the unit could support 1M V4 or 512K V6 or any combination of the two, then what have you done with the remaining ~128K 32 bit TCAM entries? (i.e. why didn't you set the V6 size to 128K or the V4 size to ~872K entries?)

(I'm just curious about the maths here! ;-))

Geoff

On Thu, Aug 14, 2014 at 1:08 PM, Brian E Carpenter < brian.e.carpenter(a)gmail.com> wrote:

On 14/08/2014 12:47, Dave Mill wrote:

...

Good question.

I'm also slightly curious about the meaning of "1M" and "512k". Are they really supposed to be "1Mi" and "512Ki" (IEC units)? The powers of 2 are actually 524,288 and 1,048,576, so the discrepancy is significant compared to 64k.

It appears to mean 524,288 in this case. Now we're on 768k this actually means: IP: Raw Size 786432, User Size 786432(0 reserved)