On Thu, 24 May 2001, Simon Blake wrote:

There appears to be a strong relationship between broadcast storm levels and whether port 17 on the APE switch is shutdown or not. If it's shutdown, there's no storm, if it's enabled, there's so much traffic on APE that I can't telnet from the route server to the 3524. Port 17 appears to be Tangent/Attica, so conclude from that what you will.

We've already brought this to Tangent's attention - they're investigating currently, we're expecting results by early afternoon.

00:15:37.046026 203.97.2.203.2913 > 192.203.154.44.179: S1768928500:1768928500]

It's this that's bizarre. clix to asiaonline? Callplus/Attica transit international and domestic data via clix, but we have no relationship with AOL to my knowledge. Pesky Network Gnomes. Needless to say, we will not be bringing our ape link up again until Tangent can assure us/Citilink that all is well. JSR -- John S Russell | "What the hell is he building in there... Operations Manager | he has a router...and a table saw..." Attica/Callplus NZ | - Tom Waits, Mule Variations --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

On Thu, 24 May 2001, Juha Saarinen wrote:

It's another DDoS attack! Quick, someone call IDG and the Hirrold! Stop the presses!

Nah, I'm getting pretty bored with being quoted in IDG. :) JSR -- John S Russell | "What the hell is he building in there... Operations Manager | he has a router...and a table saw..." Attica/Callplus NZ | - Tom Waits, Mule Variations --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

On Thu, 24 May 2001, Juha Saarinen wrote:

:: Nah, I'm getting pretty bored with being quoted in IDG. :)

"Chinese hackers launch stealth Trojan InfoWar on NZ cyberspace routers!!! Run for the hills, they'll eat your hamsters next!!!"

"ISOCNZ Member calls for Hacker's Heads On Silver Platter! Chinese government complies with request!" JSR -- John S Russell | "What the hell is he building in there... Operations Manager | he has a router...and a table saw..." Attica/Callplus NZ | - Tom Waits, Mule Variations --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

Closed mouth gathers no foot. On Thu, May 24, 2001 at 12:01:19PM +1200, J S Russell wrote:

Nah, I'm getting pretty bored with being quoted in IDG. :)

To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

On Thu, 24 May 2001, Simon Blake wrote:

IHUG, you're leaking IPX, stop it please. In fact, what does that second MAC address (0050.0fb7.e04a) do, and can I block it?

Just as an FYI for people, it wasn't IPX, it was CGMP. :) David Robb --- Senior Network Engineer IHUG NZ "The Earth is a single point of failure" --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

Yup, trez broken old version of tcpdump - it thinks both CGMP and spanning tree are IPX :-(. Apologies for the gross character assignation :-).

However, the plot thickens. Telecom have now turned off spanning tree on their connection, and at that exact same moment

a) the Tangent connections burst into life

b) Clear started broadcasting spanning tree packets

From what I understand (and remember) from the conversation I had with Tangent at 1am this morning their broadcast storm control has been going mental trying to compensate for wierdness that they say is _coming_ from APE. I am unaware of how Attica can be originating that much broadcast traffic, as I have moved the APE connection from a VLAN on our switch cloud to it's own interface on one of our routers. I am going to give Tangent a call tonight and try and get an update from

Hey all. I just crawled out of bed and caught up with this thread. them - I might also have to try configuring a helper-address and see if I can get a dump of what the hell is going on. If anyone has any suggestions fire me an email or give me a call. 021837867. Cheers. James Tyson --- Samizdat New Media Solutions --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

Evening On Thu, 24 May 2001, Chris Wedgwood wrote:

On Thu, May 24, 2001 at 06:01:39PM +1200, James Tyson wrote:

From what I understand (and remember) from the conversation I had with Tangent at 1am this morning their broadcast storm control has been going mental trying to compensate for wierdness that they say is _coming_ from APE.

Storm? How many packets and what kind?

Sigh. Read back, Chris, there was an *enormous* packet storm last (Wednesday) night. From about 10pm to about midnight, when we finally got the Tangent port shutdown, every port was running hard at 35Mb+ output, except the Tangent port which was running at about 50Mb+ input, with about 75% of that being broadcasts. It rendered APE insensible, I couldn't even ping the switch from the route server, it was working so hard. I listed the packet trace of the broadcasts in an earlier message, I won't bother doing so again.

I was up there with a packet sniffer earlier today (so i could steal all your email) and i didn't see anything i could decribe as a packet storm, the number of broadcasts was low, about two or three per second, mostly arp and some spanning tree from a couple of sources.

That's because the Tangent port was still turned off when you were sniffing. When I did turn it on again this morning, the storming did start again, but eased after a period of time, I think because Tangent were making changes. Today, we've seen some particularly weird stuff happening, mainly to do with spanning tree and odd vlan stuff on other providers switches. At this stage, we seem to have lost L2 connectivity between Attica and Netgate (according to Peter Mott), but that has apparently been worked around with a static route through IHUG. I'm assuming, given the lack of any better evidence, that this is somehow related to the spanning tree packets coming out of Clear, although I think that once we get the Clear spanning tree packets stopped, somebody else will start up. So, that being the case, I've decided to get anal. My plan is to configure access to APE in much the same way as access is configured om WIX - block all MAC addresses that don't have an APE address associated with them, or a good justification for their existance. That means that packets from switches attached to APE will get dropped by the Citylink switch, which should stop most of the evil spanning tree plotting that the other switches seem to get up to. The downside is that moves adds and changes will all have to go through the Citylink NOC before they'll work. I hope that won't be too much of a hassle - yawl presumably don't change ethernet interfaces on your APE routers that regularly. So, here's my current filter plan. If you're responsible for an APE connection, please review the MAC addresses listed below, and make sure that any blocks I have planned for your connection won't cause a catastrophe. If a port you're responsible for isn't showing MAC's at the moment, it might be worthwhile letting me know what MAC address you plan to use when that port is in use. For the connections that only have one current MAC, I've already secured those ports - that's ports 3,5 and 11-16 - those users shouldn't notice any difference in switch operation. For the remainder, unless somebody presents really compelling reasons not to, I'll initiate these filters late this (Friday) morning - sorry for the short notice, but we really do still have significant problems on APE, and I'd like to get them sorted before I get sucked into the mire of ISOCNZ. Port 1 - Management VLAN (not on APE) Port 2 - Quest, currently shutdown - no MAC's Port 3 - Telecom (Global Gateway/Netgate) 00d0.06cd.c400 192.203.154.48 Allowed Port 4 - Ihug 0050.0fb7.e04a Block 0050.54d7.0320 192.203.154.36 Allow Port 5 - Mercury Primary 0090.2776.85d1 192.203.154.28 Allowed Port 6 - Clearnet, and Plain 00c0.ca18.0373 192.203.154.12 (Plain) Allow 00e0.52e9.c565 Block 00e0.52ea.fb65 Block 02e0.5209.ed01 Block Port 7 - Mercury (Backup?) - no MAC's Port 8 - Clear IP Express customers 0002.7d36.9005 Block 0010.79cc.9800 192.203.154.8 (Clix) Allow 0050.7302.f222 192.203.154.45 (Zivo) Allow 0090.27bd.cacb 192.203.154.30 (Clear ?) Allow Port 9 - Unused Port 10 - Telstra-Saturn 0030.80b2.b903 Block 00e0.1ee9.10d0 192.203.154.32 (TS) Allow Port 11 - Asiaonline 0003.6cea.cc54 192.203.154.44 Allowed Port 12 - Xtra 0010.14b6.4000 192.203.154.60 Allowed Port 13 - Route Server 1 0090.2770.dfe9 192.203.154.1 Allowed Port 14 - Actrix 02e0.3b0c.df04 192.203.154.16 Allowed Port 15 - Route Server 2 00c0.df25.f901 192.203.154.2 Allowed Port 16 - Walker Wireless 0001.638c.d838 192.203.154.49 Allowed Port 17 - Tangent - No MAC's Port 18 - MBone router - No MAC's Port 19-24 Unused Cheers Si --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

So, that being the case, I've decided to get anal. My plan is to configure access to APE in much the same way as access is configured om WIX - block all MAC addresses that don't have an APE address associated with them, or a good justification for their existance. That means that packets from switches attached to APE will get dropped by the Citylink switch, which should stop most of the evil spanning tree plotting that the other switches seem to get up to.

Your doing source MAC filtering, obviously. Is this likely to break IGMP, CGMP, etc? Cheers. James Tyson --- Samizdat New Media Solutions --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

On Fri, 25 May 2001, Simon Blake wrote:

IGMP, I wouldn't have thought so, CGMP, probably. Worst case, I guess I could do it by hand, and assign each port to a multicast VLAN. (Sez me, knowing little about vlans, less about multicast, and nothing about IGML and CGMP :-).

CGMP (when I had it turned on) has a source MAC of the thing sending them (funny that), ie the routers. So MAC filtering shouldn't break it. And having CGMP enabled is probably a good thing, because it'll stop peoples links getting flooded with other peoples multicast pr0n streams. David Robb --- Senior Network Engineer IHUG NZ "The Earth is a single point of failure" --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog

Righto. Apologies for the 2-3 short storms this morning between 9:30am-10am, that was me :-(. Specifically, I reenabled Quest's port, and everything went pearshaped. From what I can gather, this is because United are offering circuits with redundant VLAN's that assume that the connection at the end is going to provide spanning tree blocking. Spanning tree was turned off on the Citylink APE switch, so as soon as they were plugged in, boom, it all went yucky.

That being the case, I've taken Kevin(a)Telecoms suggestion, and renabled spanning tree on the APE vlan, with a priority of 10, which means it should win all ST elections, and become the root of spanning tree on APE. That has fixed the United problems, Quest appear to be running fine, I'd be interested to know if the Netgate/Mercury problems are improved, and also if the Tangent circuit for Attica can be made to work.

Well, it would appear that our link is up and happy. I havent been able to get more than 10Mb/s speeds from any ftp servers however. I will continue to play around with it. Cheers. James Tyson --- Samizdat New Media Solutions --------- To unsubscribe from nznog, send email to majordomo(a)list.waikato.ac.nz where the body of your message reads: unsubscribe nznog