Hi all, Through our own stupidity we were inadvertently blocking a recent allocation from APNIC and thought I'd mention it here. We had a bogon list with 112.0.0.0/5 in it and misread it as /8 several times before growing a brain and sorting it out. Just thought I'd mention it so folks can recheck their bogon lists to ensure that the following aren't being stopped by a larger aggregation. 116/8 APNIC 117/8 APNIC 118/8 APNIC 119/8 APNIC 120/8 APNIC Cheers, Gerard -- Netspace Services Limited http://www.netspace.net.nz Phone +64 4 917 8098 Mobile +64 21 246 2266 Level One, 220 Thorndon Quay, Thorndon PO Box 12-082, Thorndon, Wellington 6004, New Zealand
On 29/06/2007, at 11:37 AM, Gerard Creamer wrote:
We had a bogon list with 112.0.0.0/5 in it and misread it as /8 several times before growing a brain and sorting it out. Just thought I'd mention it so folks can recheck their bogon lists to ensure that the following aren't being stopped by a larger aggregation.
We're still using static bogon filters? Team Cymru provide a bogon BGP feed which make keeping up with bogon changes a cinch. 1 - Peer with the Cmyru bogon route server. 2 - Profit! http://www.cymru.com/BGP/bogon-rs.html Cheers, Jonny.
Jonny Martin wrote:
On 29/06/2007, at 11:37 AM, Gerard Creamer wrote:
We had a bogon list with 112.0.0.0/5 in it and misread it as /8 several times before growing a brain and sorting it out. Just thought I'd mention it so folks can recheck their bogon lists to ensure that the following aren't being stopped by a larger aggregation.
We're still using static bogon filters?
Team Cymru provide a bogon BGP feed which make keeping up with bogon changes a cinch. 1 - Peer with the Cmyru bogon route server. 2 - Profit!
Ignoring some people/organisations that may have corporate or architectural (or simply rule with an iron fist) approaches that prevent or restrict giving the potential to explode your network to a third party, of course. However, there is a safer approach still: don't use bogon filters at all. I've managed to convince myself they have caused, and continue to cause, far more damage than good. YMMV of course.
I've never particularly seen it as worthwhile to filter against any more bogons than RFC1918 - they're the ones likely to cause problems in filters etc... -Michael Fincham Unleash On Fri, 2007-06-29 at 11:14 +1000, Alastair Johnson wrote:
Jonny Martin wrote:
On 29/06/2007, at 11:37 AM, Gerard Creamer wrote:
We had a bogon list with 112.0.0.0/5 in it and misread it as /8 several times before growing a brain and sorting it out. Just thought I'd mention it so folks can recheck their bogon lists to ensure that the following aren't being stopped by a larger aggregation.
We're still using static bogon filters?
Team Cymru provide a bogon BGP feed which make keeping up with bogon changes a cinch. 1 - Peer with the Cmyru bogon route server. 2 - Profit!
Ignoring some people/organisations that may have corporate or architectural (or simply rule with an iron fist) approaches that prevent or restrict giving the potential to explode your network to a third party, of course.
However, there is a safer approach still: don't use bogon filters at all. I've managed to convince myself they have caused, and continue to cause, far more damage than good.
YMMV of course.
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog -- -Michael Fincham
Unleash Technology Solutions
Michael Fincham wrote:
I've never particularly seen it as worthwhile to filter against any more bogons than RFC1918 - they're the ones likely to cause problems in filters etc...
There were cases a few years ago of spammers squatting on unallocated space. I don't know if this is still a problem, and with more and more spam coming from botnets now, I'm guessing it's not. Dean
Dean Pemberton wrote:
Michael Fincham wrote:
I've never particularly seen it as worthwhile to filter against any more bogons than RFC1918 - they're the ones likely to cause problems in filters etc...
There were cases a few years ago of spammers squatting on unallocated space. I don't know if this is still a problem, and with more and more spam coming from botnets now, I'm guessing it's not.
You would be guessing wrong then. Take a brief perusal of http://www.completewhois.com/hijacked/index.htm for a brief and incomplete list. Latest entries added 4th May apparently. Then take a look at their list as of yesterday, of bogons in-use compiled from live BGP vs the direct RIR-whois feeds of allocated IP ranges. http://www.completewhois.com/bogons/active_bogons.htm Shows a few NZ companies (BNZ, Telecom, Datacom Systems, TelstraClear) amidst a pile of internationals apparently squatting on in space that APNIC is not reporting as registered yet. AJ
On 1/07/2007, at 1:52 AM, TreeNet Admin wrote:
Dean Pemberton wrote:
Michael Fincham wrote:
I've never particularly seen it as worthwhile to filter against any more bogons than RFC1918 - they're the ones likely to cause problems in filters etc...
There were cases a few years ago of spammers squatting on unallocated space. I don't know if this is still a problem, and with more and more spam coming from botnets now, I'm guessing it's not.
You would be guessing wrong then.
Then take a look at their list as of yesterday, of bogons in-use compiled from live BGP vs the direct RIR-whois feeds of allocated IP ranges. http://www.completewhois.com/bogons/active_bogons.htm Shows a few NZ companies (BNZ, Telecom, Datacom Systems, TelstraClear) amidst a pile of internationals apparently squatting on in space that APNIC is not reporting as registered yet.
Did you check the APNIC whois database? The above prefixes all look legit to me. Cheers, Jonny.
TreeNet Admin wrote:
Dean Pemberton wrote:
There were cases a few years ago of spammers squatting on unallocated space. I don't know if this is still a problem, and with more and more spam coming from botnets now, I'm guessing it's not.
You would be guessing wrong then.
[...]
Shows a few NZ companies (BNZ, Telecom, Datacom Systems, TelstraClear) amidst a pile of internationals apparently squatting on in space that APNIC is not reporting as registered yet.
There's a difference between spam originating from spam farms using hijacked or purely unallocated space, and administrative oversights by large multinationals. I'd be willing to bet some of the 'problems' caught by CompleteWhois were caused during the swamp-space-shuffle a few years ago with the old 202.27/16, 202.36/15, 202.49/16, 202.50/16, etc, swamp blocks that TNZ was maintaining being transferred to APNIC. aj
On 29/06/2007, at 1:14 PM, Alastair Johnson wrote:
Jonny Martin wrote:
On 29/06/2007, at 11:37 AM, Gerard Creamer wrote:
We had a bogon list with 112.0.0.0/5 in it and misread it as /8 several times before growing a brain and sorting it out. Just thought I'd mention it so folks can recheck their bogon lists to ensure that the following aren't being stopped by a larger aggregation.
We're still using static bogon filters?
Team Cymru provide a bogon BGP feed which make keeping up with bogon changes a cinch. 1 - Peer with the Cmyru bogon route server. 2 - Profit!
Ignoring some people/organisations that may have corporate or architectural (or simply rule with an iron fist) approaches that prevent or restrict giving the potential to explode your network to a third party, of course.
How does network explosion happen? My recommended way of using Team Cymru bogon filters is to get the BGP feed, and filter it so that you only accept prefixes that fall within your list of currently known bogon prefixes, with the prefix length that you currently know. From there, all they have the ability to do is to withdraw bogons, not introduce new ones. The only network explosion I could see would be large amounts of advertisement/withdrawl churn eating control plane cycles, but these same networks peer with direct competitors already, so it's not really introducing any new attack vectors.
However, there is a safer approach still: don't use bogon filters at all. I've managed to convince myself they have caused, and continue to cause, far more damage than good.
YMMV of course.
Fair enough. I don't have any data to suggest how many attacks/whatever they prevent these days, but if they don't have much effect that may be because people don't bother hijacking bogon space, because of the (perceived?) widespread deployment of filters to prevent it. -- Nathan Ward
Nathan Ward wrote:
How does network explosion happen?
"Potential"
My recommended way of using Team Cymru bogon filters is to get the BGP feed, and filter it so that you only accept prefixes that fall within your list of currently known bogon prefixes, with the prefix length that you currently know. From there, all they have the ability to do is to withdraw bogons, not introduce new ones.
The only network explosion I could see would be large amounts of advertisement/withdrawl churn eating control plane cycles, but these same networks peer with direct competitors already, so it's not really introducing any new attack vectors.
Right - these are all good mitigation tactics and should be applied to any peer. I'd also assume you would max-prefix the session to something reasonable as well, because if you're going to allow a prefix length range of say /8 to /24, you have the POTENTIAL for mass injection of prefixes. Peering with direct competitors or any other random network vs peering with something that influences your network 'security' and operation are two quite different things. If you're using that BGP-fed bogon list to trigger uRPF for instance, it's an entirely new potential attack vector. Have I heard of anything happening like that? No. Do I believe Team Cymru would ever do anything like that? No. Can accidents happen? Yes. There are risk averse operators and corps out there that for reasons like these would not peer with a third party for that.
I don't have any data to suggest how many attacks/whatever they prevent these days, but if they don't have much effect that may be because people don't bother hijacking bogon space, because of the (perceived?) widespread deployment of filters to prevent it.
I'm not hugely convinced they did all that much to stop attack traffic to begin with. If more networks wisely implemented uRPF and other techniques of its ilk on their subscriber/customer aggregation platforms, there would be far less need for bogon filtering and all the headaches that have gone with it. I've dealt with far too much pain when getting IP space in 219/8, 220/8, 222/8, etc, to ever want to implement a bogon filter myself. Of course, other operators that choose to blanket blackhole all APNIC space are another headache :\. aj. -- "Be liberal in what you accept, but conservative in what you send"?
On 1/07/2007, at 1:27 AM, Alastair Johnson wrote:
Nathan Ward wrote:
How does network explosion happen?
"Potential"
My recommended way of using Team Cymru bogon filters is to get the BGP feed, and filter it so that you only accept prefixes that fall within your list of currently known bogon prefixes, with the prefix length that you currently know. From there, all they have the ability to do is to withdraw bogons, not introduce new ones.
The only network explosion I could see would be large amounts of advertisement/withdrawl churn eating control plane cycles, but these same networks peer with direct competitors already, so it's not really introducing any new attack vectors.
Right - these are all good mitigation tactics and should be applied to any peer. I'd also assume you would max-prefix the session to something reasonable as well, because if you're going to allow a prefix length range of say /8 to /24, you have the POTENTIAL for mass injection of prefixes.
Why would you accept a range? I'd just accept the shortest prefix, if a longer prefix is de-bogoned it just means you don't count the still-bogon part as bogon until you update those filters. Think of it as a way to shorten bogon lists only, not modify them. Sure you don't get full coverage for a bit, but you certainly get more than just not filtering at all. I've done this before on a router that did little more than do network wide blackholes and that sort of thing, it worked great - of course, Cymru didn't try to dump large numbers of prefixes at me, so.. y'know.
Peering with direct competitors or any other random network vs peering with something that influences your network 'security' and operation are two quite different things. If you're using that BGP-fed bogon list to trigger uRPF for instance, it's an entirely new potential attack vector.
Sure, but I'm betting that it can be done smartly.
Have I heard of anything happening like that? No. Do I believe Team Cymru would ever do anything like that? No. Can accidents happen? Yes.
There are risk averse operators and corps out there that for reasons like these would not peer with a third party for that.
Yep, I can understand that they exist, I'm just not convinced that it's terribly justified :-)
I don't have any data to suggest how many attacks/whatever they prevent these days, but if they don't have much effect that may be because people don't bother hijacking bogon space, because of the (perceived?) widespread deployment of filters to prevent it.
I'm not hugely convinced they did all that much to stop attack traffic to begin with.
If more networks wisely implemented uRPF and other techniques of its ilk on their subscriber/customer aggregation platforms, there would be far less need for bogon filtering and all the headaches that have gone with it.
Does anyone have numbers of this sort of stuff? Dean - was there any data in your blackhole network whatever datasets that had info about this?
I've dealt with far too much pain when getting IP space in 219/8, 220/8, 222/8, etc, to ever want to implement a bogon filter myself. Of course, other operators that choose to blanket blackhole all APNIC space are another headache :\.
Indeed. I wouldn't recommend implementing bogon filters unless you do it really smartly, because as you say, more bad than good. The solution to a number of the "third party is scary" problems here is simply using BGP triggered blackholes to do this internally, and make sure you pay really really close attention to the mailing lists, or maybe rig up some thing so when Cymru change their announcements you get a notification or perhaps it drops it in to your table after a few hours of delay. -- Nathan Ward
Nathan Ward wrote:
On 1/07/2007, at 1:27 AM, Alastair Johnson wrote:
range of say /8 to /24, you have the POTENTIAL for mass injection of prefixes.
Why would you accept a range? I'd just accept the shortest prefix, if a longer prefix is de-bogoned it just means you don't count the still-bogon part as bogon until you update those filters. Think of it as a way to shorten bogon lists only, not modify them. Sure you don't get full coverage for a bit, but you certainly get more than just not filtering at all.
Fair point; but you're introducing additional touch requirements to your network devices, and you lose out on some usefulness of your bogon filtering, if say, 96/6 suddenly has one of those /8's de-bogoned. If you're prepared to have your bogon filter value deteriorate like that, then I would have to question their value at all.
two quite different things. If you're using that BGP-fed bogon list to trigger uRPF for instance, it's an entirely new potential attack vector.
Sure, but I'm betting that it can be done smartly.
Everything can be done smartly - in theory. However, it is a risk, and runs the potential to be a VERY BIG risk if you have lots of SLAs and real money at risk.... and I'd say some risk assessors would not accept it.
There are risk averse operators and corps out there that for reasons like these would not peer with a third party for that.
Yep, I can understand that they exist, I'm just not convinced that it's terribly justified :-)
50ms link protection probably isn't terribly justified either, but people strive, or even fight, for it. I wouldn't open that potential vector without some serious thought and consideration -- and as Rob T. points out, they offer multiple ways to implement it, or not at all. Up to the network operator -- some are risk averse (me, these days), some are not.
I've dealt with far too much pain when getting IP space in 219/8, 220/8, 222/8, etc, to ever want to implement a bogon filter myself. Of course, other operators that choose to blanket blackhole all APNIC space are another headache :\.
Indeed. I wouldn't recommend implementing bogon filters unless you do it really smartly, because as you say, more bad than good.
Yep. That remains my view, really.
The solution to a number of the "third party is scary" problems here is simply using BGP triggered blackholes to do this internally, and make sure you pay really really close attention to the mailing lists, or maybe rig up some thing so when Cymru change their announcements you get a notification or perhaps it drops it in to your table after a few hours of delay.
Again, good options. Probably not something I'll be doing, though. *I* do not see the value for the operational overhead and additional network-touch required. That's really all my original point in reply to Jonny's email was anyway. Do whatever you need with your network :). aj.
On 1/07/2007, at 3:17 AM, Alastair Johnson wrote:
Nathan Ward wrote:
On 1/07/2007, at 1:27 AM, Alastair Johnson wrote:
range of say /8 to /24, you have the POTENTIAL for mass injection of prefixes.
Why would you accept a range? I'd just accept the shortest prefix, if a longer prefix is de-bogoned it just means you don't count the still-bogon part as bogon until you update those filters. Think of it as a way to shorten bogon lists only, not modify them. Sure you don't get full coverage for a bit, but you certainly get more than just not filtering at all.
Fair point; but you're introducing additional touch requirements to your network devices, and you lose out on some usefulness of your bogon filtering, if say, 96/6 suddenly has one of those /8's de-bogoned.
Well, you only introduce as much touch as you'd have to in order to do manual bogon filtering, and you can limit the impact of that by doing the central box thing.
If you're prepared to have your bogon filter value deteriorate like that, then I would have to question their value at all.
We can call it nearly-bogon-filtering, which will /totally/ legitimise it.
Up to the network operator -- some are risk averse (me, these days), some are not.
You're just old and crabby ;-P
The solution to a number of the "third party is scary" problems here is simply using BGP triggered blackholes to do this internally, and make sure you pay really really close attention to the mailing lists, or maybe rig up some thing so when Cymru change their announcements you get a notification or perhaps it drops it in to your table after a few hours of delay.
Again, good options. Probably not something I'll be doing, though. *I* do not see the value for the operational overhead and additional network-touch required.
Yeah, I guess I don't care either way, but some data on the matter would be interesting. Rob, you're in a non-sleeping timezone (or you're up late like the rest of us), do you have anything? -- Nathan Ward
On 28-Jun-2007, at 19:54, Jonny Martin wrote:
Team Cymru provide a bogon BGP feed which make keeping up with bogon changes a cinch. 1 - Peer with the Cmyru bogon route server. 2 - Profit!
Taking a BGP feed of bogons is good if you are trying to avoid packets with bogon destinations following a default out towards a transit provider. They're no good for filtering long prefix bogons announced to you by peers, though, and if you accept those then traffic with bogon destinations can follow them regardless of any cymru-fed covering routes with null next-hops. If these statements are woefully out-of-date, where have I been, get with the programme, etc then someone should spell out precisely what fancy modern router policy knobs they are turning to get the desired result. While you're doing that I'll just nod wisely and mutter things about "yeah, I was just pretending not to know that stuff, because I was making a point. Yeah, that's it. You're all so lucky I'm here." And things of that nature. Joe
Joe Abley wrote:
They're no good for filtering long prefix bogons announced to you by peers, though, and if you accept those then traffic with bogon destinations can follow them regardless of any cymru-fed covering routes with null next-hops.
If these statements are woefully out-of-date, where have I been, get with the programme, etc then someone should spell out precisely what fancy modern router policy knobs they are turning to get the desired result. While you're doing that I'll just nod wisely and mutter things about "yeah, I was just pretending not to know that stuff, because I was making a point. Yeah, that's it. You're all so lucky I'm here." And things of that nature.
I can't think of a silver-bullet off the top of my head that would achieve this, but I'm sure you could achieve something through gratuitous policy and possibly RIB filters as well. That said, you can neatly use it to automagically uRPF-based blackhole for ingress/egress packet filtering. That helps a lot, but as you point out, if you had received a BGP route for a more specific bogon you may be in trouble. You could use the session to build your own filters/ACLs every time it changes, which is what some people do (and what the Cymru page suggests)... aj.
Gerard Creamer wrote:
Hi all,
Through our own stupidity we were inadvertently blocking a recent allocation from APNIC and thought I'd mention it here.
We had a bogon list with 112.0.0.0/5 in it and misread it as /8 several times before growing a brain and sorting it out. Just thought I'd mention it so folks can recheck their bogon lists to ensure that the following aren't being stopped by a larger aggregation.
116/8 APNIC 117/8 APNIC 118/8 APNIC 119/8 APNIC 120/8 APNIC
One other thing to watch with these allocations is that while we're supposed to live in a classless IP world the Cisco auto-summary command sees these blocks as /8 allocations and it's use may cause confusion to those used to using IP blocks starting with numbers > 192. (:-) At least one ISP peering with the Exchanges has been caught - no auto-summary could be your friend here.
Hi, team. Believe it or not, we agree with some of the concerns raised in this thread with regards to bogon filtering. This is one reason why we offer up so very many ways to filter bogons, including the afore mentioned bogon route-server project, as well as ways to keep bogons out of your RIBs (think uRPF cleanliness), bogons off your name servers, et al.: http://www.cymru.com/Bogons/ It's also why we've offered to turn over the filtering services to the RIRs should they wish to take on the task. We're also strong advocates of BCP38 and features such as uRPF. You'll see mention of them in our templates. :) We do a lot of outreach to folks in an attempt to keep filters, if applied, current. If anyone has any ideas on places where we should spread the word, either through mailing lists or at conferences, we're all ears! We hope to have a couple of team members in the AP region by the end of the year, and thus participate more frequently in regional conferences, present on best practices, security, what the miscreants are doing and why, etc. Thanks, Rob. -- Rob Thomas Team Cymru http://www.cymru.com/ cmn_err(do_panic, "Out of coffee!");
Rob Thomas wrote:
We do a lot of outreach to folks in an attempt to keep filters, if applied, current. If anyone has any ideas on places where we should spread the word, either through mailing lists or at conferences, we're all ears! We hope to have a couple of team members in the AP region by the end of the year, and thus participate more frequently in regional conferences, present on best practices, security, what the miscreants are doing and why, etc.
Fantastic! :) NZNOG 2008 is just around the corner, and I'm sure the program committee would love to hear from someone in the Cymru corner. aj.
NZNOG 2008 is just around the corner, and I'm sure the program committee would love to hear from someone in the Cymru corner.
Consider us very interested. :) -- Rob Thomas Team Cymru http://www.cymru.com/ cmn_err(do_panic, "Out of coffee!");
participants (10)
-
Alastair Johnson
-
Andy Linton
-
Dean Pemberton
-
Gerard Creamer
-
Joe Abley
-
Jonny Martin
-
Michael Fincham
-
Nathan Ward
-
Rob Thomas
-
TreeNet Admin