Greylisting New Zealand whitelist
Andy wrote:
You can see the routes advertised at the APE and WIX on the pages: but it does provide a couple of lists that by some definition are in NZ.
I don't know what "some definition" is but by most definitions there are quite a few non-NZ IPs there ( Specificly things like 203.59.0.0/17 that ihug advertising on behalf of iinet). Anyway, you don't want random IPs home luser IPs for greylisting, you want IPs that are sending you non-spam email. -- Simon J. Lyall | Very Busy | Web: http://www.darkmere.gen.nz/ "To stay awake all night adds a day to your life" - Stilgar | eMT.
On Tue, 9 Jan 2007, Simon Lyall wrote:
Andy wrote:
You can see the routes advertised at the APE and WIX on the pages: but it does provide a couple of lists that by some definition are in NZ.
I don't know what "some definition" is but by most definitions there are quite a few non-NZ IPs there ( Specificly things like 203.59.0.0/17 that ihug advertising on behalf of iinet).
Anyway, you don't want random IPs home luser IPs for greylisting, you want IPs that are sending you non-spam email.
For private use, I found the easiest thing to do was to watch my mail logs for a couple of days and then create appropriate regex. Much of my mail is mailing list stuff, so a dozen or so rules took care of most of it. For enterprise level use, I see value in a consolidated list of regex expressions that'll cover the MTAs of Major ISPs - perhaps only as a starting point. For example i've noticed that Ihug's SMTP seems to be mainly care of alias[x].ihug.co.nz so I have the following whitelist rule: /^alias\d+.ihug.co.nz$/ Similarly Xtras seem to fit into the following patterns: /^fep\d+.xtra.co.nz$/ /^mta\d+.xtra.co.nz$/ (I've also got some in there for Google/Gmail, Yahoogroups, LUGs, NZNOG, etc...) I suspect whats looked for is a 'starting point' that covers the NZ ISP community, and I too would be interested. If no-one's done it yet, perhaps some coordination amongst those deploying Greylisting would be smart. The idea being not to get too granular too quickly, though, as that might reduce the effectiveness of it. (You'd have to trust the entries before you used em, instead of importing blindly.) Mark.
On Tue, 9 Jan 2007, Nathan Ward wrote:
Mark Foster wrote:
/^alias\d+.ihug.co.nz$/ /^fep\d+.xtra.co.nz$/ /^mta\d+.xtra.co.nz$/
So uh, do you check that the forward matches the reverse?
Yup, I believe i do. Failure to match forward and reverse puts the score up quite substantially. Theres nothing stopping me from replacing hostname based stuff with IP based stuff - but if someone decides to go and forge their RDNS just to get around my greylisting, well, i'll be flattered...
While this idea is not totally without merit - the vast bulk of MX servers operated in NZ are owned by corporates, not ISPs. So therefore I think that the database would be incomplete at best. Is there any precedent / case studies for this concept implemented anywhere else in the world?
John @ netTRUST wrote:
While this idea is not totally without merit - the vast bulk of MX servers operated in NZ are owned by corporates, not ISPs.
So therefore I think that the database would be incomplete at best.
Is there any precedent / case studies for this concept implemented anywhere else in the world?
Perhaps the bulk of servers (of course, I'm not aware of any stats to back up your claim, but it sounds like a valid assumption) but where does the bulk of mail originate from? Also, is there zero spam from corporate servers? They get hacked/run as open relays/etc. a lot more than ISP servers (assumption based on your assumption of there being more corporate servers, and my additional assumption that ISP staff know how to secure servers a bit better than the accountant who figured out how to get Pegasus up and going). On that note, is there zero spam from ISP servers? I'd suggest not. -- Nathan Ward
On that note, is there zero spam from ISP servers? I'd suggest not.
Greylisting is meant to cause spam sent from non-MTAs (i.e. drones) to get deferred; they don't tend to have mechanisms to deal with 400 series deferrals. It won't do much at all for spam delivered via an Actual MTA (Corporate, ISP or otherwise) except delay it a little.
On that note, is there zero spam from ISP servers? I'd suggest not.
Greylisting is meant to cause spam sent from non-MTAs (i.e. drones) to get deferred; they don't tend to have mechanisms to deal with 400 series deferrals.
spamhaus has just introduced zen. and people using sbl-xbl.spamhaus.org for RBL's might want to change to zen.spamhaus.org instead See: http://www.spamhaus.org/zen/ Zen adds the ability for ISP's to add to the PBL to list ranges which shouldn't send email out or for example dialup ranges which have a policy of no servers (ie email). I've used this for a short while now with no complaints but its reduced the amount of spam from bots quite a bit with the inclusion of the PBL in compared to the old sbl-xbl Thanks Craig http://www.spam.co.nz
About a couple of months ago I wrote a note to this list describing our experiences with grey listing. I'll briefly summarise it again. We did not start with a white list instead we selected those addresses that we would grey list. Postfix allows series of rules: 1/ if no reverse dns then greylist 2/ if reverse dns matches a large Regular expression designed to catch most dynamic, broadband etc. then greylist 3/ pass for normal handling This worked very well for us cutting the amount of spam we accepted to well under half. But remember that grey listing is merely a stop gap measure as soon as spammers think that greylising is really hurting them they will rework their spam engines to handle retries -- it won't be that difficult, particularly since they don't have to queue whole messages, just a message id and 'sender' and recipient. Russell.
Russell Fulton wrote:
But remember that grey listing is merely a stop gap measure as soon as spammers think that greylising is really hurting them they will rework their spam engines to handle retries -- it won't be that difficult, particularly since they don't have to queue whole messages, just a message id and 'sender' and recipient.
I think it's generally accepted that most (all?) anti-spam "solutions" are stop gap in some way, shape or form. -- Nathan Ward
Nathan Ward wrote:
Russell Fulton wrote:
But remember that grey listing is merely a stop gap measure as soon as spammers think that greylising is really hurting them they will rework their spam engines to handle retries -- it won't be that difficult, particularly since they don't have to queue whole messages, just a message id and 'sender' and recipient.
I think it's generally accepted that most (all?) anti-spam "solutions" are stop gap in some way, shape or form.
Definitely! What worries me particularly about grey listing is that, at the moment, it is *very* successful and hugely reduces the load on our servers. I have warned our management that they have to budget for more hardware for the mail system because if grey listing becomes ineffective then our current system could go under really quickly! We may not have much notice -- look at who got caught short by last years' rise in September ;) We survived that by implementing grey listing but not before one of our 4 MTAs was showing considerable signs of stress. What management really don't like is that we can not predict when it will happen nor how much notice we will get. My guess is that 80% of spam is generated by a handful of big players who use a small number of software packages to deliver their spam. I also guess that once one decides to start retrying then they will all follow suit in short order. We could see grey listing become largely ineffective in the space of a month or so. Even if we ordered the new hardware as soon as we thought there might be a problem we could not get it installed and set up before the existing system melted down. In actual practice we would probably steal time on some less loaded VM farms to run extra MTAs while we waited for new hardware to arrive but this may not be possible at some times of the year. Russell
On Tue, 9 Jan 2007, Russell Fulton wrote:
But remember that grey listing is merely a stop gap measure as soon as spammers think that greylising is really hurting them they will rework their spam engines to handle retries
That is true, but once they start doing that they'll have to hang around for *much* longer. That improves the odds of getting a human to respond in time to make an effective difference, and improves the cacheability of DNS-BL into the bargain (since then we won't have to keep the TTLs so low). So even when the spammers catch on, it'll still have some benefit. -Martin
On Tue, Jan 09, 2007 at 02:17:46PM +1300, Nathan Ward wrote:
John @ netTRUST wrote:
While this idea is not totally without merit - the vast bulk of MX servers operated in NZ are owned by corporates, not ISPs.
So therefore I think that the database would be incomplete at best.
Is there any precedent / case studies for this concept implemented anywhere else in the world?
Perhaps the bulk of servers (of course, I'm not aware of any stats to back up your claim, but it sounds like a valid assumption) but where does the bulk of mail originate from?
I have had great success with simply 'nz' in postgrey_whitelist_clients.local, this assumes that the percentage of spam is insignificant from *.nz mailservers and seems to be true (for the time being). Andrew
I would be happy to give time to start and moderate a list, looking at automating requests for inclusion and automating data extraction in various formats with different criteria for different greylisting implementations. Normal provisos - the list may wreck your mail, kill your cat, burn down your house, or hundreds of other nasty things, none of which should be blamed on me. And if you don't trust me don't use my list. Gerard On 9/01/2007 1:33 p.m., Mark Foster wrote:
On Tue, 9 Jan 2007, Simon Lyall wrote:
Andy wrote:
You can see the routes advertised at the APE and WIX on the pages: but it does provide a couple of lists that by some definition are in NZ.
I don't know what "some definition" is but by most definitions there are quite a few non-NZ IPs there ( Specificly things like 203.59.0.0/17 that ihug advertising on behalf of iinet).
Anyway, you don't want random IPs home luser IPs for greylisting, you want IPs that are sending you non-spam email.
For private use, I found the easiest thing to do was to watch my mail logs for a couple of days and then create appropriate regex. Much of my mail is mailing list stuff, so a dozen or so rules took care of most of it.
For enterprise level use, I see value in a consolidated list of regex expressions that'll cover the MTAs of Major ISPs - perhaps only as a starting point.
For example i've noticed that Ihug's SMTP seems to be mainly care of alias[x].ihug.co.nz so I have the following whitelist rule:
/^alias\d+.ihug.co.nz$/
Similarly Xtras seem to fit into the following patterns:
/^fep\d+.xtra.co.nz$/ /^mta\d+.xtra.co.nz$/
(I've also got some in there for Google/Gmail, Yahoogroups, LUGs, NZNOG, etc...)
I suspect whats looked for is a 'starting point' that covers the NZ ISP community, and I too would be interested.
If no-one's done it yet, perhaps some coordination amongst those deploying Greylisting would be smart. The idea being not to get too granular too quickly, though, as that might reduce the effectiveness of it. (You'd have to trust the entries before you used em, instead of importing blindly.)
Mark.
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
-- Netspace Services Limited http://www.netspace.net.nz Phone +64 4 917 8098 Mobile +64 21 246 2266 Level One, 220 Thorndon Quay, Thorndon PO Box 12-082, Thorndon, Wellington 6004, New Zealand
participants (9)
-
Andrew Thompson -
Craig Whitmore -
Gerard Creamer -
John ＠ netTRUST -
Mark Foster -
Martin D Kealey -
Nathan Ward -
Russell Fulton -
Simon Lyall