I have privately implimented exactly what youre suggesting on my personal MTA. My rejection is actioned via an iptables script, and when I receive spam I tend to block at the /24 level at the minimum - manually now, unfortunately, with the demise of most of the RBLs.... Its all context driven, though.. Spam from Asian networks often winds up being blocked at the network level - eg whatever I can pull from whois, I block. (/14 or bigger in some cases). I havn't blocked anything at the /8 except for 200.* which finally frustrated the hell out of me one day... The catch is that I have other people who use my mail server, so i've got to make sure i keep them in mind when i put blocks in place. The system I use is very rough but when people agree to use my MTA they're made aware that the call in the end will be mine. In one case theyve provisioned a secondary MX which doesn't have the restrictions, and is not restricted by me.. The idea has merit - I reccomend that people who can admin their own mail services do so - but unfortunately its not something that I would personally ever reccomend to those people who are not clooful enough to manage it. That should then become the ISPs responsibility but its always the difference between - 'trying to hard' and 'not trying hard enough'. How much is too much? Does the admin of said machine have to then manually block networks? Id rather see the networks in question blocked at ISP border routers personally but I guess that wont happen in the short term. (This is a WAN, not a LAN.. sigh) Mark. On Fri, 26 Sep 2003, Steve Withers wrote:

Further comments on IP and domain blocking for *personal* mail servers: Just checked my maillog from yesterday.

70% of rejected mail connects came from hotmail, yahoo, earthlink and aol.

30% came from the 61.* and 218.* Korean IP spaces

10% was rejected by ordb / relay denied / other blocked domains

I have wondered if ISPs want to encourage customers to set up individually customisable mailservers on broadband connections - some sort of appliance - that acts as their mail server.

Let the business and competent private users decide what they will and won't receive....with benefits to the ISP in terms of reduced bandwidth consumed as spam isn't deliverable to these people. Just lots of rejected connect attempts. This may even be a managed service an ISP could offer a customer / business. If payment is on data-volume, this could help reduce such charges - offsetting any service fee to some extent.

Am I right in thinking Mailmarshall still allows the spam to be delivered? It just filters it.

The method above prevents delivery.

It would be impossible to do this at ISP level....but it may be a service line an ISP might like to offer a client who wants to define what they do and do not receive.

-- Steve Withers

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

$author = "Craig Spiers" ;

Yea, I don't think blocking ranges as large as /14 /8 and so on is the answer here.. Imagine how many legitimate mailservers your blocking in that ip range, and for what reason? Just because 1 machine, on 1 IP address.. Was spamming you.. Isnt it a lot easier to just go.. Ahh shit spam.. Delete that..

You reserve that level of blocking for situations where it's more then 1 incident, something proportional to the size of the space your contemplating dropping and the level of response from abuse(a). I had no qualms blocking entire IP space of particular asian ISPs when the problem got out of hand and didn't look like resolving any time soon. marty -- Take these tears Wash your skin I'm havin' trouble breathin' Since you walked in "Million Tears" - Kasey Chambers

Juha wrote:

The problem with much spam is that while you can decide to drop .cn, .kr, and .ng, plus 200/8, much of it arrives via seemingly legit sources.

Unless, like me, you need to receive legitimate mail from .cn, .kr and .ng.... :-( Keith Davidson

On Fri, 2003-09-26 at 11:12, Juha Saarinen wrote:

Basically, neither DNS blacklisting nor filtering work well enough currently. And no, challenge-and-response systems aren't the answer either.

Nothing is perfect, I agree. However, it is possible for a given individual or small group to reduce the amount of spam being received by over 90%....and that is mail simply not delivered at all.....no further filtering required. Whether this is good enough or not depends on expectations. -- Steve Withers

On Fri, 2003-09-26 at 11:23, Simon Byrnand wrote:

At 10:38 26/09/2003 +1200, Steve Withers wrote:

...

Further comments on IP and domain blocking for *personal* mail servers: Just checked my maillog from yesterday.

70% of rejected mail connects came from hotmail, yahoo, earthlink and aol.

Umm, are you sure about that ?

You're right. Wherever it comes from, by blocking these domains I do not receive this mail. :-)

If you look at the message headers you'll find that nearly all of that (well for hotmail and yahoo anyway) is just spammers forging hotmail and yahoo addresses, the messages wont actually be passing through hotmail and yahoo servers...

True......I should have made that distinction. I simply block these domains.

Anyone in the world can send an email through any server in the world and make the From and reply addresses seem to be a hotmail address....

True... Either way - and from whatever source - I get rid of 70% of the spam directed at me by blocking these domains. (Based on the logs from yesterday - admittedly a small sample)

At 16:33 26/09/2003 +1200, PhoneNet wrote:

At 11:23 26/09/2003 +1200, you wrote:

...

At 10:38 26/09/2003 +1200, Steve Withers wrote:

...

Further comments on IP and domain blocking for *personal* mail servers: Just checked my maillog from yesterday.

70% of rejected mail connects came from hotmail, yahoo, earthlink and aol.

Umm, are you sure about that ?

If you look at the message headers you'll find that nearly all of that (well for hotmail and yahoo anyway) is just spammers forging hotmail and yahoo addresses, the messages wont actually be passing through hotmail and yahoo servers...

Anyone in the world can send an email through any server in the world and make the From and reply addresses seem to be a hotmail address....

or any address or so it would seem going by the sheer volume of bounced email I'm receiving on my phonenet (at) xtra.co.nz account (an account that doesn't post email or newsgroup messages but is there as a leftover from my former business) - I'm assuming they harvested the address somehow from the old websites.... and I'm not smart enough to know what to do to stop this spam going out under ficticious names but with my email address showing as the sender and reply-to :-(

The short answer is that there is nothing you can do. If someone out there in internet land somewhere wants to send an email with your address as the "From" address, they can do so. Unless the recipients are clueful enough to check the message headers they're likely to fall for it too... I get quite a few bounces from spam that has supposedly been sent "from" some of our contact addresses which we *never* send email from, I just take it in stride and it just fuels my determination to fight spammers even more :) Fortunately we havn't had any complaints for quite some time about spam comming "from" us, so maybe people are now generally aware that the "From" address of Spam is usually bogus... Regards, Simon

On Fri, 26 Sep 2003, Brian Gibbons wrote:

A human being can scan down 20 emails in their Inbox and immediately descriminate between Spam and valid email because they have an educated eye and brain.

Now that gives me an idea... Manual Spam Filtering. The new cottage industry. Earn Money Reading Email! "I didn't want to believe it at first, but I made $5,000 in just two weeks on the Manual Spam Filtering Programme. It's great!" -- B. Gibbons, Auckland NZ -- Juha Saarinen

At 12:27 26/09/2003 +1200, Mark Foster wrote:

On Fri, 26 Sep 2003, Brian Gibbons wrote:

...

...

From: "Simon Byrnand"

...

[half hearted effort to block spam] (And in that category I include manual blocking of huge swarths of ip space, outright blocking based on most RBL blacklists, Mailmarshall, and Challenge response systems, all of which have unacceptable collateral damage

Well said.

...

I recently dealt with an ISP in the US who had blocked 202.0.0.0/8 (!) because "all we ever get from that netblock is spam from China".

And there is the issue.

If ISPs use network blocking as a mechanism to block Spam then the ultimate outcome will be be a block on all networks and zero email delivery.

A human being can scan down 20 emails in their Inbox and immediately descriminate between Spam and valid email because they have an educated eye and brain.

My comment on this is simply that I do not block at the /8 - I use whois, and DNS, and calculate exactly how wide a block I can put in without blocking someone *elses* network.. and I do that. If I cant do it by network then I do it by /32, starting with the offending MTA.

I don't agree with blocks such as 202/8 (been the victim of one of those) but I think educated, selective blocking is quite acceptable - at least untill those networks involved actually do something about the whole 'spam' thing. What amazes me is the number of people out there who still thing opt-out is acceptable..

That approach (and point of view to the problem) is one that a lot of people hold, (including Spews, albeit more militant) but it doesn't address the basic issue of collateral damage. If you as an individual decide to block ranges like that, so be it, however a large entitity like an ISP or institution can't do this without the risk of collateral damage. At the end of the day *WHY* should someone trying to send a legitimate message have their message rejected because someone else that happens to use the same ISP is either spamming or has an insecure machine which is being exploited to send spam. Hence my comments about how each message (when processed on an ISP scale) *must* be treated on its own merits. Don't tar everyone with the same brush. Before you say "they should just move to another ISP", in some parts of the world there AREN'T any alternatives to a given ISP. Say you're on ADSL with the only ISP in your area that provides it (quite common in some areas of Europe) and your ISP has other customers whose machines keep getting exploited to relay spam, what are you going to do when you can't send your email because your ISP is blacklisted ? Move to another ISP and go down to a dialup connection ? To give an analogy imagine you live on the same street as a car conversion racket, and every time the police get a tipoff they come and raid EVERY house on the street. When you complain that your house keeps getting raided by the police for no reason they say "well you live on the same street as them and you're not doing anything about stopping them, so tough". How rediculous. Do they think that if they raid everyones houses enough times all the neighbours will finally get so fed up that they'll go and beat up the crooks themselves ? Or do they expect that people that get sick of being raided all the time will move house to another street ? :) Might sound like a silly analogy, but this is *exactly* whats happening to the innocent bystanders in the "war against spam"....IMHO people implementing spam filtering, at least on any scale, should be doing their utmost to minimize collateral damage, and not take a "well if we blacklist this whole ISP maybe they'll do something about their spammers" approach... Private individuals that run their own mail servers for themselves and/or a small group of friends and family are free to block whatever they please of course :) Regards, Simon

On Friday, Sep 26, 2003, at 09:22 Canada/Eastern, DPF wrote:

On Fri, 26 Sep 2003 12:03:04 +1200, "Brian Gibbons" wrote:

...

...

From: "Simon Byrnand"

...

[half hearted effort to block spam] (And in that category I include manual blocking of huge swarths of ip space, outright blocking based on most RBL blacklists, Mailmarshall, and Challenge response systems, all of which have unacceptable collateral damage

Well said.

People may be interested that NZ marketing companies (as in operating 100% opt in e-mail lists) have advised that around 20% of their e-mails are getting blocked by anti spam type technologies (esp Mail Marshall) which is actually quite shocking that such a high percentage of e-mails that people want to receive are being blocked.

People are persuaded to opt-in to things in all kinds of tricky ways. I think a better interpretation is that these allegedly opt-in companies are sending mail which people demonstrably don't want to receive. (Nobody would endure a spam filtering service which had a 20% false positive rate, so the only natural conclusion to draw is that the opt-in messages which are blocked aren't considered false positives by the subscribers to those spam filtering services). Joe

On Sat, 2003-09-27 at 14:12, DPF wrote:

...

(Nobody would endure a spam filtering service which had a 20% false positive rate, so the only natural conclusion to draw is that the opt-in messages which are blocked aren't considered false positives by the subscribers to those spam filtering services).

You would be right if people had a choice of subscribing but many people are forced to accept whatever their employer puts in place.

He who pays the piper calls the tune? Surely any business operating an email system does so for the benefit of the company/enterprise etc. If individuals want to receive emails that don't match that policy then they need to have a personal account that allows them to do so. Which brings us back to advice offered for example to people in Frank's position that they get the mail from this list directed to a hotmail etc account or run their own server where they can set policy for themselves.

From: "Hamish MacEwan" So the tune is the one that suits the enterprise's policy, but in many cases, Frank's one amongst them, the diversity of even the most narrowly focussed enterprise is beyond the wit of any filter.

I disagree.

Indeed, if we could train a filter to recognise spam "better than a human," the long search for AI will be over.

I am sent somewhere between 150 - 300 Spam messages daily. A few months ago my daily drudge was delete - delete mark,bound delete. Unfortunately I would fail to recognise one or two legitimate emails and delete them - this was a serious problem. I spent four months developing and training a Spam filter. Now I get one Spam every day or two, and no false positives.

From my point of view the filter is better than I am at discriminating between Spam and valid email. This is not AI, it is just that humans are not very good at boring repetitive tasks.

Cheers BG.

On Sat, 2003-09-27 at 17:39, Juha Saarinen wrote:

On Sat, 27 Sep 2003, Andy Linton wrote:

...

He who pays the piper calls the tune?

In Frank's case, that'd be the taxpayer, who as per usual has precious little say...

The same as any shareholder in a medium/largish private firm does about daily, operational matters - none. That is what management is for. :-) It's a shame that the most popular spam-handling programs still allow the stuff to be received at all.....what a waste of bandwidth! -- Steve Withers

On Sat, 2003-09-27 at 16:16, Andy Linton wrote:

Surely any business operating an email system does so for the benefit of the company/enterprise etc. If individuals want to receive emails that don't match that policy then they need to have a personal account that allows them to do so. Which brings us back to advice offered for example to people in Frank's position that they get the mail from this list directed to a hotmail etc account or run their own server where they can set policy for themselves.

However, some employers also have policies that forbid access to alternate mail sources (hotmail & ISP accounts) for legitimate reasons. This leaves some one like Frank having to follow work related lists that run foul of the corporate filter in their own time from home. I think that the real answer is that Corporate policies have to be flexible enough to work around such problems. This is very obvious in an academic environment. We tag spam and leave it up to the users to decide what to do about it, some have filters that simply delete all tagged messages, some (like me) get all tagged messages dumped in a folder which a check a couple of times a day (this takes under 30 seconds normally), other don't do anything special and delete them by hand. I believe that this is essentially a management issue and has nothing to do with technology.

_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog

-- Russell Fulton, Network Security Officer, The University of Auckland, New Zealand.

Thats not nice (It is arrogant). I am sure that he has bills to pay (And family to support?), like almost everyone else. Finding a "new job" is not always easy or fair or as simple as saying "He has an option of finding a new job". How do you know this? Michael Hallager Networkstuff Limited

He also has the option of finding a new job, if the policies of his employer don't suit him.

-----Original Message----- From: DPF [mailto:david(a)farrar.com] Sent: Saturday, 27 September 2003 2:12 p.m. To: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] Argghhh.... [Fwd: Your e-mail message was blocked]

On Fri, 26 Sep 2003 10:47:55 -0400, Joe Abley wrote:

...

On Friday, Sep 26, 2003, at 09:22 Canada/Eastern, DPF wrote:

...

On Fri, 26 Sep 2003 12:03:04 +1200, "Brian Gibbons" wrote:

...

...

From: "Simon Byrnand"

...

[half hearted effort to block spam] (And in that category I include manual blocking of huge swarths of ip space, outright blocking based on most RBL blacklists, Mailmarshall, and Challenge response systems, all of which have unacceptable collateral damage

Well said.

People may be interested that NZ marketing companies (as in operating 100% opt in e-mail lists) have advised that around 20% of their e-mails are getting blocked by anti spam type technologies (esp Mail Marshall) which is actually quite shocking that such a high percentage of e-mails that people want to receive are being blocked.

People are persuaded to opt-in to things in all kinds of tricky ways.

I think a better interpretation is that these allegedly opt-in companies are sending mail which people demonstrably don't want to receive.

I think making assumptions without any evidence is very dangerous. I know many people who get e-mail blocked they want to receive but as we just heard from Frank at MED, they are unable to change company policies.

...

(Nobody would endure a spam filtering service which had a 20% false positive rate, so the only natural conclusion to draw is that the opt-in messages which are blocked aren't considered false positives by the subscribers to those spam filtering services).

You would be right if people had a choice of subscribing but many people are forced to accept whatever their employer puts in place.

Also what we are talking about is a 20% false positive from marketing companies. Not a General 20% false +ve rate. These types of emails tend to have subjects and content that triggers spam filters. I know many people that have had false positives from advertising and marketing companies. Even when the email was not a mass distribution, sometimes just the hyperbole in a requirements email is enough to make you vomit let alone trigger spam content based spam filtering. Chris