DNS/SMTP/Mailservers "doing it wrong"
Why can't people correctly configure Mail and DNS correctly these days. If all legitimate mail servers had their forward and reverse DNS mapping configured correctly I could simply use the postfix reject_unknown_reverse_client_hostname, and instantly get rid of 10s of 1000's of unwanted spam messages. However there are many servers set up in NZ where the forward and reverse DNS do not match, or there are no PTR records at all. After all this is outlined in RFC 1912 (section 2.1) and RFC 1033 Most of the big ISP mail servers are OK, but then they have customers running their own mailservers and this is where the problem lies. I have some numbers from tests over the weekend and it doesn't look good. I am being too hopeful to expect mailservers to be configured correctly? I'm stuck between wanting to do things properly and help reduce SPAM on one side, and having customers on the other saying why can't I get mail from XXX. While I can make an effort to try and contact the administrators of these mail servers the shear numbers make it difficult and even then i imagine the response would be 'well i can deliver mail to xxxx so it must be your systems that are broken'. I know this isn't a silver bullet by any means, but every little helps. Any comments? Phil
The two RFCs you quote are informational, and are not Internet standards. AFAIK, it is recommended but not required that mail servers' DNS A and PTR records match, and thus if you turn on such features you will be enforcing a requirement that does not exist in any Internet standard. Having said that, I do agree that where possible forward and reverse DNS should match. -j On Mon, 2008-04-07 at 11:09 +1200, Phil Snowdon wrote:
Why can't people correctly configure Mail and DNS correctly these days. If all legitimate mail servers had their forward and reverse DNS mapping configured correctly I could simply use the postfix reject_unknown_reverse_client_hostname, and instantly get rid of 10s of 1000's of unwanted spam messages.
However there are many servers set up in NZ where the forward and reverse DNS do not match, or there are no PTR records at all. After all this is outlined in RFC 1912 (section 2.1) and RFC 1033
Most of the big ISP mail servers are OK, but then they have customers running their own mailservers and this is where the problem lies. I have some numbers from tests over the weekend and it doesn't look good.
I am being too hopeful to expect mailservers to be configured correctly? I'm stuck between wanting to do things properly and help reduce SPAM on one side, and having customers on the other saying why can't I get mail from XXX.
While I can make an effort to try and contact the administrators of these mail servers the shear numbers make it difficult and even then i imagine the response would be 'well i can deliver mail to xxxx so it must be your systems that are broken'.
I know this isn't a silver bullet by any means, but every little helps. Any comments?
Phil
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog -- Jasper Bryant-Greene Network Engineer Unleash Technology Solutions
Reject the mail. It won't just be you they have problems sending mail to. However, I wouldn't get your hopes up about it stopping a lot of SPAM. -----Original Message----- ... However there are many servers set up in NZ where the forward and reverse DNS do not match, or there are no PTR records at all. After all this is outlined in RFC 1912 (section 2.1) and RFC 1033 ...
Phil Snowdon wrote:
I am being too hopeful to expect mailservers to be configured correctly?
Yes, but mostly because you have a slightly odd idea of what is "correct". The DNS does not require that forward and reverse zones be managed by the same organisation, and so you can get disjoint management policies. Like many people, I run an SMTP server at home. My cable provider manages the reverse lookup zone and the likelihood that they will let me have my personal domain name loaded into it is so close to zero that I have never asked. Since I manage my own forward domain, it says whatever I want. Cheers -- Lloyd Parkes Senior Systems Programmer Open Systems Ph: +64 4 890 2437
On Sun, Apr 6, 2008 at 4:23 PM, Lloyd Parkes
The DNS does not require that forward and reverse zones be managed by the same organisation, and so you can get disjoint management policies.
That shouldn't matter. Whoever runs the reverse DNS (most likely your ISP) can certainly make sure that they have a matching forward entry. It doesn't matter what other forward entries exist (ie, any that you may have setup for your own domain) - this will never be checked.
Like many people, I run an SMTP server at home. My cable provider manages the reverse lookup zone and the likelihood that they will let me have my personal domain name loaded into it is so close to zero that I have never asked. Since I manage my own forward domain, it says whatever I want.
$5 says that you would pass the check that Phil is suggesting. Take your IP address and do a reverse lookup on it. Then take what is returned and do a forward lookup. Do you get the original IP address? If so, Phil would accept your email. Your (additional) forward mapping from your own domain plays no part. Scott.
$5 says that you would pass the check that Phil is suggesting. Take your IP address and do a reverse lookup on it. Then take what is returned and do a forward lookup. Do you get the original IP address? If so, Phil would accept your email. Your (additional) forward mapping from your own domain plays no part.
Exactly. Much in the same way you do not need multiple PTR records for virtual hosting/email. As long as the IP resolves to a host and that forward matches back to the IP. That identifies the host to the world at large. I don't care what domains or other services you are hosting on that single address, you can have as many forward domains pointing to it as you want.
So why doesn't the forward and reverse match on the Spam emails? Sure the people who look after those ranges should be following the RFCs as well?
While I know this isn't going to solve any spam issue (hell i was getting spam from 'correctly' configured dynamic dsl ranges in brasil at the time) and it is informational, but it's more along the lines of letting the world know that a competent admin is in charge of that server and they're doing all they can to be be 'a good internet citizen' much like spf isn't going to stop spam, but it helps to identify whether mail is coming from the right place or not. just to be clear - we are not currently blocking _anyone_ with no reverse PTR or non-matching PTR/Forwards. It just would be nice to be able to reject up front in the SMTP conversation with a simple test rather than going through the full AV/Anti-spam weighting system, This was just my monday morning rant since nznog has been quiet for a while, but it's generated some interest. There's just such an issue with the volume of spam at the moment, that if we started making information stuff mandatory i think it would make things a lot easier to manage. Much like a having a driving test before you take a car on the pulic roads, maybe there should be some sort of test before you can put a host on the internet. Phil
On 2008-04-07 11:54, Phil Snowdon wrote: ...
So why doesn't the forward and reverse match on the Spam emails? Sure the people who look after those ranges should be following the RFCs as well?
Not if they are forging the mail headers, or using extremely short lived domains that only exist for the purpose of spamming.
While I know this isn't going to solve any spam issue (hell i was getting spam from 'correctly' configured dynamic dsl ranges in brasil at the time) and it is informational, but it's more along the lines of letting the world know that a competent admin is in charge of that server and they're doing all they can to be be 'a good internet citizen'
Exactly. Using this to unconditionally reject mail is not good practice, since it isn't standard anyway. Using it to raise the spam score seems reasonable.
much like spf isn't going to stop spam, but it helps to identify whether mail is coming from the right place or not.
SPF is pretty much broken; DKIM is much more likely to be meaningful. But dropping mail (as opposed to marking it as suspect) with any of these techniques carries a real risk of false positives (just like pretty much any black list mechanism). Content filtering has a much better chance.
just to be clear - we are not currently blocking _anyone_ with no reverse PTR or non-matching PTR/Forwards. It just would be nice to be able to reject up front in the SMTP conversation with a simple test rather than going through the full AV/Anti-spam weighting system,
Sorry, but I don't see any way round that. Brian
This was just my monday morning rant since nznog has been quiet for a while, but it's generated some interest.
There's just such an issue with the volume of spam at the moment, that if we started making information stuff mandatory i think it would make things a lot easier to manage. Much like a having a driving test before you take a car on the pulic roads, maybe there should be some sort of test before you can put a host on the internet.
Phil
_______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
On Mon, Apr 7, 2008 at 12:26 PM, Brian E Carpenter
On 2008-04-07 11:54, Phil Snowdon wrote:
just to be clear - we are not currently blocking _anyone_ with no reverse PTR or non-matching PTR/Forwards. It just would be nice to be able to reject up front in the SMTP conversation with a simple test rather than going through the full AV/Anti-spam weighting system,
Trivial greylisting does that ... no need to worry about who or what is sending you spam, just distrust everyone equally. The downsides are the training period, where everything is initially rejected, and the unpredictable/uncontrollable backoff period that all emails will suffer when they trip the greylister -- especially when you sign up for some new forum (that doesn't take OpenID) and the sign-up email you need doesn't get through quickly enough ... The other downside is the sort of attitude that pushes your spam problem back off to the legitimate servers ... but then again, that's what the 400 series is for, for a suitably generous definition of "error" ... 420 feels about right. It also kicks back about 80% of all incoming connections ... some of those can be "legitimate" (i.e. non-spam, but sent from unexpectedly oddly configured servers). Them's the breaks. -jim
On Sun, Apr 6, 2008 at 4:09 PM, Phil Snowdon
However there are many servers set up in NZ where the forward and reverse DNS do not match, or there are no PTR records at all. After all this is outlined in RFC 1912 (section 2.1) and RFC 1033
These are both informational RFCs, so there's no requirement to implement what they say (as nice as it would be...) I am being too hopeful to expect mailservers to be configured
correctly? I'm stuck between wanting to do things properly and help reduce SPAM on one side, and having customers on the other saying why can't I get mail from XXX.
iiNet made the change to blocking all servers without reverse DNS some time ago (they don't enforce that it must match forward - just that it exists). It caused some paid, but not a huge amount. The forums on whirlpool.net.auwill give some idea of the problems they had. I'm not sure how much it actually bought them - to a large extent you're just blocking what RBL's/reputation services would have stopped in the first place anyway. Enforcing forward/reverse has at least one fairly major downfall - if someone ever tries to change the name of a server/IP address, the different caching/propagation between the forward and reverse names will most likely mean that you'll end up blocking mail from them for a period of time. This happened about 6 months ago for a major ISP somewhere (unfortunately I can't remember exactly who). Yes, there are ways to do this right, but are you going to trust every ISP/corporate admin to get it right? Scott.
On Mon, 7 Apr 2008, Phil Snowdon wrote:
Why can't people correctly configure Mail and DNS correctly these days. If all legitimate mail servers had their forward and reverse DNS mapping configured correctly I could simply use the postfix reject_unknown_reverse_client_hostname, and instantly get rid of 10s of 1000's of unwanted spam messages.
So why doesn't the forward and reverse match on the Spam emails? Sure the people who look after those ranges should be following the RFCs as well? It sounds like you are picking up with something that is an indicator of spam but not an absolute marker ( Like emails with Chinese characters going to my home domain) . In which case you should be carefully running the stats to assign a score that maximises your false negatives and and minimises your false positives with the relative weighting of each depending on your site population. But if you are serious about generating false positives why don't you insist the HELO address matches as well.. -- Simon J. Lyall | Very Busy | Web: http://www.darkmere.gen.nz/ "To stay awake all night adds a day to your life" - Stilgar | eMT.
On 6 Apr 2008, at 18:09 , Phil Snowdon wrote:
However there are many servers set up in NZ where the forward and reverse DNS do not match, or there are no PTR records at all. After all this is outlined in RFC 1912 (section 2.1) and RFC 1033
See also draft-ietf-dnsop-reverse-mapping-considerations-06, which google knows how to find and which is currently in working group last- call (I think). The general consensus, promulgated in that document, is that it's good for people to take time to set up reverse DNS, because having correct reverse DNS is helpful. However, it's bad for people to require reverse DNS to be correct, because more often than not, it's not. Joe
participants (9)
-
Brian E Carpenter -
Jasper Bryant-Greene -
Jim Cheetham -
Joe Abley -
Lloyd Parkes -
Phil Snowdon -
Philip D'Ath -
Scott Howard -
Simon Lyall