How common is this scenario? It's such a pain in the ass and I'm getting sick of dealing with it.....
OrgA emails OrgB.
Email goes through MailHostingCompany's SMTP server.
OrgB rejects OrgA email because SPF disallows HosterA's smtp server from sending email on behalf of OrgA.
In the headers below, 210.48.108.215 appears to be an SMTP server at grunge.hosts.net.nz
===================================================
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed:
admin(a)safegas.co.nz
SMTP error from remote mail server after RCPT TO:
Why is OrgA's SPF record broken? Are you saying that OrgA needs to add SPF records to allow every ISP and Hosting company to relay email on its behalf? -----Original Message----- From: Jasper Bryant-Greene [mailto:jasper(a)metaname.co.nz] Sent: Thursday, 22 July 2010 3:46 p.m. To: Regan Murphy Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection On 22/07/2010, at 3:39 PM, Regan Murphy wrote:
So OrgA's SPF record is misconfigured (doesn't accurately describe how they send mail). Shouldn't be that much of a pain to fix? Certainly one of the less hairy mail problems out there... -jasper
Every ISP and Hosting company shouldn't be relaying email for OrgA. I say fix the SPF DNS record properly if you want to use it. -----Original Message----- From: nznog-bounces(a)list.waikato.ac.nz [mailto:nznog-bounces(a)list.waikato.ac.nz] On Behalf Of Regan Murphy Sent: Thursday, 22 July 2010 3:48 p.m. To: Jasper Bryant-Greene Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection Why is OrgA's SPF record broken? Are you saying that OrgA needs to add SPF records to allow every ISP and Hosting company to relay email on its behalf? -----Original Message----- From: Jasper Bryant-Greene [mailto:jasper(a)metaname.co.nz] Sent: Thursday, 22 July 2010 3:46 p.m. To: Regan Murphy Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection On 22/07/2010, at 3:39 PM, Regan Murphy wrote:
So OrgA's SPF record is misconfigured (doesn't accurately describe how they send mail). Shouldn't be that much of a pain to fix? Certainly one of the less hairy mail problems out there... -jasper _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
On 22/07/2010, at 3:47 PM, Regan Murphy wrote:
Why is OrgA's SPF record broken?
Are you saying that OrgA needs to add SPF records to allow every ISP and Hosting company to relay email on its behalf?
No, only the hosts which send mail on OrgA’s behalf. This is usually only a handful of hosts at most. In this case, it’s probably just their server (if they send any mail directly), and/or their hosting company’s mail server (if they use it as a so-called “smarthost”). http://en.wikipedia.org/wiki/Sender_Policy_Framework is quite comprehensive. -jasper
SPF of "v=spf1 a mx ptr -all" is supposed to mean that all MX hosts for a domain are the only authorised senders of mail for that domain, correct? And the IP must match the PTR record for that host. oasystems.co.nz: witch.oasystems.co.nz is the MX 10 for oasystems.co.nz witch.oasystems.co.nz resolves to 202.180.74.56 (A) 202.180.74.56 resolves to witch.oasystems.co.nz (PTR) Where is this SPF record broken? This line from the rejection: "210.48.108.215 is not allowed to send mail from oasystems.co.nz" is saying the host grunge.hosts.net.nz is not allowed to send email on behalf of oasystems.co.nz which is correct, it's not. -- Regan -----Original Message----- From: Jasper Bryant-Greene [mailto:jasper(a)metaname.co.nz] Sent: Thursday, 22 July 2010 3:54 p.m. To: Regan Murphy Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection On 22/07/2010, at 3:47 PM, Regan Murphy wrote:
Why is OrgA's SPF record broken?
Are you saying that OrgA needs to add SPF records to allow every ISP and Hosting company to relay email on its behalf?
No, only the hosts which send mail on OrgA's behalf. This is usually only a handful of hosts at most. In this case, it's probably just their server (if they send any mail directly), and/or their hosting company's mail server (if they use it as a so-called "smarthost"). http://en.wikipedia.org/wiki/Sender_Policy_Framework is quite comprehensive. -jasper
On 22/07/2010, at 4:04 PM, Regan Murphy wrote:
This line from the rejection: "210.48.108.215 is not allowed to send mail from oasystems.co.nz" is saying the host grunge.hosts.net.nz is not allowed to send email on behalf of oasystems.co.nz which is correct, it's not.
So SPF is behaving as intended: a host is sending mail on behalf of a domain which is not listed in that domain’s SPF record, so the mail is being rejected. To change this, you could either update the SPF record to include that host, or stop that host from sending mail for the domain. Although many SPF failures are due to sender misconfiguration, this can also happen due to a misconfiguration on the receiver’s part. Looking at your original message again, the headers would suggest that this might be the case here; the receiver might not be checking SPF at the border, but rather one or more mail hops further inside their network. See the first item at http://www.openspf.org/FAQ/Common_receiver_mistakes -jasper
Although many SPF failures are due to sender misconfiguration, this can also happen due to a misconfiguration on the receiver's part. Looking at your original message again, the headers would suggest that this might be the case here; the receiver might not be checking SPF at the border, but rather one or more mail hops further inside their network. See the first item at http://www.openspf.org/FAQ/Common_receiver_mistakes
This is the scenario I am seeing. On reflection my summarized version of events in the original email was poorly constructed... Anyway, I'm running into this problem a lot lately and almost at the point of ditching SPF entirely because of it. The "cons" - admin problems - seem to outweigh the "pros" so far.
On 22 Jul 2010, at 05:46, Regan Murphy wrote:
Anyway, I'm running into this problem a lot lately and almost at the point of ditching SPF entirely because of it. The "cons" - admin problems - seem to outweigh the "pros" so far.
It breaks lots of things - e.g. mail forwarding, maybe some mailing list implementations, but there is a feature in SPF which I really like (and use). If a domain should not be used for sending email at all, you can publish that in SPF with "v=spf1 -all". I check for this and can filter a few hundred mails a day sometimes.. anything that I can match on before running spamassassin is a great thing for resource preservation ! This sometimes comes up on the mailop.org mailing list. Best wishes andy
Hi All, Bear in mind that MX records are used to work out where mail should be sent to, and does not necessarily reflect where it is coming from and the SPF records are to verify that messages are coming from an authorised source. If the sending host happens to be the same as the receiving host, the "v=spf1 a mx ptr -all" record makes sense, but if they are relaying through their ISP, they'll need to add the relay hosts to that record. To me, it looks like oasystems.co.nz are relaying through gunge.hosts.net.nz (210.48.108.215) and this is not reflected in the published SPF record. For the SPF record to be valid, there needs to be an ip4:210.48.108.215 entry added. -----Original Message----- From: nznog-bounces(a)list.waikato.ac.nz [mailto:nznog-bounces(a)list.waikato.ac.nz] On Behalf Of Regan Murphy Sent: Thursday, 22 July 2010 4:04 p.m. To: Jasper Bryant-Greene; pid(a)ifm.net.nz Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection SPF of "v=spf1 a mx ptr -all" is supposed to mean that all MX hosts for a domain are the only authorised senders of mail for that domain, correct? And the IP must match the PTR record for that host. oasystems.co.nz: witch.oasystems.co.nz is the MX 10 for oasystems.co.nz witch.oasystems.co.nz resolves to 202.180.74.56 (A) 202.180.74.56 resolves to witch.oasystems.co.nz (PTR) Where is this SPF record broken? This line from the rejection: "210.48.108.215 is not allowed to send mail from oasystems.co.nz" is saying the host grunge.hosts.net.nz is not allowed to send email on behalf of oasystems.co.nz which is correct, it's not. -- Regan -----Original Message----- From: Jasper Bryant-Greene [mailto:jasper(a)metaname.co.nz] Sent: Thursday, 22 July 2010 3:54 p.m. To: Regan Murphy Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection On 22/07/2010, at 3:47 PM, Regan Murphy wrote:
Why is OrgA's SPF record broken?
Are you saying that OrgA needs to add SPF records to allow every ISP and Hosting company to relay email on its behalf?
No, only the hosts which send mail on OrgA's behalf. This is usually only a handful of hosts at most. In this case, it's probably just their server (if they send any mail directly), and/or their hosting company's mail server (if they use it as a so-called "smarthost"). http://en.wikipedia.org/wiki/Sender_Policy_Framework is quite comprehensive. -jasper _______________________________________________ NZNOG mailing list NZNOG(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/nznog
To me, it looks like oasystems.co.nz are relaying through gunge.hosts.net.nz (210.48.108.215) and this is not reflected in the published SPF record. For the SPF record to be valid, there needs to be an ip4:210.48.108.215 entry added.
No. OA Systems MTA only sends mail direct. It appears that safegas.co.nz have their mail configured to come via hosts.net.nz, and their on-premise mail server is enforcing SPF which should break for anyone sending them email.
MX records for domain safegas.co.nz Your 1 MX records are: 10 mta.hosts.net.nz ip=210.48.108.65
-----Original Message----- From: Martin Wilkinson [mailto:Martin(a)wilko.org] Sent: Thursday, 22 July 2010 4:22 p.m. To: Regan Murphy; Jasper Bryant-Greene; pid(a)ifm.net.nz Cc: nznog(a)list.waikato.ac.nz Subject: RE: [nznog] SPF Mail rejection Hi All, Bear in mind that MX records are used to work out where mail should be sent to, and does not necessarily reflect where it is coming from and the SPF records are to verify that messages are coming from an authorised source. If the sending host happens to be the same as the receiving host, the "v=spf1 a mx ptr -all" record makes sense, but if they are relaying through their ISP, they'll need to add the relay hosts to that record. To me, it looks like oasystems.co.nz are relaying through gunge.hosts.net.nz (210.48.108.215) and this is not reflected in the published SPF record. For the SPF record to be valid, there needs to be an ip4:210.48.108.215 entry added.
I've seen a lot of clients' outbound mail rejected in the last 12
months due to SPF issues; particularly mail being delivered to
secureMX, or Xtra. The plan of attack was usually:
1. do they appear on any RBL's? (self explanatory, I know, just
thought I'd list it)
2. does the HELO ID of their server match the MX record for the domain?
3. does the MX record (and HELO ID) contain the terms 'smtp' or
'mail'? (strange I know, but it's made a difference)
4. does a PTR record exist for the mail server's FQDN?
5. does the domain have a valid SPF record?
If all these criteria are met, you really shouldn't have any problems
passing go & collecting $200.
Love,
Simon W
On 22 July 2010 17:22, Mark Wakefield
-- "Catapultam habeo. Nisi pecuniam omnem mihi dabis, ad caput tuum saxum immane mittam."
For me, it's a yes to 4 criteria (our EHLO host id does not explicitly contain smtp or mail).... Every single time I've run into problems, so far, where my clients have not received our email or we have had mail rejections, it's been a misconfiguration of email/spam software at the receiving end. It's hard to quantify the benefits of having an SPF record - although I hope its helping prevent our IP from being RBL'd due to abuse by others - but its easy to see the admin costs of having one and having to constantly chase up problems. -----Original Message----- From: nznog-bounces(a)list.waikato.ac.nz [mailto:nznog-bounces(a)list.waikato.ac.nz] On Behalf Of SIMON WALKDEN Sent: Thursday, 22 July 2010 6:11 p.m. To: Mark Wakefield Cc: nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection I've seen a lot of clients' outbound mail rejected in the last 12 months due to SPF issues; particularly mail being delivered to secureMX, or Xtra. The plan of attack was usually: 1. do they appear on any RBL's? (self explanatory, I know, just thought I'd list it) 2. does the HELO ID of their server match the MX record for the domain? 3. does the MX record (and HELO ID) contain the terms 'smtp' or 'mail'? (strange I know, but it's made a difference) 4. does a PTR record exist for the mail server's FQDN? 5. does the domain have a valid SPF record? If all these criteria are met, you really shouldn't have any problems passing go & collecting $200. Love, Simon W
SPF for your own domain helps get past some of the more annoyingly twitchy providers out there - yahoo springs to mind. A record such as the one i've put in, can help: ;; QUESTION SECTION: ;blakjak.net. IN TXT ;; ANSWER SECTION: blakjak.net. 7200 IN TXT "v=spf1 a mx ptr ?all" note ?all. Doesn't enforce an absolute match. I choose not to test for spf on inbound mail, at the moment. Like any anti-spam measure it's merely part of the solution (a means to verify that mail is coming from whence it is intended, on the presumption that the admin for the domain is the one modifying the DNS) but that's all its for. Sender verification. With that in mind it's _not_ hard to quantify the benefit. It's just that many folks misinterpret or misunderstand the signifiance of what spf delivers. If you're having trouble getting your email to folks because you've published an spf record that's wrong, fix your spf record. At that point you've done the right thing and the problem is no longer yours? - Mark On Thu, 22 Jul 2010, Regan Murphy wrote:
I choose not to test for spf on inbound mail, at the moment.
At this stage I only use SPF on inbound to increase an overall spam score - not block it outright.
If you're having trouble getting your email to folks because you've published an spf record that's wrong, fix your spf record. At that point you've done the right thing and the problem is no longer yours?
Even if you do everything right with SPF, the problem *is* still yours if the company at the other end is misconfigured. This is because they are an entity I, or my staff, or my clients, need to be able to email. To make matters worse, these companies often have no dedicated IT staff and you have to move the earth to find out who you can talk to when you need *them* to do something to fix it... -----Original Message----- From: Mark Foster [mailto:blakjak(a)blakjak.net] Sent: Thursday, 22 July 2010 9:22 p.m. To: Regan Murphy Cc: SIMON WALKDEN; Mark Wakefield; nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection SPF for your own domain helps get past some of the more annoyingly twitchy providers out there - yahoo springs to mind. A record such as the one i've put in, can help: ;; QUESTION SECTION: ;blakjak.net. IN TXT ;; ANSWER SECTION: blakjak.net. 7200 IN TXT "v=spf1 a mx ptr ?all" note ?all. Doesn't enforce an absolute match. I choose not to test for spf on inbound mail, at the moment. Like any anti-spam measure it's merely part of the solution (a means to verify that mail is coming from whence it is intended, on the presumption that the admin for the domain is the one modifying the DNS) but that's all its for. Sender verification. With that in mind it's _not_ hard to quantify the benefit. It's just that many folks misinterpret or misunderstand the signifiance of what spf delivers. If you're having trouble getting your email to folks because you've published an spf record that's wrong, fix your spf record. At that point you've done the right thing and the problem is no longer yours? - Mark
On Thu, 22 Jul 2010 at 10:02:07 +0000, Regan Murphy wrote:
At this stage I only use SPF on inbound to increase an overall spam score - not block it outright.
The best thing about having SPF records for my domain and using SPF to block outright is stopping the significant number of bots that use addresses from my domain as mail-from as well as recipient. A quick couple of greps tells me this is about 10% of all inbound SMTP connections to my mail server, which means 10% less load on dspam and delivery. This may just be a luxury of having a vanity domain due to consequences that I haven't thought of but it works well for me. Nigel
stopping the significant number of bots that use addresses from my domain as mail-from as well as recipient.
I have a spoofing rule that blocks those regardless of SPF configuration. Of course I've also had to add exceptions for a few sites and lists that 'legitimately' spoof addresses.. -----Original Message----- From: Nigel Roberts [mailto:nigel(a)nobiscuit.com] Sent: Thursday, 22 July 2010 10:33 p.m. To: Regan Murphy Cc: Mark Foster; nznog(a)list.waikato.ac.nz Subject: Re: [nznog] SPF Mail rejection On Thu, 22 Jul 2010 at 10:02:07 +0000, Regan Murphy wrote:
At this stage I only use SPF on inbound to increase an overall spam score - not block it outright.
The best thing about having SPF records for my domain and using SPF to block outright is stopping the significant number of bots that use addresses from my domain as mail-from as well as recipient. A quick couple of greps tells me this is about 10% of all inbound SMTP connections to my mail server, which means 10% less load on dspam and delivery. This may just be a luxury of having a vanity domain due to consequences that I haven't thought of but it works well for me. Nigel
On 22/07/2010, at 6:10 PM, SIMON WALKDEN wrote:
In what way does it make a difference?
I would love to be able to use HELO/EHLO when deciding what to accept but my own testing shows far too many false positives. A few years ago and in another role I saw: http://blog.nominet.org.uk/tech/2005/08/06/incorrect-heloehlo-information-is... but I've seen similar problems recently when I last looked. I suspect we need a "Strict HELO/EHLO Day" where enough of the world's sysadmins unite to put strict settings on what they accept to force those with crap in their HELO/EHLO to do something about it. Jay
-- Jay Daley Chief Executive .nz Registry Services (New Zealand Domain Name Registry Limited) desk: +64 4 931 6977 mobile: +64 21 678840
On Thu, 2010-07-22 at 03:47 +0000, Regan Murphy wrote:
Why is OrgA's SPF record broken?
Are you saying that OrgA needs to add SPF records to allow every ISP and Hosting company to relay email on its behalf?
... and that's why SPF is a broken concept.
All mail must be sent by the authorised servers *only* which, for a
mobile workforce ( even mailing from home ) *MUST* use the company
servers, which means smtp auth of some kind, or only using a web-based
interface.
Doable but an administrative nightmare. The alternative is to use an SPF
record that allows anything, which is pointless.
Just my $0.02 ( and yes, my SPF's ok, sort of! ),
Steve
--
Steve Holdoway
I think most SMTP servers these days offer out of the box SMTP authentication, and plugging them into a back end database is relatively straight forward. I also think the majority of the workforce that is mobile uses proprietary solutions to address this already (such as VPN, Microsoft's RPC over HTTPS, etc). Suffice it to say, I think the issue for mobile workers sending email securely was solved long ago. -----Original Message----- All mail must be sent by the authorised servers *only* which, for a mobile workforce ( even mailing from home ) *MUST* use the company servers, which means smtp auth of some kind, or only using a web-based interface. Doable but an administrative nightmare. The alternative is to use an SPF record that allows anything, which is pointless. Just my $0.02 ( and yes, my SPF's ok, sort of! ), Steve
participants (11)
-
Andy Davidson -
Jasper Bryant-Greene -
Jay Daley -
Mark Foster -
Mark Wakefield -
Martin Wilkinson -
Nigel Roberts -
Philip D'Ath -
Regan Murphy -
SIMON WALKDEN -
Steve Holdoway