According to Kaspersky, in 2013 over 70% of emails were spam. One the results of this is that valid emails sent from your own infrastructure risk ending up in users' spam folders.

PostRFP sends out plenty of emails for user registration and for notifying of significant RFP events - invitation published, RFP response submitted, deadline reminders etc.. When these emails end up in the Spam folder it causes inconvenience for our users. Worse, because we manage high value RFP transactions, it's essential that we maintain our credibility. Asking users to dig our emails out of their spam folders is far from ideal...

SPF (Sender Policy Framework) records are one of the crucial techniques for ensuring that outgoing email isn't categorised as spam. An SPF record is a short block of text that lives alongside the DNS record for your domain. This text should provide the IP address or hostname of servers that are permitted to send outgoing emails from users within your domain.

"v=spf1 ip4:192.168.0.2 ~all"

This is a record that says "I'm using version 1 of SPF and I permit IP address 192.168.0.2 to send emails on my behalf. Other sources are permitted, but should be treated with suspcion" (that's what the ~all means).

The background to this is that spammers like to hijack email (STMP) servers, and send their spam from there. Spammers often use real addresses as the "From" field on emails - spoofing the identity of the real person to try to sneak through on whitelist policies. To counteract this, when a mail server receives incoming messages it checks that source address of the sending server matches the SPF record for the domain of the sender.

So far, so good. It seems all one has to do is write the SPF record, associated with your DNS, and everthing is fine. There are lots of sites with advice about writing SPF records:

• http://www.unlocktheinbox.com/spfwizard/ - good wizard t0 create records
• http://appmaildev.com/en/spf/ test existing records
• http://www.openspf.org/The official source of information - written by engineers; largely incomprehensible

Google Apps have their own page about setting up SPF records: https://support.google.com/a/answer/178723

We updated our SPF records for a new IP address weeks ago, but Google continued to dump our messages in Spam folders. At first we thought that Google was caching the SPF records, but it went on too long. Reluctantly, we decided it might be time to start using a 3rd party service for email delivery. Capsule CRM is an SAAS web application that we admire, so we looked at how they send emails. That led us to Mandrill an STMP service from the people behind a Mailchimp - they should clearly be experts at avoiding spam classification!

Mandrill have a great tool for setting up SPF records, and they validate them immediately - it's very easy to diagnose problems. Within minutes of signing up the problem was clear: we used capital letters for "IP" in our SPF record. Doh! Our original version looked something like this:

"v=spf1 IP4:192.168.0.2 -all"

Wrong. It should be:

"v=spf1 ip4:192.168.0.2 -all"

The interesting thing is that plenty of mail receipients (e.g. Yahoo), don't care about the use of upper case. And plenty of software libraries will validate records with uppercase "IP" fine. But Google is picky. So plenty of valuable hours lost there. Hopefully we can help someone else figure this out before they spend too much time on it.