How To Optimize SPF Record?

If you have a domain that sends emails, you probably have some default SPF record, already set by the hosting provider. So why do you need to optimize SPF record?

That record usually consists of either A or IP4 / IP6 and MX mechanisms, if you have dedicated hosting, or of MX and INCLUDE mechanisms, if you use shared hosting. 

That default SPF record ensures that emails, sent from your website or VPS machine, located at your hosting provider, are authenticated. 

In real life, however, people also use a variety of other 3-rd party systems to send emails from their domain (e.g. CRM, or Cloud email providers such as G Suite or Office 365), so the default record requires modification to cover servers, which 3rd party systems send emails from to keep their emails authenticated.

Creating a new or modifying an existing SPF record

You can create a new SPF record or modify the existing one, using one of the following ways:

• use our SPF record generator form, to create new or update existing SPF record. This tool generates syntax-correct record and at the same time highlights the issues with used mechanisms, if such is detected during generated record validation.

• use our SPF record raw validator tool if you are well familiar with SPF record syntax and prefer to work with the raw record. This tool validates draft record, shows expanded SPF record tree, and highlights issues if found.

Validating SPF record for correct syntax and limitations

There are several hidden challenges you may face while maintaining an efficient working SPF record. Some of them relate to record syntax and some to limitations/restrictions, inherited from times the SPF was established as a standard, back in 2006.

Here is the list of most widely known cases:

1. Using the PTR mechanism

PTR mechanism asks to perform DNS reverse-mapping of sending IP address and then perform DNS lookup of returned hostnames to verify if sending IP is among resolved IP addresses. In cases when IP resolves to many hostname records, like e.g. here, it will initiate several DNS lookups and may lead to reaching 10 DNS lookups limit, described in one of the following points. It is not possible to validate the PTR mechanism during the SPF record check since it is a dynamic one and depends on sending IP, thus we just show the warning saying that its usage is not recommended.

2. Using CDN to proxy domain DNS

If your domain is protected by CDN like e.g. Cloudflare and DNS A / AAAA records are proxied, these records will be resolved to proxied IP and not to the actual IP address of the host that sends emails. In such cases, you have to replace A mechanism with IP4 / IP6 and specify the actual IP address of sending the server.

3. 10 DNS lookups

SPF too many DNS lookups is the most commonly faced challenge, happening when people add additional email sources to SPF record. As per RFC7208, to avoid unreasonable load on the DNS the maximum number of allowed DNS lookups should be limited to 10. Each of a, mx, or include mechanisms, as well as redirect modifier, tells the receiving mail server to perform a DNS lookup to resolve the hosts to IP addresses. If SPF record, resolved from include mechanism, contains another include mechanism then DNS lookups count increases by one, and so on. It is not so easy to count the number of lookups and identify the limit’s exceed without the use of a special lookup tool.
SPF record is being read from left to right, thus the mechanism/modifier, located in string after 10th lookup, will be ignored and emails coming from the source, defined by those ignored mechanisms, will fail SPF authentication with “permerror” check result code.

4. Two void lookups

This case is mentioned in the same section of RFC7208, defining 10 DNS lookup limitation. If A, MX, INCLUDE mechanism in SPF record can resolve hostname but returned answer is void (e.g. include:existing-domain-with-no-spf.com), or a cannot resolve hostname (e.g. a:non-exising-domain.com) then it is referred to as “void lookups”. The recommendation is to stop processing SPF record lookup once void results count has reached 2. Exceeding this limit should produce “permerror” check result code as well.

5. Recursive lookups

This is also quite hard to detect the case when one include mechanism contains direct or indirect include mechanist to itself, which creates an infinite loop and immediately leads to 10 DNS lookup limitation restriction. Two examples of such records:
example.com TXT v=spf1 include:example.com ~all
OR
sub1.example.com TXT v=spf1 include:sub2example.com include:sub3.example.com ~all
sub3.example.com TXT v=spf1 include:sub1.example.com ~all

6. White space symbol usage

If you accidentally type extra white space symbol in mechanism name or domain name (e.g. inclu de:_spf.google.com or a:so mecrm.com or ip 4:12.3 4.56.78 )  the rest of string, after unrecognized element, may be ignored so the correct sources, listed after bad element, will drop when receiving mail server performs a lookup of your domain SPF record. Also, the white space in the hostname, A and INCLUDE mechanisms point to, is treated as a void lookup.

7. More than one SPF records

You need to be sure that only one TXT record, starting with “v=spf1“, exists in domain DNS. Having more than one TXT record, starting with “v=spf1” will cause unpredicted results of lookup by receiving mail server and SPF validation may fail.

8. SPF record length

Single TXT record data string’s maximum length is limited to 255 bytes, though, according to RFC7208, multiple data strings, each with up to 255 bytes long, can be concatenated into one TXT record, if you need a longer SPF record. E.g.TXT “v=spf1 …. first string-” “second string…” will be parsed as TXT “v=spf1 …. first string-second string…”

You will also need to ensure that your domain host provider’s DNS zone management interface supports TXT records longer than 255 bytes. If it is not the case you have to consider moving your domain DNS zone to some alternative system, which supports long TXT records, e.g. Cloudflare.Besides the above said there is another important limitation specific to SPF type of TXT records. It is recommended to keep record size small enough, so the result of the DNS query response will fit with 512 bytes. Otherwise, there is a possibility of exceeding a UDP packet size limit, defined in DNS protocol.  Though this limitation was revised in RFC6891, staying below 512 bytes still should eliminate cases receiving parties with older, RFC6891 non-ready DNS implementation, fail to validate emails from domain having long SPF record.In fact, the DNS query, executed to read SPF record, returns not only the SPF record but all TXT records, published in domain DNS. Since your domain may (and probably does) have other TXT records, like Google sites, Microsoft O365 or other systems verification TXT records, you have to remember to count them as well when calculating the size left for SPF record (512 – DNS hostname – all other TXT records length).You can use dig example.com txt command in Linux or Dig Web Interface online tool to see how many TXT records you have and calculate the safe size for the SPF record.See the example of a domain with many TXT records:

All above cases can be detected and displayed by our SPF Record Lookup tool, see the screenshot:

Optimizing SPF record

Below you can find the steps that would help you to optimize SPF record:

1. Change sources order

SPF record is being read from left to right, so it makes sense to put the most critical or frequently used email sources at the beginning of the record.

2. Clean up the record from duplicate sources

Sometimes you may need to free up costly SPF space from duplicate sources, add new ones, particularly when you are about to reach 10 DNS lookups limit. Use our SPF Record Lookup tool to detect duplicate sources in the SPF record’s expanded tree.

3. Get rid of legacy / obsolete sources

If you are analyzing DMARC reports for a sufficient period of time (say during 1-2 months) you would be able to see some sources, added to the SPF record, which was not sending emails during that period.  Ensure whether the associated systems still are used in your company, and remove them if they are no longer used.

4. Reduce the number of INCLUDEs

Each INCLUDE mechanism takes one DNS lookup. If the INCLUDE mechanism does not contain directly added IP4/IP6 mechanisms but contains other INCLUDEs, which also take per one DNS lookup, you can try to replace those INCLUDEs with IP4/IP6.
However, it is a very tricky step, especially when INCLUDE refers to 3rd party ESP service. You may easily break your emails SPF compliance because you will never know if one day your ESP decides to send emails from new IPs and update their SPF. Also, you need to remember about the recommended limiting of SPF record size to fit in DNS query UDP response packet (512 bytes).

5. Avoid using MX and A mechanisms

MX mechanism takes one DNS lookup from 10 DNS lookups limit, and it also triggers a DNS lookup for hostnames your MX records points to. While these lookups do not count against the SPF record DNS lookups limit, they however may increase the latency of emails delivery time.
It would be more reasonable to use IP4 / IP6 mechanisms instead and include IP ranges your MX sends mails from, to save on DNS query response times.
Using IP4/IP6 directives instead is suitable for A mechanism too.

6. Remove sender-domain-unaligned sources

As you know SPF check runs against the domain in Return-path (smtp.mailfrom) email address or domain in smtp.helo (when smtp.mailfrom is empty). Usually, the domain in “Header From” and “Return-path” addresses are the same. In Gmail, you can see “Header From” address in the from field and “Return-path / smtp.helo” domain in mailed-by.
See the example below:

Header From: VMware <[email protected]>
Return-Path: <[email protected]>

Some ESPs, however, use their own domain in Return-path email address, see example below:

Header From: Booked Scheduler <[email protected]>
Return-Path: <bounce-mc.us19_134631002.4790858-956d20b353@mail249.suw12.mcsv.net>

Configuration guides of many of those “non-compliant” ESPs either recommend or demand you to add their SPF include string in your domain SPF record. Obviously this is a result of poor knowledge of how SPF verification works. The incomplete list of well-known ESPs that use their own domain in Return-path addresses is: Active Campaign, Aweber, Mailchimp, Marketo, Salesforce.

Since, as we mentioned earlier, the receiving server performs a check of SPF record of the domain, mentioned in Return-path address, and not in Header From address, there is no requirement to add ESP’s SPF include string to your domain SPF record, to occupy costly space and trigger useless DNS lookups.

7. Move part of sources to the subdomain

When, due to usage of many INCLUDE mechanisms, 10 DNS lookups limit is fully exhausted, you may try reconfigure some email sources to send from a subdomain and not root domain (see the VMware email sample in point 6) and move those sources INCLUDE string to new SPF record, created for the subdomain.

8. Use our EasySPF for smart and dynamic flattening of SPF record

And finally, if you do not like to implement above 7 recommendations yourself, which will definitely take a lot of your time for optimization as well as assume regular checking of 3rd party ESPs’ SPF record for possible changes (applicable if you implement recommendation #4 you can use our EasySPF solution which does all above recommendations for you automatically, without any intervention from your side.
You will just need to activate the EasySPF service and replace your SPF record with our, customized one for smart SPF.