SpamAssassin Filter for New TLDs (.xyz .info .ninja etc)
Have you seen an increased spam from new TLDs (top level domains like these)?
.link, .xyz, .info, .ninja
This short tutorial demonstrates how to create a filter to add points for messages that are not from a list of preferred TLDs.
Important: This filter is not for everyone and you should adjust for best results considering your mail traffic and typical senders. You also should be familiar with editing plain text configuration files.1
Lets get started:
The local configuration for SpamAssassin is stored in this directory:
/Library/Server/Mail/Config/spamassassin
Within this directory, you can customize SpamAssassin with configuration files for filters, whitelists, blacklists, score overrides and more. These config files must end in .cf and are processed in alphabetical order. When the same setting is repeated, the last occurrence wins.
Your additions should load last, so we’ll call this new filter: z_tld.cf
Lets think about the goal.
We want to reduce spam, but still accept/deliver legitimate mail from these TLDs.
SpamAssassin runs hundreds of tests, and they all have a subtle effect on the final spam score.
We don’t want to be too heavy handed. For our example: we’re going to add 1.5 points to the final score.
Here’s our filter:
file: /Library/Server/Mail/Config/spamassassin/z_tld.cf
# add points if the From address is not a valid host in a listed TLD
header LOCAL_FROM_TLD From !~ /@[a-z0-9\-\.]+\.(com|org|net|mil|edu)/i
describe LOCAL_FROM_TLD From address is not a valid host in a listed TLD
score LOCAL_FROM_TLD 1.5
Lets break it down:
header:
This is the meat of the filter. We are searching the From header for mail not !~ matching the regex expression. The regex expression has two parts.
Part 1: /@[a-z0-9\-\.]+\. catches a legit hostname (mail.company) without the TLD (com, org, etc). spammer@spam!domain.com would be caught because ! is not allowed in a hostmame.
Part 2: (com|org|net|mil|edu) is the list of TLDs we do not penalize. Edit this list to include any TLD you typically receive mail from. Note: the filter ends in /i. A spamassassin expression begins with / ends with / and the i means case insensitive.
describe:
Description of the filter
score:
We are adding 1.5 points.
Remember, this is a negative match !~, so we add 1.5 points when the TLD is not com|org|net|mil|edu.
Shortcut
If you decide to implement this ‘as-is’, copy/paste the following in Terminal:
echo '# TLD Filter
# adds points if the From address is not a valid host in a listed TLD
header LOCAL_FROM_TLD From !~ /@[a-z0-9\-\.]+\.(com|org|net|mil|edu)/i
describe LOCAL_FROM_TLD From address is not a valid host in a listed TLD
score LOCAL_FROM_TLD 1.5' | sudo tee -a /Library/Server/Mail/Config/spamassassin/z_tld.cf
sudo launchctl stop org.amavis.amavisd
Test and Verify Results
Test your mail system, make sure you are able to send/receive.
Watch the amavis log located at /Library/Logs/Mail/amavis.log and you should see hits.
From your mail application, check for the x-spam-status header.
Check if syntax, typos or other errors in this filter have caused any errors:
sudo -u _amavisd -H spamassassin --lint -D 2>&1 | grep LOCAL_FROM_TLD
Reference
http://commons.oreilly.com/wiki/index.php/SpamAssassin/SpamAssassin_Rules
Document Version 1.0, 11.2.2016
- If you are unsure about how to edit a configuration file, have a look at our tutorial on how to edit text configuration files on OS X Server ↩
Here’s a way to modify this to penalize bad domains. Just insert the spammer’s domain names in the TLD list in your z_tld.cf file. Here I am punishing .xyz, .top, .pro, and .eu TLDs with 1.5 points. Change this to suit your conditions.
TLD Filter
adds points if the From address is a bad host in a listed TLD
header LOCAL_FROM_TLD From =~ /@[a-z0-9-.]+.(xyz|top|pro|eu)/i
describe LOCAL_FROM_TLD From address is a bad host in a listed TLD
score LOCAL_FROM_TLD 1.5
I have a question about postfix and spamassassin. How can I know if macosx server mail services is filtering outgoing mail for spam ? Just in case one of my user is infected and starts sending spam.
Hi Simon
OS X Server scans inbound and outbound mail by default.
You can check the settings with
sudo serveradmin settings mail | grep scan_enabledThis should return
mail:postfix:virus_scan_enabled =yesmail:postfix:spam_scan_enabled = yes
Its possible to over-ride the settings above for authenticated or local users, so its best that you verify scanning.
Here’s a handy way to check how your mail was received by a 3rd party.
Send email to: check-auth@verifier.port25.com
You’ll receive a reply with a mail authentication report.
At the bottom of the report, you’ll find your original email including headers.
The headers reveal Virus and Spam scanning status.
Examples
X-Virus-Scanned: amavisd-new at topicdesk.com
virus scanning was performed by my server
X-Spam-Flag: NO
X-Spam-Score: 2.501
X-Spam-Level: **
X-Spam-Status: No, score=2.501……
Received: from topicdesk.com ([127.0.0.1])
by localhost (topicdesk.com [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id Dc9Ucu8EO6-q for nobody@topicdesk.com;
Tue, 15 Nov 2016 20:32:27 -0500 (EST)
spam scanning was performed by topicdesk.com
I found this helpful. One more tip: to block an entire TLD, I added lines like this to my whitelist (since I don’t have a domain blacklist):
top REJECT
rocks REJECT
Blocks these two TLDs
Hello Tom,
Blocking an entire TLD has been our goal for years. we have implemented all sorts of filter settings, etc. given to us by support but none have worked. It honestly is like there is some known vulnerability that spammers are able to exploit whereby they bypass SpamAssassin altogether. It has also been elusive to get clear information for support about how to stop the thousands of daily spams from these domains. The TLDs we would like to block altogether are .top, .pro, .click, .xyz, .review, .download, .date, .win, .racing, .party, .work, .tokyo, .science.
Can it really be as simple as adding to the SpamAssassins whitelist the text “top REJECT”, “pro REJECT”, etc.?
Please elaborate further on your tip.
Thanks for the nifty tutorial, gang. I’ve been using a “sender_access” file with a hash for blocking spammy TLD’s via Postfix. That said, I implemented your instructions above, but am seeing this in /Library/Logs/Mail/amavis.log:
/Applications/Server.app/Contents/ServerRoot/usr/bin/amavisd[83186]: SA info: config: SpamAssassin failed to parse line, no value provided for “score”, skipping: score LOCAL_FROM_TLD 1.5 | sudo tee -a /Library/Server/Mail/Config/spamassassin/z_tld.cf
Any suggestions/hints on fixing this so that amavisd understands it?
Following up. The command “sudo -u _amavisd -H spamassassin –lint -D 2>&1 | grep LOCAL_FROM_TLD” returns nothing, so I figure the file’s fine.
Oh Dog. Now I see the problem: a typo in the file.