spamtrainer
spamtrainer assists Mac OS X Server mail services administrators in updating and maintaining their SpamAssassin bayes database.
spamtrainer will read the designated HAM and SPAM mailboxes, update the SpamAssassin databases and delete mail that has been learned from.
It has an array of additional functions. Among them the possibility to backup/restore your bayes database and many more.
This script is written for OS X 10.8.x, 10.9.x , 10.10.x, 10.11.x, 10.12.x and 10.13.x with Server 2.x/3.x/4.x/5.x. A separate release is available for earlier OS X versions.
See current changelog for a complete list of fixes and additions.
Please read our FAQs as well.
spamtrainer is a free download.
Hi,
I have installed and isued the command : /usr/local/sbin/spamtrainer -i
Then I ran: /usr/local/sbin/spamtrainer
I get an message there is no folder?
Starting spamtrainer…
Training from user folders
Learning SPAM…
Learned new SPAM (junk mail)
Either there is no mailbox called 1F21C7F0-4E19-433C-A2FA-E1CFC482E303 in the mailpartition
or it has never been fed with mail.
Syncing SpamAssassin Database
Displaying SpamAssassin Database Stats
0.000 0 2873 0 non-token data: spam
0.000 0 68996 0 non-token data: ham
Done!
Output produced by spamtrainer Version 2.2.1
I’m running 10.11.6 with server 5.1 and I have setup user junkmail and notjunkmail.
Why is the script telling me no folder?
You need to feed those mailboxes with at least one ham and spam message.
I have a similar error. In this case, my spamtrainer is saying that there is no mailbox called A23F2247-D869-44E9-85F5-7BBA01EAB46C in the mailpartition or it has never been fed with mail, which is true… the Junkmail mailbox is A8829322-26B9-4143-857F-5E0573136445. Where do I set the correct mailbox?
You have to feed both mailboxes with at least one spam/ham message.
The problem is that spamtrainer is looking in the wrong place… the junkmail mailbox is not the one its looking for. The actual junkmail account does have spam; the notjunkmail account does have ham.
spamtrainer gets the UUID of the users from the system. So if the lookup for user junkmail returns a different UUID than the one you would expect, there might be a duplicate user, a typo or some other directory error. Any chance the user exists in the local directory and OD? Or maybe a user by different name used an alias junkmail?
No, there is no other user or alias called junkmail. Is there a setting to change to spamtrainer looks in the right place?
You can use the -p flag to give it a name of a mailbox, but it will still look up the UID so the result would be the same.
Hi,
I know I have multiple mails in the junkmail box. Could it be as we host more than one domain it looks for the wrong folder?
No. The domains are not relevant.
BOTH mailboxes need to contain mail.
I added the notjunkmail and put some mail in it now it runs like this:
Starting spamtrainer…
Training from user folders
Learning SPAM…
Learned new SPAM (junk mail)
Learning HAM…
Learned new HAM (not junk mail)
Syncing SpamAssassin Database
Displaying SpamAssassin Database Stats
0.000 0 2875 0 non-token data: spam
0.000 0 69036 0 non-token data: ham
Done!
Output produced by spamtrainer Version 2.2.1
This looks good?
Yes
Fine, One last question.
What happens when I only have spam in junkmail folder and none in notjunkmail?
Will it still learn? and will it only delete the folders after when the automated script runs?
When I run it from command line it does not delete any of the mails.
So tomorrow morning it should be empty I guess?
It will learn what it finds. No need to have mail in both. This is only required initially, otherwise Dovecot won’t create the mailboxes.
To delete mail after learning, you need to use the -d option.
Cool works just fine!
I’m curious how fast it learns and when spam will be less.
I’ve used to block IP numbers but they change all the time so we had many spam.
I’ll let you know how it goes and will spread the news if satisfied.
Thanks!
The Bayes DB contributes to the spam scores. It is not by any means a blacklist. To reduce spam on your mail server, you will need a combination of a well trained Bayes DB, fine tuned rules and rule scores, proper amavisd settings, a couple of RBLs, DNSBLs and ideally Postgrey.
Looks like spam trainer is no go on macOS 10.12.x? I had no issues with Server 5.2 on 10.11.x, then upgraded to 10.12.2 and noticed mail still in junkmail. When I run “sudo /usr/local/sbin/spamtrainer -i” I get the error: “This script can be used only on Mavericks and Mountain Lion.” When running spamassassin at the terminal, it hangs.
It hasn’t been updated yet. A 10.12 compatible version will be released within 1-2 months.
Okay, thanks for the heads up!
Hi!
First of all; THANK YOU for providing this for free. That said, its now almost 4 months after this post and still no update for 10.12.x. Do you have an eta on an updated version?
I emailed topicdesk with a patch to ensure spamtrainer works up through HighSierra. I have been manually patching this script with every release since the last update, and emailing topicdesk. Don’t know why they don’t implement the support, but the patch is fairly easy.
Hi,
Any progress with Mac Os Sierra supported version?
Thank you.
See 2 comments above yours 😉
Might I ask for the status of Spamtrainer on Mac OSXSierra, again? It is becoming urgent …
Hello, I have installed the newest version under Sierra and get the message: This script can be used only on Mavericks and Mountain Lion.
What is the reason?
See a few comments above yours ?
Hi,
I’ve found your work really usefull, but like many other people above, Sierra broke everything.
I was waiting an update after readings you post above, did you give up ?
BR
It is simply delayed because we are busy on other fronts.
Hi,
is this software something like replace of that, that is in Mac OS X Server? While it should be done automatically by the system, without deleting the old mails in mailbox.
Not sure I understand your question.
HI, I mean this function should be implemented in in OS X Server (after installing Server.app) but only what it doesn’t do is cleaning the mailboxes.
Or it isn’t so, and therefor you have created this app?
And if yes is possible to set it, that it doit automatically at set time?
Thanks.
Yes and yes. You can setup a cron job or startup item by issuing:
spamtrainer -i
What you mean with first yes? 🙂 It isn’t implemented in Server.app?
Spamtrainer has nothing to do with Server.app. It replaces the built in system script.
OK understand and it has something better as the internal script?
Hello Athanasios:
Many thanks for Spamtrainer. I’ve been using it for many years and it is a great utility. Is there any chance you will be updating it to work with MacOS Sierra in the near future?
Thanks,
Yes.
I had posted a patch to make it work (again) with shared folders, and to run under Sierra. Was this ever implemented?
If you could please provide this patch to the public, it would be highly appreciated!
I posted it here and it was removed from here. I dont want to violate any lecenses. Let me look into the license.
Hi Athanasios:
Keeping this thread fresh…do you have an ETA on the configuration refresh for spamtrainer so that it will work with MacOS Sierra?
Thanks,
Sven
This is my MacOS patch that works for me: (note, this re-enables shared folder logic that I use). Save the below text as spamtrainer.patch, and use the patch tool to apply the patch. Be sure to backup the current /usr/local/sbin/spamtrainer first.
— spamtrainer 2017-06-16 12:47:22.000000000 -0700
+++ /usr/local/sbin/spamtrainer 2017-06-16 12:49:08.000000000 -0700
@@ -32,7 +32,7 @@
color_1=
echo -en "\033[37;40m"
color_2=
echo -en "\033[37;44m"
PROJECT_NAME=$(basename “$0”)
-PROJECT_VERSION=2.2.1
+PROJECT_VERSION=2.2.2
if [ “$TERM” != “dumb” ]; then
@@ -70,6 +70,11 @@
os_version=
uname -r | cut -d. -f1
case $os_version in
+ 16)
+ imap_user=”_dovecot”
+ spamav_user=”_amavisd”
+ server_root_path=”/Applications/Server.app/Contents/ServerRoot”
+ ;;
15)
imap_user=”_dovecot”
spamav_user=”_amavisd”
@@ -125,13 +130,13 @@
junk_mailbox=”junkmail”
notjunk_mailbox=”notjunkmail”
-if [[ “$os_version” == “13” ]] || [[ “$os_version” == “14” ]] || [[ “$os_version” == “15” ]]; then
+if [[ “$os_version” == “13” ]] || [[ “$os_version” == “14” ]] || [[ “$os_version” == “15” ]] || [[ $”os_version” == “16” ]]; then
junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
notjunk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$notjunk_mailbox"
fi
if [[ “$os_version” == “12” ]]; then
– junk_mailbox_GUID=
$server_root_path/usr/bin/cvt_mail_data -i "$junk_mailbox"
– notjunk_mailbox_GUID=
$server_root_path/usr/bin/cvt_mail_data -i "$notjunk_mailbox"
+ junk_mailbox_GUID=
$server_root_path/usr/bin/cvt_mail_data -i "$junk_mailbox"
+ notjunk_mailbox_GUID=
$server_root_path/usr/bin/cvt_mail_data -i "$notjunk_mailbox"
fi
mailpartition_PATH=`cat /Library/Server/Mail/Config/dovecot/partition_map.conf | grep "^default:" | sed 's/default://'`
@@ -594,7 +599,7 @@
echo “Checking if there is a startup item for ‘learn_junk_mail’ or ‘spamtrainer'”
echo
-if [[ “$os_version” != “14” ]] && [[ “$os_version” != “15” ]]; then
+if [[ “$os_version” != “14” ]] && [[ “$os_version” != “15” ]] && [[ “$os_version” != “16” ]]; then
if (grep -q learn_junk_mail $plist_PATH/*)
then
plist_NAME=
grep -l learn_junk_mail $plist_PATH/*
@@ -808,13 +813,14 @@
# Check for SPAM collecting user account
if [ -d $mailpartition_PATH/”$junk_mailbox” ]
then
– echo “Learning SPAM…”
+ echo “Learning SPAM… ($mailpartition_PATH/$junk_mailbox)”
cd $mailpartition_PATH/"$junk_mailbox"/cur
for file in * ; do
if [ -f $file ]; then
cat $file | sudo -u $spamav_user -H $sa_PATH --dbpath /Library/Server/Mail/Data/scanner/amavis/.spamassassin --spam --no-sync >> /dev/null;
+ cat $file | sudo -u $spamav_user -H $sa_PATH –dbpath /Library/Server/Mail/Data/scanner/amavis/.spamassassin –spam –no-sync >> /dev/null;
if [[ “$purgemail” == “yes” ]]; then
rm $file
fi
@@ -825,6 +831,7 @@
for file in * ; do
if [ -f $file ]; then
cat $file | sudo -u $spamav_user -H $sa_PATH –dbpath /Library/Server/Mail/Data/scanner/amavis/.spamassassin –spam –no-sync >> /dev/null;
+ cat $file | sudo -u $spamav_user -H $sa_PATH –dbpath /Library/Server/Mail/Data/scanner/amavis/.spamassassin –spam –no-sync >> /dev/null;
if [[ “$purgemail” == “yes” ]]; then
rm $file
fi
@@ -842,7 +849,7 @@
# Check for HAM collecting user account
if [ -d $mailpartition_PATH/”$notjunk_mailbox” ]
then
– echo “Learning HAM…”
+ echo “Learning HAM… ($mailpartition_PATH/$notjunk_mailbox)”
cd $mailpartition_PATH/"$notjunk_mailbox"/cur
@@ -880,7 +887,7 @@
echo “Displaying SpamAssassin Database Stats”
-sudo -u $spamav_user -H $sa_PATH –dbpath /Library/Server/Mail/Data/scanner/amavis/.spamassassin –dump magic | grep “am” | sed “s/: n/: /” | sort -k 3
+sudo -u $spamav_user -H $sa_PATH –dbpath /Library/Server/Mail/Data/scanner/amavis/.spamassassin –dump magic | grep “am” | sed “s/: n/: /” | awk ‘{print $7″: “$3}’ | sort -k 3
if [[ “$log_stats” == “yes” ]]
then
@@ -948,7 +955,7 @@
m) mail_address=”$OPTARG”
mail_report=yes;;
s) sa_PATH=$OPTARG;;
– t) if [[ “$os_version” != “12” ]] && [[ “$os_version” != “13” ]] && [[ “$os_version” != “14” ]] && [[ “$os_version” != “15” ]]; then
+ t) if [[ “$os_version” != “12” ]] && [[ “$os_version” != “13” ]] && [[ “$os_version” != “14” ]] && [[ “$os_version” != “15” ]] && [[ “$os_version” != 16 ]]; then
shared_folder=yes
fi;;
i) startup_check_new
@@ -977,29 +984,43 @@
echo
else
echo
– if [[ “$os_version” != “12” ]] && [[ “$os_version” != “13” ]] && [[ “$os_version” != “14” ]] && [[ “$os_version” != “15” ]]; then
+ if [[ “$os_version” != “12” ]] && [[ “$os_version” != “13” ]] && [[ “$os_version” != “14” ]] && [[ “$os_version” != “15” ]] && [[ “$os_version” != “16” ]]; then
junk_mailbox=”user/$junk_mailbox”
notjunk_mailbox=”user/$notjunk_mailbox”
else
if [[ “$os_version” == “12” ]]; then
– junk_mailbox_GUID=
$server_root_path/usr/bin/cvt_mail_data -i "$junk_mailbox"
+ junk_mailbox_GUID=
$server_root_path/usr/bin/cvt_mail_data -i "$junk_mailbox"
notjunk_mailbox_GUID=
$server_root_path/usr/bin/cvt_mail_data -i "$notjunk_mailbox"
fi
if [[ “$os_version” == “13” ]]; then
– junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
+ junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
notjunk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$notjunk_mailbox"
fi
if [[ “$os_version” == “14” ]]; then
– junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
+ junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
notjunk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$notjunk_mailbox"
fi
if [[ “$os_version” == “15” ]]; then
– junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
+ junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
+ notjunk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$notjunk_mailbox"
+ fi
+ if [[ “$os_version” == “16” ]]; then
+ junk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$junk_mailbox"
notjunk_mailbox_GUID=
$server_root_path/usr/bin/mail_data_tool -g "$notjunk_mailbox"
fi
junk_mailbox=$junk_mailbox_GUID
notjunk_mailbox=$notjunk_mailbox_GUID
if [[ $junk_mailbox_GUID =~ “No GUID found for user” && -d “$mailpartition_PATH/$junk_mailbox” ]]; then
shared_folder=yes
unset junk_mailbox_GUID
else
junk_mailbox=$junk_mailbox_GUID
fi
if [[ $notjunk_mailbox_GUID =~ “No GUID found for user” && -d “$mailpartition_PATH/$notjunk_mailbox” ]]; then
shared_folder=yes
unset notjunk_mailbox_GUID
else
notjunk_mailbox=$notjunk_mailbox_GUID
fi
fi
fi
fi
Hi there, and many thanks! You say “this re-enables shared folder logic that I use.” Is that different then the conventional spam trainer installation?
In an earlier version, you could create a public folder for spam and notspam, that is linked into every mailbox. I have been manually reenaing this with each release, and did not remove it for this patch. The patch appears mangled here, due to certain character sequences resulting in box drawings. I will pmay another day to see if I can get it to post clean.
How can I fix this?
bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /Library/Server/Mail/Data/scanner/amavis/.spamassassin/bayes.lock.mail.eps-amsterdam.nl.5972 for /Library/Server/Mail/Data/scanner/amavis/.spamassassin/bayes.lock: Permission denied
Displaying SpamAssassin Database Stats
ERROR: Bayes dump returned an error, please re-run with -D for more information
Deleting learned HAM and SPAM
Thanks
Sounds like the file permissions are not OK. Make sure the user _amavisd can read/write /Library/Server/Mail/Data/scanner/amavis and its contents.
Thanks I changed that and works again!
This spamtrainer doesn’t work on a NON server mac then? It’s looking for the server.app, and of course I don’t have it.
I have instead installed the spamassassin components and postfix etc etc onto a Mac Mini running my mailserver
Correct. It works only in server.