info:mail
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
info:mail [2015/01/22 19:52] – [Warning: NYI] hartmut | info:mail [2015/01/29 19:10] – [Sieve] hartmut | ||
---|---|---|---|
Line 24: | Line 24: | ||
To add custom sieve scripts, I recommend installing the [[http:// | To add custom sieve scripts, I recommend installing the [[http:// | ||
+ | |||
+ | Important: sieve scripts will only work when formatted with DOS-type CRLF linebreaks. The above extension takes care of that. | ||
+ | |||
+ | If you set up a new user, do not forget that dovecot looks for a .dovecot.sieve script in that user's home directory (which may be also a symlink to an include script in a sieve folder). | ||
For specific information about managing sieve on antiguru.de look at [[moritz: | For specific information about managing sieve on antiguru.de look at [[moritz: | ||
- | ===== Mail aggregation aka unified inbox (proposal) | + | ===== Mail aggregation aka unified inbox ===== |
- | ====Warning: NYI==== | + | Objective: collect email messages from several legacy freemail inboxes, pass them through a reliable virus and spam filter, and dispatch them to the IMAP inbox on the server. |
- | The setup described does now fulfill the expectations. Amavisd does now scan and decorate mails from fetchmail. But it has not yet been implemented on the ' | + | Benefits: reliable and controllable spam filtering, POP3 inboxes made accessible via IMAP |
+ | |||
+ | Drawback: on clients that can't be configured otherwise, answering mails received that way will result in the answers carrying your IMAP inbox address as sender. | ||
+ | |||
+ | ==== TLDR; ==== | ||
+ | |||
+ | Now that the system wide setup is working, it is simple to add a service that is polling an external inbox into your user's IMAP inbox here. Just create a file called .fetchmailrc in your user directory and adjust the | ||
+ | permissions so that fetchmail is going to accept them. | ||
+ | < | ||
+ | $ touch ~/ | ||
+ | $ chmod 0600 / | ||
+ | </ | ||
+ | Then, edit this file to match your credentials for the inbox you want to poll: | ||
+ | < | ||
+ | #log to system log - enable after verifying your setup | ||
+ | #set syslog | ||
+ | poll pop.gmx.net protocol pop3: | ||
+ | | ||
+ | # use secure connection relying on CA certificates | ||
+ | ssl | ||
+ | # do not delete from server (for testing) | ||
+ | keep | ||
+ | # get all messages, not just the ones that arrived after the last poll (use this after commenting out keep) | ||
+ | #fetchall | ||
+ | </ | ||
+ | Now, run fetchmail verbosely: | ||
+ | |||
+ | $ fetchmail -v | ||
+ | |||
+ | and check the output. If all is well, change the .fetchmailrc to delete messages from the remote inbox and log to system log, and add it to your crontab, so that it will be executed regularly by the system. | ||
+ | |||
+ | |||
+ | $ crontab -e | ||
+ | |||
+ | Add a line like this to your crontab: | ||
+ | < | ||
+ | */5 * * * * / | ||
+ | </ | ||
+ | (this will poll every 5 minutes). Then, save and exit the editor. Now, the fetchmail job will run unattended as set in the crontab. | ||
+ | |||
+ | Once this works, one could filter mail using the sieve method described above. | ||
+ | |||
+ | |||
+ | ==== For the administrator ==== | ||
+ | |||
+ | To fine-tune the setup, there are some things to be taken into account: | ||
+ | * spam scores: In / | ||
+ | * spam from the inboxes polled by fetchmail will be silently deleted when the score of sa_kill_level_deflt (now 20) is reached. You might want to review the logs for some time to look for false-positives being discarded. | ||
+ | * on the server, the rewriting of the subject lines of messages considered spam has been disabled. This makes it easier to deal with false positives. Just move them out of the spam folder. | ||
+ | * learning: do not delete true spam messages. After collecting lots of them, they should be used to train the spam filter. | ||
+ | * Ports: now, all clients submitting messages on port 465 (smtps) are considered local. Check if this is a valid assumption. If not, modify the system to accept these submissions on port 587 (submission). See below. | ||
+ | |||
+ | ==== Setup details | ||
+ | |||
+ | The setup described does now fulfill the expectations. Amavisd does now scan and decorate mails from fetchmail. | ||
After lots of trials which didn't work, I decided to move away from the production server. | After lots of trials which didn't work, I decided to move away from the production server. | ||
On my trusted Sparcstation, | On my trusted Sparcstation, | ||
- | First, you've to configure postfix. | + | But, as it turned out, there were some differences to the production system that had to be taken into acoount. See below. |
- | To the end of / | + | |
+ | ====Postfix==== | ||
+ | |||
+ | First, you' | ||
+ | To the end of / | ||
mailbox_command = / | mailbox_command = / | ||
+ | |||
+ | Also, we need to add or change the amavisfeed identifier/ | ||
+ | |||
+ | content_filter = amavisfeed: | ||
+ | |||
+ | All other things in main.cf stayed unchanged. | ||
To the end of / | To the end of / | ||
Line 87: | Line 155: | ||
# the MX record (or backup mailers) should point to this IP address | # the MX record (or backup mailers) should point to this IP address | ||
# set to your external IP address | # set to your external IP address | ||
- | 192.168.100.233: | + | # this does not work when you want to run this service chrooted |
- | -o content_filter=amavisfeed: | + | #192.168.100.233: |
+ | # -o content_filter=amavisfeed: | ||
# ========================================================================== | # ========================================================================== | ||
Line 112: | Line 181: | ||
</ | </ | ||
- | Important: There may not be whitespace before the first lines of the rules in master.cf, otherwise they' | + | Important: There may not be whitespace before the first lines of the rules in master.cf, otherwise they' |
- | Also important: further up in master, | + | Also important: further up in master, |
< | < | ||
- | #smtp inet n | + | smtp inet n |
- | # | + | |
</ | </ | ||
- | You need to disable them as shown here. Otherwise, at least when you forget to comment out the smtp line, postfix cannot start ('port already in use'). | + | and change it to: |
+ | < | ||
+ | # regular incoming mail, originating from anywhere (usually from outside) | ||
+ | # this works chrooted if you've set a content filter in main.cf | ||
+ | # like so: content_filter = amavisfeed: | ||
+ | # (this name must match the one given above). | ||
+ | # the MX record (or backup mailers) should point to this IP address | ||
+ | # set to your external IP address | ||
+ | EXTERNAL.IP.OF.MX: | ||
+ | </ | ||
+ | Of course, you'll have to enter the external IP address of your mail server (MX record) here. | ||
+ | This will run the smtp service that can be reached from outside in a chroot environment. | ||
+ | For roaming users reaching the server via smtps, we add a content filter to map them to our local port for amavisd: | ||
+ | < | ||
+ | smtps | ||
+ | -o smtpd_tls_wrappermode=yes | ||
+ | -o smtpd_sasl_auth_enable=yes | ||
+ | -o smtpd_client_restrictions=permit_sasl_authenticated, | ||
+ | -o milter_macro_daemon_name=ORIGINATING | ||
+ | -o content_filter=amavisfeed: | ||
+ | </ | ||
+ | (just the last line was added!). | ||
+ | |||
+ | ===Still problematic=== | ||
+ | |||
+ | But this assumes that only roaming clients will connect to the smtps port, which possibly is false. | ||
+ | |||
+ | So, I think it would actually be preferable to separate the roaming user service from the other smtp(s) services. That's where the port used for mail submission comes in (port 587), it is in postfix' | ||
+ | We'll have to take care that the chroot works for this too, and move the content flter for port 10042 to this service in master.cf. | ||
+ | |||
+ | |||
+ | Finally, look for a line like this and comment it out as shown. | ||
+ | < | ||
+ | # | ||
+ | </ | ||
+ | If you get an error like: postfix cannot start ('port already in use'), you should stop postfix and restart it, because simply reloading will not cause it to release ports and addresses. | ||
So, what does this configuration achieve? We set up postfix to take back mails (from amavis) on port 10025, and we prepare postfix to | So, what does this configuration achieve? We set up postfix to take back mails (from amavis) on port 10025, and we prepare postfix to | ||
* listen on the external IP address as smtp server and hand over mail from there to amavis at port 10040 | * listen on the external IP address as smtp server and hand over mail from there to amavis at port 10040 | ||
+ | * listen on the smtps port and hand over email messages received there to amavis at port 10042 | ||
* listen on the local loopback email address at port 2345 as smtp server and hand over mail from there to amavis at port 10041 | * listen on the local loopback email address at port 2345 as smtp server and hand over mail from there to amavis at port 10041 | ||
* listen on the local loopback email address as smtp server and hand over mail from there to amavis at port 10042 | * listen on the local loopback email address as smtp server and hand over mail from there to amavis at port 10042 | ||
+ | |||
+ | Backward compatibility: | ||
+ | |||
+ | ====Amavisd-new==== | ||
Of course, this setup has to be matched by a corresponding configuration for amavisd-new. | Of course, this setup has to be matched by a corresponding configuration for amavisd-new. | ||
Apart from uncommenting the spam filter lines in / | Apart from uncommenting the spam filter lines in / | ||
- | < | ||
- | (is this needed? will have to find out) | ||
all other changes could be made to / | all other changes could be made to / | ||
< | < | ||
# be more verbose | # be more verbose | ||
- | log_level =2 ; | + | log_level = 2; |
# ports we listen on | # ports we listen on | ||
Line 149: | Line 255: | ||
# some global settings | # some global settings | ||
- | # do not quarantine | + | # local domains |
- | $QUARANTINEDIR = undef; | + | @local_domains_maps = ( [" |
+ | |||
+ | # do not quarantine | ||
+ | $virus_quarantine_to = $QUARANTINEDIR; | ||
+ | $spam_quarantine_to | ||
# Spam levels | # Spam levels | ||
# always tag | # always tag | ||
Line 161: | Line 271: | ||
# regular incoming mail, originating from anywhere (usually from outside) | # regular incoming mail, originating from anywhere (usually from outside) | ||
$policy_bank{' | $policy_bank{' | ||
- | # just use global settings, no special overrides | ||
}; | }; | ||
# incoming mail from fetchmail, considered externally originating | # incoming mail from fetchmail, considered externally originating | ||
$policy_bank{' | $policy_bank{' | ||
- | log_level => 2, | + | log_level => 2, |
+ | # if ' | ||
+ | # explicitely setting this flag. If set, this means: we declare that all | ||
+ | # mail passed through this policy is local-destined and has to be scanned | ||
+ | | ||
# no bounces for spam, not even for score below spam_dsn_cutoff_level_maps: | # no bounces for spam, not even for score below spam_dsn_cutoff_level_maps: | ||
final_spam_destiny => D_DISCARD, | final_spam_destiny => D_DISCARD, | ||
}; | }; | ||
+ | |||
# mail locally submitted on the host on which MTA runs | # mail locally submitted on the host on which MTA runs | ||
+ | # or submitted via the smtps port by roaming users | ||
$policy_bank{' | $policy_bank{' | ||
+ | originating => 1, | ||
# NOTE: this is just an example; ignoring internally generated spam | # NOTE: this is just an example; ignoring internally generated spam | ||
# may not be such a good idea, consider zombified infected local PCs | # may not be such a good idea, consider zombified infected local PCs | ||
Line 184: | Line 300: | ||
So far, so good. I've only got one question: how does amavisd detect which mail messages are to be considered local/ | So far, so good. I've only got one question: how does amavisd detect which mail messages are to be considered local/ | ||
- | |||
- | Maybe we'll have to investigate the originating flag of amavisd-new. If set to 1, this means that mail is considered to be outbound (originating from the server). | ||
- | |||
The correct setting of local_domains_maps is also important: | The correct setting of local_domains_maps is also important: | ||
Line 192: | Line 305: | ||
recipient (inbound or all-internal), | recipient (inbound or all-internal), | ||
[from the amavisd mailing list] | [from the amavisd mailing list] | ||
+ | |||
+ | If the automatic decision if a mail is considered local does not work as wanted, we'll have to investigate the originating flag of amavisd-new. If set to 1, this means that mail is considered to be outbound (originating from the server), and should not be processed. I added this flag to two of the policy banks laid out above. | ||
As last step, you'll have to add the port for the fetchmail rule above to the user's .fetchmailrc: | As last step, you'll have to add the port for the fetchmail rule above to the user's .fetchmailrc: | ||
Line 205: | Line 320: | ||
</ | </ | ||
- | Result: this will process mails from fetchmail | + | Note: if you've got an active firewall and experience timeouts running |
- | The next step will now be to move this system to the production machine. | + | Result: |
====Details: | ====Details: |
info/mail.txt · Last modified: 2024/02/20 11:46 by hartmut