Fetchmail (Debian)
This article describes different use cases for Fetchmail as well as how they can be implemented with Dovecot IMAP Server on Debian 10.
Introduction
There are four major use cases for downloading mails with Fetchmail:
- Download manually via IMAP or POP protocol
- Download periodically via IMAP or POP protocol by daemon
- Download periodically via IMAP or POP protocol by cronjob
- Download instantly via IMAP IDLE connection
IMAP IDLE can be used by default only for one account. There is a script on the internet which creates multiple instances which enables the administrator to create multiple IMAP IDLE connections each to a different account. It's use and how to alter it to work on Debian is described below.
Install Fetchmail on Debian
Install:
apt-get install fetchmail
Locations and Permissions of Configuration Files
| File | Content | Owner | Permissions |
|---|---|---|---|
| /etc/fetchmailrc | System config | fetchmail:root | 600 |
| ~/.fetchmailrc | User config | user:user | 600 |
| ~/.netrc | Passwords for user config | user:user | 600 |
As configuration files contain user names and passwords their permissions should be set to 600.
Manual Download
Download INBOX to Default Inbox
Start Fetchmail in user context either using user config with:
fetchmail
Or with a specific config:
fetchmail -f </path/to/specific/fetchmailrc-config-file>
Example for ~/.fetchmailrc which downloads all mails from INBOX folder at the specified host:
poll host.domain.tld protocol IMAP
user "<username@host.domain.tld>" with pass "<password>" is "<username@localhost>" here
ssl fetchall
Remember to restrict permissions for that file in order to prevent other users to see your password.
There is also another variant of storing passwords which is described in section Passwords in Configuration Files
Download Selected Remote Folder to Specific Local Folder
An email fetched by fetchmail will be delivered to SMTP port 25 on the local machine by default. That prevents a specification to which local folder an email should be delivered to. In order do deliver emails from a selected remote folder to a specific local folder one has to use the MDA parameter.
When using the MDA parameter the MTA for SMTP running at port 25 will be bypassed and the email will be directly handled by the specified MDA.
Example for delivering emails from the remote folder called "Sent" to the local folder called "Sent":
poll host.domain.tld protocol IMAP
user "<username@host.domain.tld>" is "<username@localhost>" here
folder Sent
ssl fetchall
mda "/usr/lib/dovecot/deliver -d <username@localhost> -m Sent"
Periodical Download
Fetchmail Running as Daemon
Enable Daemon in config file /etc/default/fetchmail by changing the following line to:
START_DAEMON=yes
In the system specific config file /etc/fetchmailrc a polling interval in seconds will be set for the daemon with "set daemon". Following that all mailboxes will be specified. It is possible to skip a interval for a mailbox by specifing the "interval" parameter followed by a number which says how many intervals should be skipped. Example:
set daemon 60
poll host1.domain.tld protocol IMAP interval 3
user "<username@host1.domain.tld>" with pass "<password>" is "<username@localhost>" here
ssl fetchall
smtp localhost
poll host2.domain.tld protocol IMAP interval 10
user "<username@host2.domain.tld>" with pass "<password>" is "<username@localhost>" here
ssl fetchall
smtp localhost
Fetchmail Running as Cronjob
Fetchmail can also be run in user context by specifing a user specific cronjob with crontab -e
Content can be somewhat like this for running the user specific config every day at 2:30 am:
30 2 * * * fetchmail
or like this for running a specific config:
30 2 * * * fetchmail -f </path/to/specific/fetchmailrc-config-file>
For details see section Manual Download as well as Cronjob (Debian)
Immediate Download via IMAP IDLE
Single IMAP IDLE Instance for Single Account
IMAP IDLE can be used in the system config /etc/fetchmailrc in order to have it active after a reboot. Example:
poll host.domain.tld protocol IMAP
user "<username@host.domain.tld>" with pass "<password>" is "<username@localhost>" here
idle
ssl
Hint: This works only with one Fetchmail instance and therefore only one IMAP IDLE connection per server as the connection persists as soon as it was started.
Workaround: In order to have multiple Fetchmail instances with multiple IMAP IDLE connections for different mailboxes each Fetchmail instance has to have it's own process ID:
Multiple IMAP IDLE Connections for Multiple Mailboxes
Source: http://fnxweb.com/blog/2012/07/14/using-multiple-fetchmail-instances-for-instant-gratification/
The description above is altered to match Debian folders and security.
Files:
http://www.fnxweb.com/data/fetchmail-service --> Save as fetchmail-imap-idle at /usr/local/bin
http://www.fnxweb.com/data/fetchmail.service --> Save as fetchmail-imap-idle.service at /usr/local/bin
Add execute permissions:
chmod +x /usr/local/bin/fetchmail-imap-idle
Create symbolic link to fetchmail-imap-idle.service in /etc/systemd/system:
ln -s /usr/local/bin/fetchmail-imap-idle.service /etc/systemd/system/fetchmail-imap-idle.service
Place config files at: /etc/fetchmail.conf.d/*.conf
Each config file should contain only one IMAP connection.
Example /etc/fetchmail.conf.d/my-imap-account1-inbox.conf
poll host.domain.tld protocol IMAP
user "<username@host.domain.tld>" with pass "<password>" is "<username@localhost>" here
idle
ssl
Avoid parameter fetchall in IMAP IDLE config files as this leads to somewhat intransparent behavior of the local dovecot LDA and emails don't appear at once rather than somewhat delayed.
Fix: Try to send another mail to remote mailbox. After that it seems that all missing mails appear magically in Dovecot mailbox.
Alter /usr/local/bin/fetchmail-imap-idle:
Change lines 10 and 11 as follows:
fetchmailuser=fetchmail fetchmailgroup=nogroup
Alter paths and executable names in /etc/systemd/system/fetchmail.service as follows:
[Unit] Description=A remote mail retrieval and forwarding utility After=network-online.target sendmail.service [Service] ExecStart=/usr/local/bin/fetchmail-imap-idle start ExecReload=/usr/local/bin/fetchmail-imap-idle restart ExecStop=/usr/local/bin/fetchmail-imap-idle stop Type=forking GuessMainPID=no [Install] WantedBy=multi-user.target
Activate service as follows:
systemctl enable fetchmail-imap-idle systemctl start fetchmail-imap-idle
Service can be removed if necessary with:
systemctl stop fetchmail-imap-idle systemctl disable fetchmail-imap-idle
Initial Download of Entire Mailbox
First download of an entire mailbox should be done via manual downlod rather than via IMAP IDLE with fetchall.
For further information see Manual Download
Further Information
Passwords in Configuration Files
Parameter with pass <password> can be removed from file ~/.fetchmailrc for security reasons if the file ~/.netrc exists in the user profile. It contains passwords and will be read by fetchmail at runtime. Permissions for this file see section Locations and Permissions of Configuration Files
This does not work with system config file /etc/fetchmailrc
Example for ~/.netrc:
machine <host1.domain.tld> login <username@host1.domain.tld> password <pass> machine <host2.domain.tld> login <username@host2.domain.tld> password <pass>
Keep Emails on Source Server
Edit config file and replace parameter fetchall with keep.
Use of Additional Files
fetchmail -f <config-filename> # Specify config file fetchmail --pidfile <pidfile> # Specify process ID file
Remove Unseen Flag in Sent Folder
When using Fetchmail to download Sent folder every downloaded mail will be flagged as unseen.
In order to mark them seen create following cronjob with crontab -e.
*/3 * * * * doveadm flags add -u <username@localhost> '\Seen' mailbox Sent unseen
Sources
IMAP IDLE: http://fnxweb.com/blog/2012/07/14/using-multiple-fetchmail-instances-for-instant-gratification/
