Differences between revisions 35 and 36
Revision 35 as of 2012-01-21 14:34:45
Size: 11403
Editor: OsamuAoki
Comment:
Revision 36 as of 2012-01-29 13:50:39
Size: 12142
Editor: OsamuAoki
Comment:
Deletions are marked like this. Additions are marked like this.
Line 36: Line 36:
For the above mentioned Gmail specific POP3 retrieval problem, there are several possible solutions which minimizes your mail to be blocked by SPAM filter somewhere on the Internet:
 * Solution 1: set up 2 Gmail accounts. One for SMTP of your outgoing mails. Another for POP3 of your incoming mails. (If this set-up becomes to be blocked by Gmail, please update this page)
  * Set the mail envelope address to the mail address of your SMTP account.
  * Set the mail "From: " address to your receiving mail address. (Use the real Gmail POP3 account name if possible)
 * Solution 2: Use non-Gmail SMTP service such as one provided by your connection ISP providing IP connectivity to you.
  * Set the mail envelope address to the mail address of your connection ISP account.
  * Set the mail "From: " address to your receiving mail address. (Use the real Gmail POP3 account name if possible)

(Although "Solution 2" does not use Gmail, it is likely such configuration that you can get good hints from this Gmail case described here.)

Again, there are some risks even if you use "Solution 1". Risks become larger if you use a mail forwarding service to receive mail to Gmail.
Recently, Gmail started to rewrite both envelope and header addresses to the email address of your account automatically if you use their SMTP smarthost while providing DKIM signiture added to your mail. So there is no nore point making complicated mangling of email address if you use gmail.

If you still wish to receive mail with some forwarding address such as foo-guest@alioth.debian.org, you need to use non-Gmail SMTP service such as one provided by your connection ISP providing IP connectivity to you.
  * Set the mail "From: " address to your receiving mail address such as foo-guest@alioth.debian.org or foo.bar@gmail.com which get delivered to your Gmail address.
  * Hope for your ISP not to overwrite From address.
Line 129: Line 123:
(Actually, recent Gmail rewrites source address automatically anyway. So you do not need to do the above. But Gmail may change again. This is still valid work needed for other smart host setting. So I keep it as is now.)
Line 131: Line 127:
(Although choosing "YES" for "Hide local mail name in outgoing mail?" may allow you to avoid this trouble, you risk sending unintended mails to smarthost.)

If you wish to deploy "Solution 1" to use a different visible e-mail address in the "From :" message header field from the mail envelope address set by DebianPkg:exim4 here as {{{SMTPAccountName@gmail.com}}}, you need to set it by mail clients generation mail messages such as DebianPkg:mutt etc.
(Although choosing "YES" for "Hide local mail name in outgoing mail?" may allow you to avoid this trouble, you risk sending unintended mails such as ones to "root" to smarthost.)
Line 173: Line 167:
== Hint for SMTP service with SSL == == Hint for SMTP service with SSL (Yahoo) ==
Line 179: Line 173:
You need to edit system configuration 30_exim4-config_remote_smtp_smarthost in /etc/exim4/conf.d/transport to add "protocol=smtps". (A bit intrusive change ... if you have better suggestion, please update this.) You need to edit system configuration 30_exim4-config_remote_smtp_smarthost in /etc/exim4/conf.d/transport to add "protocol=smtps". (A bit intrusive change ... if you have better suggestion, please update this.) See DebianBug:656841#5
Line 182: Line 176:

== Hint for using multiple smarthosts depending on your header From: addreses ==

If you wish to use multiple smarthosts depending on your header From: addreses, you can edit your /etc/exim4/conf.d contents.

I have made a example of it and submitted to exim4-config as wishlist bug report. If you wish to use this type of configuration, you just need to make the same content as the patch in /etc/exim4/conf.d/ and add a file in /etc/exim4/conf.d/rewrite/ with a line
{{{
ENABLE_MULTIPLE_SMARTHOSTS = yes"
}}}

See DebianBug:656841#10

== Hints for setting mutt for multiple From: addresses ==

If you wish to deploy use a different visible e-mail addresses in the "From :" message header field, you need to set it by mail clients generation mail messages such as DebianPkg:mutt etc.

Here is an example of .muttrc:

{{{
alternates "foo\.bar@gmail\.com|bar\.foo@gmail\.com|foo_bar@example\.com|foobar@debian\.org"

macro compose "1" "<edit-from>^UFoo Bar \<foobar@debian.org\>\n"
macro compose "2" "<edit-from>^UFoo Bar \<foo.bar@gmail.com\>\n"
macro compose "3" "<edit-from>^UFoo Bar \<bar.foo@gmail.com\>\n"
macro compose "4" "<edit-from>^UFoo Bar \<foo_bar@example.com\>\n"
}}}

Primary Information Sources

<!> Check Gmail service site with key word like "gmail configuring other mail clients" for new configuration requirement changes.

<!> Please read Debian Exim4 User FAQ first.

PkgExim4UserFAQ is updated by the exim4 maintainer. It should be more current and reliable than this page. "Section 10.1" on PkgExim4UserFAQ as of January 2012 mentions problems of this page. People updated this wiki page after that comment tried to address problems as much as possible. If you still find any errors, please login to edit this page to correct it.

This wiki page is only a secondary information.

Mail Address Basics

Due to pervasive SPAM problem, most SMTP services including Gmail checks legitimacy of mail messages while delivering them. Email address rewriting as discussed here may cause such SMTP services to filter your mail as illegitimate one and drop it quietly. Understanding of the difference between the mail envelope address and the mail "From: " address is essential to minimize such risks. Be careful about this problem since situation changes all the time.

Known Limitations of Gmail

If you send a mail to a mailing list via a SMTP server provided by Gmail, the returned mail message from the mailing list is treated as **read** by the Gmail POP3 service and it may not be downloaded to your local mail client by the mail remote mail retrieval and forwarding utility such as fetchmail/getmail/... . (This is true even if your visible receiving mail address in the "To: " field is one with non-Gmail mail forwarding service as long as it is forwarded to the Gmail account you used to send the mail.)

Recently, Gmail started to rewrite both envelope and header addresses to the email address of your account automatically if you use their SMTP smarthost while providing DKIM signiture added to your mail. So there is no nore point making complicated mangling of email address if you use gmail.

If you still wish to receive mail with some forwarding address such as foo-guest@alioth.debian.org, you need to use non-Gmail SMTP service such as one provided by your connection ISP providing IP connectivity to you.

Using Exim4 to Send Messages through GMail

<!> GMail is said to have limit of 100 messages/day/account when you use it to send mails via its SMTP service. Keep this in mind :)

Debian ships default with Exim4 as a mail server. You can configure this Exim4 to use the Google's SMTP servers as its smarthost using your GMail account.

Let's assume:

  • You deploy "Solution 1" strategy as mentioned above or use the same account for SMTP and POP3.
  • Your account name of your local Debian box: YOUR-USER-NAME

  • Your host name in /etc/hostname: hostname1

  • Your host name in the 127.0.1.1 line of /etc/hosts: hostname1.localdomain hostname1

  • Your envelope address is SMTPAccountName@gmail.com for the outgoing SMTP connection to Gamil's SMTP server via Exim4

  • Your Exim4 relays local SMTP connection coming with the mail envelope address: YOUR-USER-NAME@localhost and YOUR-USER-NAME@localhost.localdomain

  • Your account name for SMTP at Gmail is SMTPAccountName@gmail.com

  • Your password for SMTP at Gmail is y0uRpaSsw0RD

(If you have a real DNS resolvable IP address, the 127.0.1.1 line of /etc/hosts should not exist but should have one for that IP address.)

Initial configuration of Exim4 is invoked with:

# dpkg-reconfigure exim4-config
  • Choose "mail sent by smarthost; received via SMTP or fetchmail"
  • Set to "localhost" for "System mail name:".
  • Set to "127.0.0.1" for "IP-addresses to listen on for incoming SMTP connections" to disable external connection.
  • Leave as empty for "Other destinations for which mail is accepted:".
  • Leave as empty for "Machines to relay mail for:".
  • Set to "smtp.gmail.com::587" for "IP address or host name of the outgoing smarthost:".
  • Choose "NO" for "Hide local mail name in outgoing mail?".
  • Choose "NO" for "Keep number of DNS-queries minimal (Dial-on-Demand)?".
  • Leave as empty for "".
  • Choose "mbox format in /var/mail/" for "Delivery method for local mail".
  • Choose "YES" for "Split configuration into small files?".

This setting is for a typical desktop PC which has no resolvable host name. If you wish to do something more such as relaying mail, you need to configure this accordingly.

Submission port 587 on Gmail stystem uses STARTTLS service to ensure secure password protection.

(For servers with a real DNS resolvable host name, set it accordingly. But why are you bothering to set up smarthost to use gmail? I see no good reason to do this.)

Configuration of Exim4 for SMTP authentication uses the canonical host name.

Gmail provide the SMTP service using server at smtp.gmail.com . Get the canonical host names of such servers by running:

$ host smtp.gmail.com
smtp.gmail.com is an alias for gmail-smtp-msa.l.google.com.
gmail-smtp-msa.l.google.com has address 74.125.127.109
gmail-smtp-msa.l.google.com has address 74.125.127.108

So gamil*.google.com matches all host names.

The configure of your Exim4 to use the external SMTP service such as Gmail is stored in the /etc/exim4/passwd.client file. Run

# editor /etc/exim4/passwd.client

and add the following lines:

*.google.com:SMTPAccountName@gmail.com:y0uRpaSsw0RD

Here's how /etc/exim4/passwd.client works: the first field in this file is matched against the reverse DNS lookup of the remote smtp server (in this case GMail). In other words, Exim4 will do a DNS query to get the IP address of smtp.gmail.com, and then it will do a reverse DNS query to get the domain name of this IP (similar to the result of running "$ host smtp.gmail.com"). If this domain name (or IP, if the server IP doesn't have a reverse domain name configured) matches the first field of this file, it will try to authenticate for SMTP using this login / password.

If you restored /etc/exim4/passwd.client from the backup file, please make sure to restore its file ownership and permissions with

# chown root:Debian-exim /etc/exim4/passwd.client
# chmod 640 /etc/exim4/passwd.client

Edit your address rewrite table /etc/email-addresses by running:

# echo 'YOUR-USER-NAME: SMTPAccountName@gmail.com' >> /etc/email-addresses
# echo 'YOUR-USER-NAME@localhost: SMTPAccountName@gmail.com' >> /etc/email-addresses
# echo 'YOUR-USER-NAME@hostname1: SMTPAccountName@gmail.com' >> /etc/email-addresses
# echo 'YOUR-USER-NAME@hostname1.localdomain: SMTPAccountName@gmail.com' >> /etc/email-addresses

(Actually, recent Gmail rewrites source address automatically anyway. So you do not need to do the above. But Gmail may change again. This is still valid work needed for other smart host setting. So I keep it as is now.)

If any messages comes to your Exim4 with different envelope addresses, you need to list them too.

(Although choosing "YES" for "Hide local mail name in outgoing mail?" may allow you to avoid this trouble, you risk sending unintended mails such as ones to "root" to smarthost.)

Finally, run

# update-exim4.conf
# invoke-rc.d exim4 restart
# exim4 -qff

This 3 step process ensures to update the Exim4 configuration, to reload it, and to force a delivery attempt for all messages including frozen ones.

See "man update-exim4.conf" for more on Exim4 configuration.

Verification of Configuration Results and Trouble Shootings

After the configuration, please verify your configuration result with

# tail /var/log/exim4/mainlog

If you start receiving SMTP authentication errors in mainlog messages, run

$ host smtp.gmail.com
smtp.gmail.com is an alias for gmail-smtp-msa.l.google.com.
gmail-smtp-msa.l.google.com has address 74.125.127.109
gmail-smtp-msa.l.google.com has address 74.125.127.108

to verify the resolving host names all match with the definition in your /etc/exim4/passwd.client. Gmail may have changed its host name naming scheme.

If you see in mainlog messages that state something like

failed to open /etc/exim4/passwd.client for linear search:
Permission denied (euid=102 egid=102)

you have some file permission problems. Please restore the default permissions as in the case mentioned for the backup file described above. (euid and egid may be different on your installation.)

If you see in mainlog messages that state "Credentials Rejected", you have mail account or password problem. Please update your account name password in the /etc/exim4/passwd.client file.

Although most ISP's account name does not include "@isp-name.com" part, Gmail requires it.

Hint for SMTP service with SSL (Yahoo)

Although STARTTLS service on port 587 was supported, deprecated SMTPS protocol (SSL on port 465) was not supported by older Exim as a client.

Debian wheezy system comes with Exim 4.77 and it support SMTPS. So non-Gmail server such as Yahoo can be used safely.

You need to edit system configuration 30_exim4-config_remote_smtp_smarthost in /etc/exim4/conf.d/transport to add "protocol=smtps". (A bit intrusive change ... if you have better suggestion, please update this.) See 656841

See http://www.exim.org/exim-html-current/doc/html/spec_html/ch30.html under "4. Private options for smtp" on "protocol".

Hint for using multiple smarthosts depending on your header From: addreses

If you wish to use multiple smarthosts depending on your header From: addreses, you can edit your /etc/exim4/conf.d contents.

I have made a example of it and submitted to exim4-config as wishlist bug report. If you wish to use this type of configuration, you just need to make the same content as the patch in /etc/exim4/conf.d/ and add a file in /etc/exim4/conf.d/rewrite/ with a line

ENABLE_MULTIPLE_SMARTHOSTS = yes"

See 656841

Hints for setting mutt for multiple From: addresses

If you wish to deploy use a different visible e-mail addresses in the "From :" message header field, you need to set it by mail clients generation mail messages such as mutt etc.

Here is an example of .muttrc:

alternates "foo\.bar@gmail\.com|bar\.foo@gmail\.com|foo_bar@example\.com|foobar@debian\.org"

macro compose "1" "<edit-from>^UFoo Bar \<foobar@debian.org\>\n"
macro compose "2" "<edit-from>^UFoo Bar \<foo.bar@gmail.com\>\n"
macro compose "3" "<edit-from>^UFoo Bar \<bar.foo@gmail.com\>\n"
macro compose "4" "<edit-from>^UFoo Bar \<foo_bar@example.com\>\n"