]> Mail Autoconfig Beonex
ben.bucksch@beonex.com
next generation unicorn sparkling distributed ledger Set up a mail account with only email address and password. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Source for this draft and an issue tracker can be found at .
Introduction This protocol allows users to set up their existing email account in a new mail client application, by entering only their name, email address, and password. The mail application, by means of mail autoconfig specified here, will determine all the other parameters that are required, including IMAP or POP3 hostname, TLS configuration, form of username, authentication method, and other settings, and likewise for SMTP. Contact sync and calendar, file sharing and other services can also be set up automatically. The protocol works by first determining the domain from the email address, and the querying well-known URLs at the email provider, which return the configuration parameters in computer-readable form. Failing that, various fallback sources can be applied, like a common database of configurations for large email providers who do not directly support this protocol, or other mechanisms to determine the configuration.
Implementations This protocol is in production use since 15 years by major email clients, and the config database (used as fallback) contains configurations for over 50% of all email accounts. Currently, this protocol or parts of it has been implemented by:
  • Thunderbird
  • Evolution
  • KMail
  • Kontact
  • K9 Mail
  • FairEmail
  • NextCloud email
and likely other mail clients. The purpose of this paper is to document and specify what is deployed in the wild. A later version 2 of the protocol might be based on JSON. Additionally, there are known problems with OAuth2 in combination with mail clients, which would need to be solved by another specification.
Data format Whether the ISP or a common central database returns the configuration, the resulting document has the following data format. The MIME type is text/xml or text/xml+autoconfig.
Sample config file example.com example.net Google Mail GMail pop.example.com 995 SSL %EMAILLOCALPART% password-cleartext true true 14 optional: the user's password smtp.googlemail.com 587 STARTTLS %EMAILLOCALPART% password-cleartext client-IP-address true true optional: the user's password Configure Thunderbird 2.0 for IMAP Thunderbird 2.0 mit IMAP konfigurieren %EMAILADDRESS% http-basic https://contacts.example.com/remote.php/dav %EMAILADDRESS% http-basic https://calendar.example.com/remote.php/dav %EMAILADDRESS% http-basic https://share.example.com/remote.php/dav %EMAILADDRESS% Elise Bauer Check 'Enable IMAP and POP' in Google settings page Schalten Sie 'IMAP und POP aktivieren' auf der Google Einstellungs-Seite an ]]>
Formal definition TODO Schema for XML
Config retrieval The mail client application, when it needs the configuration for a given email address, will perform several steps to retrieve the configuration from various sources. The steps are ordered by priority. They may all be requested at the same time, but a higher priorty result that is available MUST be preferred over a lower priority one, even if the lower priority one is available earlier. Lower priority requests MAY be cancelled, if a valid higher priority result has been successfully received. The priority is expressed below with the number before the URL or location, with lower numbers meaning higher priority, e.g. 1.2 has higher priority than 4.1. In the URLs below,%EMAILADDRESS shall be replaced with the email address that the user entered and wishes to use, and %EMAILDOMAIN% shall be replaced with the email domain extracted from the email address. For example, for "fred@example.com", the email domain is "example.com", and for "fred@test.cs.example.net", the email domain is "test.cs.example.net". For full support of this specification, all "Required" and "Recommended" mechanisms MUST be implemented and working. For partial support of this specification, all "Required" mechanisms MUST be implemented and working, and in this case, you shall make explicit when advertizing or referring to auto config that there is only partial support of this specification.
Mail provider First step is to directly ask the mail provider and allow it to return the configuration. This step ensures that the protocol is decentralized and the mail provider is in control of the configuration issued to mail clients. Mail providers MUST return a working configuration, with state-of-the-art security. Configuration fields MUST NOT contain invalid or non-working configuration data. To allow the mail provider to return a configuration adjusted for that mailbox, the client sends the email address as query parameter. This allows the mail provider to e.g. separate mailboxes on geographically local mail servers, e.g. a mail server located in the same office building where an employee works. However, while the protocol allows for such heterogenous configurations, mail providers are discouraged from doing so, and are instead encouraged to provide one single configuration for all their users. For example, DNS resolution based on location, mail proxy servers, or other techniques as necessary, can be used to route the traffic and host the mail efficiently.
  • 1.1. https://autoconfig.%EMAILDOMAIN%/mail/config-v1.1.xml?emailaddress=%EMAILADDRESS% (Required)
  • 1.2. https://%EMAILDOMAIN%/.well-known/autoconfig/mail/config-v1.1.xml (Recommended)
  • 1.3. http://autoconfig.%EMAILDOMAIN%/mail/config-v1.1.xml (Optional)
For example:
  • 1.1. https://autoconfig.example.com/mail/config-v1.1.xml?emailaddress=fred@example.com
  • 1.2. https://example.com/.well-known/autoconfig/mail/config-v1.1.xml
  • 1.3. http://autoconfig.example.com/mail/config-v1.1.xml
Central database The ISPDB contains the configurations for most mail providers with a market share larger than 0.1%, and contains configurations for half of the email accounts in the world. This is a useful fallback for mail providers which do not host a config server described in the previous step. Using a central database (ISPDB) of mail configurations for the large mail providers will increase the success rate of finding a valid configuration drastically, up to 10-fold. The mail client application may choose the mail config database provider. A public mail config database is available at base URL https://autoconfig.thunderbird.net/v1.1/. (TODO change ISPDB URL) %ISPDB% below is the base URL of that database.
  • 2.1. %ISPDB%%EMAILDOMAIN% (Recommended)
For example:
  • 2.1. https://autoconfig.thunderbird.net/v1.1/geologist.com
MX Many companies do not maintain their own mail server, but let their email be hosted by a hosting company, which is then responsible for the email of dozens or thousands of domains. For these hosters, it may be difficult to set up the configuration server (step 1.1.) with valid TLS certificate for each of their customers, and to convince their customers to modify their root DNS specifically for autoconfig. On the other side, the ISPDB can only contain the hosting company and cannot know all their customers. To handle such domains, the protocol first needs to find the server hosting the email. If the previous mechanisms yield no result, the client may perform a DNS MX lookup on the email domain, and retrieve the MX server (incoming email server) for that domain. Only the highest priority MX hostname is considered. From that MX hostname, 2 values are extracted:
  • Remove the first component from the MX hostname, i.e. everything up to and including the first ., and use that as value for %MXFULLDOMAIN%.
  • Extract only the second-level domain from the MX hostname, and use that as value for %MXMAINDOMAIN%. To determine the second-level domain, use the Public Suffic List or a similarly suited method, to correctly handle domains like ".co.uk" and ".com.au".
For example:
  • For "mx.example.com", the MXFULLDOMAIN and MXMAINDOMAIN are both "example.com".
  • For "mx.example.co.uk", the MXFULLDOMAIN and MXMAINDOMAIN are both "example.co.uk".
  • For "mx.premium.europe.example.com", the MXFULLDOMAIN is "premium.europe.example.com" and the MXMAINDOMAIN is "example.com".
Then, attempt to retrieve the config for these MX domains, using the previous methods:
  • 3.1. https://autoconfig.%MXFULLDOMAIN%/mail/config-v1.1.xml?emailaddress=%EMAILADDRESS% (Recommended)
  • 3.2. https://autoconfig.%MXMAINDOMAIN%/mail/config-v1.1.xml?emailaddress=%EMAILADDRESS% (Recommended)
  • 3.3. %ISPDB%%MXFULLDOMAIN% (Recommended)
  • 3.4. %ISPDB%%MXMAINDOMAIN% (Recommended)
For example:
  • 3.1. https://autoconfig.premium.europe.example.com/mail/config-v1.1.xml?emailaddress=fred@example.com
  • 3.2. https://autoconfig.example.com/mail/config-v1.1.xml?emailaddress=fred@example.com
  • 3.3. https://autoconfig.thunderbird.net/v1.1/premium.europe.example.com
  • 3.4. https://autoconfig.thunderbird.net/v1.1/example.com
Local disk For testing purposes, you may want to define a location on the disk, relative to the application installation directory, or relative to the user configuration directory, which may contain a configuration file for a specific domain, and which your application will use, if the above methods fail.
  • 4.1. %USER_CONFIGURATION_DIR%/isp/%EMAILDOMAIN%.xml (Optional)
  • 4.2. %APP_INSTALL_DIR%/isp/%EMAILDOMAIN%.xml (Optional)
For example:
  • 4.1. /home/fred/.config/yourapp/isp/example.com.xml
  • 4.2. /opt/yourapp/isp/example.com.xml
Other mechanisms You may want to implement other mechanisms to find a configuration, for example Exchange AutoDiscover, DNS SRV, or heuristic guessing. If you implement such alternative methods, and if they are less secure than some of the mechanisms provided here, the alternative methods SHOULD be considered only with lower priority (as defined above) than the more secure mechanisms defined here. For evaluating other mechanisms, use similar criteria as outlined in section "Security considerations".
Manual configuration If the above mechanisms fail to provide a working configuration, or if the user explicitly chooses so, you SHOULD give the end user the ability to manually enter a configuration, and use that configuration to configure the account.
Config validation
User approval Independent of the mechanisms used to find the configuration, before using that configuration, you SHOULD display that configuration to the end user and let him confirm it. While doing so:
  • At least the second-level domain name(s) of the hostnames involved MUST be shown clearly and with high prominence.
  • The client MUST NOT cut off parts of long second-level domains, to avoid spoofing. At least 63 characters MUST be displayed.
  • Care SHOULD be taken with international characters that look like ASCII characters, but have a different code.
Login testing After the user confirmed the configuration, you SHOULD test the configuration, by attempting a login to each server configured. Only if the login succeeded, and the server is working, should the configuration be saved and retrieving and sending mail be started.
OAuth2 windows If the configuration contains OAuth2 authentication, or any other authentication that uses a web browser with URL redirects, you MUST show the full URL or the second-level domain of the current page to the end user, at all times, including after page changes, URL changes or redirects. This allows the end user to verify that he is logging in at the expected page, e.g. the login server of their company. (Editor's note: Not really part of autoconfig, but autoconfig can enable OAuth2, and it's important that this is implemented correctly by mail applications.)
Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.
Alternatives considered
DNSSEC Due to their top-level domain, some domains do not have DNSSEC available to them, even if they would like to deploy it. Even where the top-level domain supports it, DNSSEC is currently deployed in only 1% of domains, with adoption rates falling instead of rising, due to the difficulties of administrating it correctly. Therefore, DNSSEC cannot be relied on in this specification, and DNS must be considered insecure for the purposes of this specification.
DNS SRV DNS SRV protocols (RFC 2782, RFC 6186) are not used here, for 2 reasons:
  1. DNS SRV relies on insecure DNS and the config can therefore be trivially spoofed by an attacker. See also DNSSEC above.
  2. DNS SRV does not provide all the necessary configuration parameters. For example, we need all of:
  • the username form ("fred@example.com", or "fred", or "fred\EXAMPLE", or even a username with no relation to the email address)
  • authentication method (password, CRAM-MD5, OAuth2, SSL client certificate)
  • authentication method parameters (e.g. OAuth parameters)
and other parameters. If any of these parameters are not configured right, the configuration won't work. While these parameters could theoretically be added to DNS SRV, that would mean a new specification and render the idea void that this is a protocol that already exists, is standardized and deployed. It is unlikely that all DNS SRV records would be updated with the new values. Therefore, it does not solve the problem. This specification was created as an answer to these deficiencies and provides an alternative to DNS SRV.
CAPABILITIES In the wild deployments from actual ISPs show that protocol-specific commands to find available authentication methods, e.g. IMAP CAPABILITIES or POP3 CAPA, are not reliable. Many email servers advertize authentication methods that do not work. Some IMAP servers are default configured to list all SASL authentication methods that have corresponding libraries installed on the system, independent on whether they are actually configured to work. The client receives a long list of authentication methods, and many of them do not work. Additionally, the server response may be only "authentication failed" and may not indicate whether the method failed due to lack of configuration, or because the password was wrong. Because some authentication servers lock the account after 3 failed login attempts, and it may also fail due to unrelated reasons (e.g. username form, wrong password, etc.), the client cannot blindly issue countless login attempts. Locking the account must be avoided. So, simply attempting all methods and seeing which one works is not an option for the email client. Additionally, some email servers advertize Kerberos / GSSAPI, but when trying to use it, the method fails, and also runs into a long 2 minute timeout in some cases. End users consider that to be a broken app. Additionally, such commands are protocol specific and have to be implemented in multiple different ways. Finally, some non-mail protocols may not support capabilties commands that include authentication methods.
Security Considerations
Risk If an attacker can provide a forged configuration, the provided mail hostname and authentication server can be controlled by the attacker, and the attacker can get access to the plain text password of the user. The attack can be implemented as man-in-the-middle, so the end user might receive mail as expected and never notice the attack. An attacker gaining the plain text password of a real user is a very significant threat for the organization, not only because mail itself can contain sensitive information and can be used to issue orders within the organization that have wide-ranging impact, but given single-sign-on solutions, the same username and password may give access to other resources at the organization, including other computers or, in the case of admin users, even adminstrative access to the core of the entire organization. Multi-factor authentication might not defend against such attacks, because the user may believe to be logging into his email and therefore comply with any multi-factor authentication steps required.
DNS Any protocol that relies on DNS without further validation, e.g. http, should be considered insecure. Even if an http URL redirects to a https URL, and the domain of the https URL cannot be validated against the email domain, that is still insecure. This also applies to the DNS MX lookup and the https calls that base on its results, as described in section "MX". Such insecure configs may only be used, if the end user confirms the config, particularly the resulting second-level domains. See section "User approval".
Config updates Part of the security properties of this protocol assume that the timeframe of possible attack is limited to the moment when the user manually sets up a new mail client. This moment is triggered by the end user, and a rare action - e.g. maybe once per year or even less, for typical setups -, so an attacker has limited chances to run an attack. While not a complete protection on its own, this reduces the risk significantly. However, if the mail client does regular configuration updates using this protocol, this security property is no longer given. For regular configuration updates, the mail client MUST use only mechanisms that are secure and cannot be tampered with by an active attacker. Furthermore, the user SHOULD still approve config changes. But even with all these protections implemented, the mail client vendor should make a security assessment for the risks of making such regular updates. The mail client vendor should consider that servers can be hacked, and most users simply approve changes proposed by the app, so these give only a limited protection.
Server security Given that mail clients will trust the configuration, the server delivering it needs to be secure. Even though we call it "database", static configuration files that are generated before deployment in combination with a static web server offer better security and significantly better performance than dynamic queries from a database and responses generated on-the-fly on request. If a custom server is used, it SHOULD be updated regularly and hosted on a dedicated secure server with all unnecessary services and server features turned off. Additions and modifications to the configurations are applicable to all respective users and must be made with care. The authenticity of the configuration MUST be verified from authorative sources. Server hostnames MUST be compared with the email domain names they are serving, and if they differ, the ownership of the server hostnames MUST be validated. For these reasons, mail clients may use the public mail config database mentioned above. The risk is mitigated to some degree by section "User approval".
IANA Considerations This document has no IANA actions.
Normative References Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings.
Acknowledgments TODO acknowledge.