Agreements To Fix Forwarding

via L. Anelli 13 20122 Milano MI Italy vesely@tana.it

ARC DMARC DKIM Mailing List MLM Forwarding The widespread adoption of Domain-based Message Authentication, Reporting, and Conformance (DMARC) causes problems to indirect mail flows. This document proposes a protocol to establish forwarding agreements whereby a mailbox provider can whitelist indirect mail flows directed toward its users by maintaining per-user lists of agreements.

Introduction The ability to send messages to anyone without prior agreement is a feature at the base of the success of Internet mail. It also exposes a way to abuse, though. Domain-based email authentication can limit abusive behavior by enabling receivers to unambiguously attribute responsibility of messages, and thereby to develop domain reputation. However, indirect mail flows, such as mailing lists and aliases are seriously hindered by such authentication practices (see ). Although they constitute a minor part of the global mail traffic, such hindrance poses a problem, which is what the present protocol attempts to fix. Many mailing lists break DomainKeys Identified Mail (DKIM) signatures () by adding a subject tag or a message footer. Aliases break Sender Policy Framework (SPF) () if forwarders don't change the bounce address, or break its alignment with the From: header field if they do. In both cases, they break Domain-based Message Authentication, Reporting, and Conformance (DMARC) () if there is no DKIM signature. Rewriting the very From: field, an operation known as From: munging, is a sender side solution to this problem, adopted by most mailing lists; see for a detailed description. Aliases, instead, can be avoided by having the target pull messages using a mail retrieval protocol rather than pushing them via SMTP. As an alternative, this document proposes a cooperative method that hinges on the peculiarity that both of these forwarding methods require settings which produce well recognizable mail flows. To present by example, let's consider the procedure whereby alice@example.com subscribes to the mailing list participants@lists.example.org. The first three steps are well known practice. The extra steps are introduced by this protocol:

Alice subscribes to the list, either by visiting example.org web site or by sending a request to the list manager.
The mailing list manager (MLM) sends a confirm message with a unique token to be returned for confirmation, either via web or via mail.
Alice confirms the subscription.
The MLM looks up _fixforwarding.example.com on the DNS to check if Alice's provider supports forwarding agreements (see for the TXT record format). If found, the MLM applies for a permission to send mailing list messages that may fail DMARC checks (see for the web form it has to fill).
Upon receiving the MLM application, example.com sends a message to its user Alice, asking whether she agrees to receive that specific mail flow.
Alice confirms to her provider too.
Example.com adds participants@lists.example.org to the list of forwarders that Alice agreed upon. Then, it confirms to the MLM that the agreement is set up.
The MLM modifies Alice's subscription options removing From: munging for messages sent to her.
Example.com recognizes messages belonging to this mail flow by (1) authenticating example.org and (2) checking the List-ID: header field. As this matches Alice's list of exemptions, DMARC failures can be safely ignored.

Steps 4 to 9 are discussed in detail in the following sections. For software requirements, note that step 8 implies the ability to configure From: munging per subscriber; proposes a method to achieve such effect. Step 9 requires that the DMARC verifier be able to do per-user exemptions as described in

Terms Definitions The term forwarder is used to mean the entity that forwards messages, which is either a mailing list or an alias, according to SMTP (). A receiving domain is a domain which receives messages sent by a forwarder. Signers and verifiers are defined by DKIM (). The use of the term Mailing List Manager, almost always abbreviated MLM follows . A MLM is a kind of Mediator in parlance. It is usually composed of a Message Transfer Agent (MTA) with the usual suite of tools plus the mailing list specific software. Message is defined in . It consists of a header made up of one or more fields, and a body possibly composed of various MIME entities, the latter being defined in and companions. We use colon (:) to indicate header field names, as in From:, To: and the like. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Note About Rewriting the Bounce Address Traditionally, alias expansion re-injects messages in the mail system changing just the envelope recipient address. That is, without rewriting the bounce address, a.k.a. envelope sender. This is what SMTP prescribes. However, from an authentication perspective, this doesn't always work, because some MTAs reject SPF failures —those matching a -all directive (see )— during the early stages of the SMTP protocol; that is, before DKIM or ARC can be verified. It is handy to leave the bounce address intact in order to have the message author notified in case of failed delivery, which increases email reliability. However, nowadays MTAs try and minimize backscatter by avoiding to accept messages that can bounce. On the other hand, transient failures, such as mailbox full, became less and less frequent. If a delivery fails because the target mailbox was deleted, the forwarding setup has to be dismantled. A broken alias which unflaggingly generates bounces to any bounce address is akin to an open relay. Albeit limited to non-delivery notifications, it can be exploited for spamming. In order to avoid playing such role, a receiver can carefully verify the bounce address before accepting a message, so expect your target to behave accordingly. We recommend to replace the bounce address with the internal address of a human or a robot which can understand what to do. Nevertheless, this protocol provides for a way to enter into an agreement which covers traditional alias expansion. According to , domains consult a whitelist if they reject SPF "fail" messages before receiving the message data. If the whitelist they consult is a public DNS-based whitelist (DNSWL, ) which includes the forwarder's IP address, then traditional alias expansion can work even in the face of strict SPF policies. To recap, an MTA can check SPF after the MAIL but before the DATA SMTP commands. If the result is "fail", the MTA consults one or more DNSWLs. If the sending IP address is whitelisted, then the MTA delays the decision to reject to a later stage, when the message content is received and DKIM or ARC verified. Thus, an agreement providing for traditional alias expansion is possible if either one of the following two conditions holds:

The target MXes consult public DNSWLs and the forwarder is subscribed to at least one of them, or
the target MXes never reject before receiving and evaluating the message content.

If and which of these conditions holds can be determined based on the dnswl= tag declaration, as described in .

Underscored FixForwarding DNS Resource Record Domains participating in this protocol notify it to all interested parties by publishing an "underscored" resource record named _fixforwarding. According to , this record is labeled as a subdomain of the domain it refers, which is the domain-part of the pertinent email addresses. The record contains the conditions that an applicant MUST meet before applying for an agreement as well as an URI to post requests to. Requests details are specified in . The format of the _fixforwarding record follows the extensible "tag-value" syntax for DNS-based key records defined in DKIM . The folding white space terminal, FWS, is also defined there. The following tags are defined here:

auth: Authentication method used to identify the forwarder's domain; OPTIONAL, by default the method is ARC (, but DKIM is allowed as an alternative. A forwarder MUST implement the required method before applying. ABNF: rec-auth-tag = %s"auth" [FWS] "=" [FWS] rec-auth-tag-method rec-auth-tag-method = %s"arc" / %s"dkim"
dnswl: Either a comma-separated list of DNSWLs (see ) that the domain's MXes consult before rejecting SPF failures at the early stages of SMTP transactions, or the keyword "n.a.", for not applicable; OPTIONAL, by default an empty list. See . An empty list, the default, means this option is not available; that is, forwarders MUST change the bounce address to an SPF valid one. The "n.a." keyword means the opposite; that is, forwarders MAY leave the original bounce address intact. One or more domains mean the conditional acceptance; that is, the DNS name that results by prepending the reverse IP address of the forwarder to one of those domains resolves to an A IP address when queried. ABNF: rec-dnswl-tag = %s"dnswl" [FWS] "=" [FWS] [rec-dnswl-tag-opt] rec-dnswl-tag-opt = rec-dnswl-tag-list / rec-dnswl-tag-na rec-dnswl-tag-list = domain *("," domain) rec-dnswl-tag-na = %s"n.a." Where domain is imported from . Each domain of the list is a DNS zone that can be queried by prepending reversed IP addresses. For best interoperability, it is RECOMMENDED to use organizations that admit free subscriptions.
post: This is the URI to post web form requests to. Web format is described in . This is an HTTPS or HTTP URI.ABNF: rec-post-tag = %s"post" [FWS] "=" [FWS] rec-post-tag-uri rec-post-tag-uri = https-URI / http-URI Where https-URI and http-URI are imported from .

Forwarding Agreement Web Form Receiving domains participating in this protocol set up a web form whereby forwarders can apply for a permission to send messages that may fail DMARC checks. Forwarders who wish to enter an agreement with the receiving domain fill the form supplying the values requested, chiefly email address contacts. As there is a predefined set of fields, applicants can prepare a script which runs automatically when the availability of the form is detected by the _fixforwarding resource record, and the conditions therein are met. Yet, receiving domains SHOULD provide at the same post= URL an HTML form to be filled manually, so that it is possible to enter an agreement also without developing a special script. When no value is posted the form displays the empty fields to be filled; when values are posted it displays the results. Posted values can be encoded using either the application/x-www-form-urlencoded or the multipart/form-data media types (see ). Receiving domains SHOULD check that the values posted are acceptable and return a 400 HTTP status code otherwise. If everything is fine, the HTTP server stores the values for later processing and returns status code 202. The resulting agreement status will be sent to the base address of the applicant after checks are completed. Keep in mind that this may take a human lapse of time.

The Transistor Metaphor Three email addresses play a key role, the entry point where the messages to be forwarded arrive, the target address which is the subject of the agreement, and the base address where the state of the agreement is sent, including the results of the form submission. Following a transistor metaphor we name these fields collector, emitter and base respectively.

Form Fields

abuse: The abuse-team or similar entity email address, where complaints about perceived forwarder's misbehavior can be sent.
agreement-id: A globally unique string that identifies a request and the related agreement. This has the syntax of a Message-ID field and is RECOMMENDED that the right-hand side contain the same domain identifier used for signatures, if it is able to guarantee the uniqueness of the left-hand side within itself. Angle brackets not to be included.
base: The forwarder's email address where the receiving domain sends the agreement acceptance, rejection, renewal or cancellation. These are machine-readable email messages that may contain a human-readable part. The message format is described in .
collector: The mailing list post address, or the address that the alias is attached to.
domain: The forwarder's domain to be authenticated by the receiver. It is the domain indicated in the d= tag of DKIM-Signature: or ARC-Seal: and ARC-Message-Signature: header fields, according to the auth method (). The domain MUST match the trailing part of the list-id. It is suggested that the two identifiers match as much as practical.
emitter: The user-confirmed email address that subscribed to the mailing list, or the target address of an alias.
list-id: List-ID:s are naturally present in mailing list, and identify the list that the user subscribed to. For aliases, a list-id MUST be created, for example by concatenating an encoded representation of the collector and the signing domain. This is the globally unique list-id identifier, which MUST be included in a List-Id: header field, in angle brackets as specified by , in every forwarded message. This field is needed to unequivocally identify the mail flow. The trailing part of the list-id MUST match the domain. The List-Id: header field SHOULD be signed. Failure to do so can be exploited to damage MLM reputation by replaying suitable messages.

Agreement Management Request processing involves interaction with the user, therefore the result can become available after some time. For honest requests, the user is expected to confirm. Asking for user confirmation can also be an occasion to manage mailbox details such as what folder or label would the user like to associate with the mail flow. Mailbox providers can also implement a web application that lists the status of all forwarding agreements. It would be somewhat more reliable than the monthly subscription reminders, which some call a distributed in time database. The receiving domain MAY accept or reject the agreement. If it accepts, it stores the form values that the forwarder supplied, so as to be able to manage the agreement. The agreement-id is a good candidate for the filename or database key used to retrieve that data. Keep in mind that there is an agreement for each <collector, list-id> pair. To honor the agreement, a DMARC filter only needs two items of the agreement data, the emitter and the list-id. For ARC, the filter checks the validity of the chain and the validity of the ARC-Message-Signature: whose d= domain matches the list-id (when oldest-pass is set, it must not be bigger than the instance, i= of the list-id signer). For DKIM, a valid signature with d= matching the list-id assures of the message origin. Next, the DMARC filter checks whether the pair <recipient, list-id> corresponds to an active agreement. If the message is thus found to be part of an agreed upon mail flow, the DMARC filter MUST exempt it from undergoing the DMARC policy published by the message's From: domain. If the receiving domain sends aggregate reports, the messages thus exempted SHALL be counted under the trusted_forwarder PolicyOverrideType. The receiving domain MAY extend the exemption to other recipients. Otherwise, if a message is destined to more recipients, each one must be the subject of a forwarding agreement. After setting up DMARC exemptions, the receiving domain sends an acceptance email to the base address of the forwarder. From that moment, the forwarder can stop From: munging messages destined to the relevant subscriber. The agreement can last indefinitely. However, at any time, if the receiving domain suspects that the mail flow is not active any more, it MAY send a renewal notice to check. The agreement MUST be honored until the receiving domain sends a cancellation. See for the format of these messages. The agreement SHOULD be canceled after the user unsubscribes from the mailing list or when the mailbox is torn down. Conversely, a cancellation should imply removal, or at least questioning the forwarding itself, whether mailing list or alias.

Messages to the Base There are four types of messages that can be sent by the receiving domain to the forwarder base address. These messages define the state of the forwarding agreement. Such state is determined at the incontestable discretion of the receiving domain. The forwarder SHOULD reply to these messages as a means to acknowledge their reception. If the forwarder doesn't have any reference about the agreement, it MAY bounce the message or give a negative reply to it. The receiving domain SHOULD re-send the message if a reply or a bounce is not received within a reasonable time (days). The message types are as follows:

acceptance: The receiving domain accepts the agreement. From the time this message is received onward, messages can avoid From: munging or, if allowed by the receiving domain, bounce address rewriting.
rejection: The receiving domain, presumably after user disavowal, denies the agreement. Failed authentications can then be rejected or quarantined. The forwarding itself should be questioned. A new agreement SHOULD NOT be requested again unless further interactions occur, such as the user unsubscribing and re-subscribing.
renewal: The receiving domain needs a confirmation that this forwarding mechanism is still on. The forwarder should check that the mechanism implied by the agreement is still active before replying. Failure to reply can result in the agreement cancellation.
cancellation: The agreement is canceled. For mailing lists, the user should have already unsubscribed. The alias should be removed.

These messages are plain text messages with no attachments and no MIME multipart entities. The agreement-id and the message type are repeated in the Subject: and in the beginning of the body. The rest of the body MAY contain any text. ABNF: msg-subject = msg-tag WSP agreement-id ":" [WSP] msg-type msg-tag = "[FixForwarding]" msg-body = agreement-id-line msg-type-line msg-rest agreement-id-line = [CRLF] "agreement-id" ":" [WSP] agreement-id CRLF msg-type-line = [CRLF] "deal" ":" [WSP] msg-type CRLF msg-rest = *OCTET msg-type = %s"acceptance" / %s"rejection" / %s"renewal" / %s"cancellation" agreement-id = id-left "@" id-right subject = "Subject" ":" msg-subject CRLF message = fields CRLF msg-body Where WSP, CRLF and OCTET are imported from ; subject, id-left, id-right and fields from . Replies SHALL also be text/plain, retaining the Subject: and body of the original message, possibly prefixed as is customary for Internet mail, for example by prefixing the subject with "Re:" and body text lines with "> ". Replies MUST properly set In-Reply-To: and References: fields. The receiving domain can use those to match replies with the original message. For negative replies which are not bounces, the first word in the body, and possibly the Subject: prefix, MUST be "NO". ABNF: negative-reply = fields CRLF %s"NO" CRLF msg-body Both messages and replies MUST be duly authenticated. Negative replies are sent when the forwarder has no knowledge of the agreement-id. That means the agreement was previously canceled, never properly archived and in any case it is not active. After a negative reply, the receiving domain can cancel the agreement without further notice. Replies which are not negative replies are positive acknowledgments of the deal. Replies to cancellation and rejection have the same effect whether positive or negative, and both sides can remove any reference to the agreement afterwards.

IANA Considerations Per , please add the following entry to the "Underscored and Globally Scoped DNS Node Names" registry:

Privacy Considerations Forwarding agreements expose users' mailing list participation and forwarded identities to their mailbox provider admins. That is unavoidable. It has to be noted, though, that such knowledge can hardly be kept secret from people who have access to all their mail. Users need to trust their mailbox provider, and this protocol is not the only reason why they have to. Still, in situations where the forwarder wants to hide its forwarding from the receiving domain, it SHOULD NOT request a forwarding agreement. Instead, rewriting bounce address and From: header fields, besides bettering deliverability, may also help hiding the true origin of messages.

Security Considerations Maintaining per-user DMARC exemptions excludes gaming, especially for domains that provide mailboxes to anonymous users. Domains that restrict mailboxes to internal users can balance their knowing of the user involved in an agreement request and the reputation of the applicant to decide whether to trust the forwarder for all users rather than just for the one at hand. The forwarder doesn't know about this decision, albeit it may guess it by the speed characterizing the acceptance of further agreements. This kind of decision slightly simplifies DMARC filter settings, but not agreements management. Forwarders SHOULD verify DMARC and apply author domain policies whenever possible. Especially mailing lists, unless they collect posts from other mailing lists in turn, should reject messages from domains who publish p=reject if authentication fails. Receiving domains MUST NOT reject messages belonging to an accepted agreement for DMARC policy reasons. DMARC Filtering MUST be applied by the forwarder. Failure to do so SHOULD be reported to the forwarders abuse address. Forwarders MUST also apply some content filtering. They must reject or drop blatant spam. MLMs that provide for moderation can treat dubious messages that way. Aliases may need to forward dubious messages, since they cannot resort to spam folders or similar quarantine measures. The receiving domain MAY reject or quarantine messages whose content doesn't meet its policies, under its responsibility. While mailing lists exercise some control on the quality of messages, aliases are completely transparent. When bad messages belong to a forwarding agreement, their quality is not to be ascribed the forwarder's reputation but the author domain's.

References Normative References Informative References

No-Munging Method This protocol assumes that From: munging can be enabled or disabled on a per-user basis. Many MLMs provide instead an option to enable or disable it on a per-list basis. Here's a way to overcome this limitation. Have an umbrella list with two sibling sub-lists, one configured with From: munging, the other one without. Both sub-lists refuse all posts, and advertise the umbrella list as the destination for posts. The umbrella list accepts posts according to site and list policy. It has the sibling sub-lists as its only subscribers, and won't accept more. The sibling sub-list that does From: munging accepts subscribers under the site and list policy. When forwarding agreements are accepted, the relevant subscribers are moved to the other sub-list.