MTA-STS and DANE: how mail transport security gets closed off
Nearly every mail server can do StartTLS: the server says it can do TLS, the sender says yes, and the connection is encrypted. But "can" is not "must" — an attacker who pushes one step lower along the way gets a plaintext connection where an encrypted one was intended, and no one sees it. Against that downgrade game two standards were built, and each says something different: MTA-STS is a policy, DANE is a promise on paper.
MTA-STS and DANE measures both sides for one domain in one go: the announcement and the policy of RFC 8461, laid rule for rule beside your own MX, and the TLSA pins of RFC 6698 compared against the certificate your mail server really presents — fetched with one STARTTLS handshake, without sending mail.
This article reads what each standard asks of you, where the pitfalls sit — redirects, max_age, certificate changes — and when a high grade really means something. The DNS read-actions around it, SPF and DMARC, stand in SPF, DKIM and DMARC; here it is about the connection itself.
1. MTA-STS: first an announcement, then a policy
MTA-STS (RFC 8461) consists of two parts. The first stands in DNS: exactly one TXT record at _mta-sts.yourdomain.com, with precisely v=STSv1 and an id — an ornamentless announcement that a policy exists. The second stands on HTTPS: the policy itself, at https://mta-sts.yourdomain.com/.well-known/mta-sts.txt, with three lines that matter: mode (none, testing or enforce), mx (which mail servers the policy allows) and max_age (how long a receiver may keep following the policy).
The details are the policy: the policy may not be reached via a redirect — RFC 8461 says redirects must not be followed, so a stray hop counts as a failure. max_age may not exceed 31,557,600 (one year). And the mx rules must match your real MX: a policy that does not name your own mail server is refused by a correct receiver — precisely the failure that lies open for downgrades. The tool lays all those rules beside your MX and names which rule stumbles.
2. DANE: a pin on your certificate
DANE for mail (RFC 6698) places one or more TLSA records at _25._tcp.mail.yourdomain.com. Such a record names four things: which role the certificate plays (usage), which part of the certificate is pinned (selector: the whole DER or the public key), how it is compared (matching: hash or raw bytes) and the fingerprint itself. usage 3 (DANE-EE) pins the server's own certificate — the form that can do without CA trust and is therefore the strongest.
usage 0–2 pins a certificate of a certification authority, and that is a different conversation: then you still trust the chain beneath it. A TLSA measurement is only a comparison when something stands opposite it: for that the tool writes exactly one protocol line on port 25 — STARTTLS — lets the TLS handshake run and lays the fingerprints of what the server presents beside each pin. No pin on a mail server? Then that mail server is not even approached: no socket, no measurement, n/b.
3. The pitfall: every certificate change is a DNS moment
A pin is a promise you manually honour. Let your certificate expire and replace it with another — then DNS holds a pin on a certificate that no longer exists, and every correct receiver following DANE will refuse your mail or fail on TLS. That is not a bug of DANE, that is the function: a pin that no longer fits, does not fit. After every certificate change the TLSA record must therefore be set again, and a certificate that rotates automatically (for instance Let's Encrypt with changing keys) asks for a plan: pin the public key that stays, or pin through selector and matching on something stable.
Max_age is the same moment in the other direction: too long a term (above a year) forbids the policy, too short (say under a day) makes every DNS change a race. Practically: test first — mode testing lets receivers read and report the policy without refusing mail — and then set enforce. And keep TLS-RPT on: there receivers report what their TLS attempts cost, so you see the policy working instead of hoping.
4. What the grade does and does not say
The tool judges what it measured: policy present and usable, mx rules that match your MX, pins that agree with the live certificate. An A means: what DNS promises agrees with what your mail server presents. It does not mean every sender follows the policy — MTA-STS is only guardrails where both sides support it, and DANE is truly enforced by only a handful of providers.
Start at the measurement side: MTA-STS and DANE for policy and pins, Mail security for what else stands in your mail DNS (MTA-STS reading, TLS-RPT, DKIM), and Mail ports to see whether port 25 reacts at all. Those three together are the entire TLS layer of your mail, and none of the three ever sends anything.