In a few places, we call Printf-like functions, but for the format we
use either non-format messages (which is not tidy, but okay), or
variable messages (which can be problematic if they contain %-format
directives).
The patch fixes the calls by either moving to Print-like functions, or
using `Printf("%s", message)` instead.
These were found by a combination of `go vet` (which complains about
"non-constant format string in call"), and manual inspection.
staticcheck found a couple of minor code cleanup improvements, like
unused variables or an out-of-order defer, mostly in tests.
This patch fixes those problems by making the necessary adjustments.
They're all fairly small, and should not change the logic in any
significant way.
When constructing the "Received" header, in some cases we want to
include the remote IP address in addition to the EHLO domain.
The way we did that is not fully compliant with RFC 5321 (section 4.4),
and this has the potential to confuse some tools that parse the header.
This patch fixes this problem by adjusting the order of the two pieces
of data, which makes it comply with the RFC.
Before:
Received: from [1.2.3.4] (ehlo.domain.example.com)
After:
Received: from ehlo.domain.example.com ([1.2.3.4])
Thanks to nolanl@github for reporting this problem in
https://github.com/albertito/chasquid/issues/76.
This commit updates the uses of math/rand to math/rand/v2, which was
released in Go 1.22 (2024-02).
The new package is generally safer, see https://go.dev/blog/randv2 for
the details.
There are no user-visible changes, it is only adjusting the name of
functions, simplify some code thanks to v2 having a better API, etc.
The RFCs are very clear that in DATA contents:
> CR and LF MUST only occur together as CRLF; they MUST NOT appear
> independently in the body.
https://www.rfc-editor.org/rfc/rfc5322#section-2.3https://www.rfc-editor.org/rfc/rfc5321#section-2.3.8
Allowing "independent" CR and LF can cause a number of problems.
In particular, there is a new "SMTP smuggling attack" published recently
that involves the server incorrectly parsing the end of DATA marker
`\r\n.\r\n`, which an attacker can exploit to impersonate a server when
email is transmitted server-to-server.
https://www.postfix.org/smtp-smuggling.htmlhttps://sec-consult.com/blog/detail/smtp-smuggling-spoofing-e-mails-worldwide/
Currently, chasquid is vulnerable to this attack, because Go's standard
libraries net/textproto and net/mail do not enforce CRLF strictly.
This patch fixes the problem by introducing a new "dot reader" function
that strictly enforces CRLF when reading dot-terminated data, used in
the DATA input processing.
When an invalid newline terminator is found, the connection is aborted
immediately because we cannot safely recover from that state.
We still keep the internal representation as LF-terminated for
convenience and simplicity.
However, the MDA courier is changed to pass CRLF-terminated lines, since
that is an external program which could be strict when receiving email
messages.
See https://github.com/albertito/chasquid/issues/47 for more details and
discussion.
The aliases.Resolver.Exists function currently returns the "clean"
address (with the drop characters and suffixes removed), which is relied
upon in its only caller.
That, however, makes the logic more difficult to follow, hiding some
of the address manipulation behind what should be a read-only check.
So this patch reorganizes that code a little bit, removing the
"cleaning" of the address as part of Exists, and making it explicit when
needed instead.
This patch does not have any user-visible change in behaviour, it is
just internal reorganization.
This is in preparation for further patches which will improve the
handling of some aliases corner cases.
This patch changes several internal packages to receive and pass tracing
annotations, making use of the new tracing library, so we can have
better debugging information.
ioutil package was deprecated in Go 1.16, replace all uses with their
respective replacements.
This patch was generated with a combination of `gofmt -r`, `eg`, and
manually (for `ioutil.ReadDir`).
Today, we close the connection after 10 errors. While this is fine for
normal use, it is unnecessarily large.
Lowering it to 3 helps with defense-in-depth for cross-protocol attacks
(e.g. https://alpaca-attack.com/), while still being large enough for
useful troubleshooting and normal operation.
As part of this change, we also remove the AUTH-specific failures limit,
because they're covered by the connection limit.
When we receive unknown commands, we use the first 6 bytes for
troubleshooting (e.g. put them in traces and exported metrics).
While this is safe, since the different places know how to quote them
properly, it makes things more difficult to analyse, since it's not
uncommon to see be binary blobs.
This patch makes us use the ascii-quoted version instead, to make things
easier to analyze.
When we fail to check if a user exists, we currently return a permanent
error, which can be misleading and also make things more difficult to
troubleshoot.
This patch makes chasquid return a temporary error in that case.
Thanks to Thor77 (thor77@thor77.org) for suggesting this change.
This patch implements support for incoming connections wrapped in the
HAProxy protocol v1.
This is useful when running chasquid behind a HAProxy server, as it
needs the original source IP to perform SPF checks.
This patch is a reimplementation of one originally provided by Denys
Vitali in pull request #15, except the logic for the protocol handling
is moved to a new package, and the smtpsrv.Conn handling of the source
IP is simplified.
It is marked as experimental for now, since we want to give it a bit
more exposure just in case the option/api needs adjustment.
Thanks a lot to Denys Vitali (@denysvitali in github) for sending the
original patch for this, and helping test it!
Some utilities might want to access the EHLO/HELO domain in the
post-data hook (for example, to do additional SPF validations).
This patch implements that support, including sanitizing the EHLO domain
on the environment variable to reduce the risk of problems.
The EHLO parameter is generally referred to as "domain", even though it
can take either a domain or an address.
For clarity, rename the variable and comments to match.
This is stylistic only, there are no functional changes.
This patch makes chasquid's monitoring server expose an OpenMetrics
metrics endpoint.
It adds a new package "expvarom" which implements an HTTP handler that
exports expvar variables in the OpenMetrics text format.
Then, the handler is registered by the monitoring server at /metrics
(where most things expect it to be).
The existing exported variables are also extended with descriptions,
which is optional, but improves the readability of the metrics.
When we can't authenticate due to a transient issue, for example if we
rely on Dovecot and it is not responding, we should use a differentiated
error code to avoid confusing users.
However, today we return the same error code as when the user enters the
wrong password, which could confuse users as their MUA might think their
credentials are no longer valid.
This patch fixes the issue by returning a differentiated error code in
that case, as per RFC 4954.
Thanks to Max Mazurov (fox.cpp@disroot.org) for reporting this problem.
The linter complains that we're not checking for errors, but on some
cases it's on code paths were it is reasonable to do so (e.g. we're
closing the connection and it's a best-effort write).
This patch adjusts the code to make those cases explicit.
When receiving a message on a TLS socket, we currently don't check the
Handshake result, so connections often fail in a way that is not easy to
troubleshoot.
This patch fixes that by checking the result and emitting a nicer error
message before closing the connection.
When the client closes the connection, which is very common, chasquid
logs it as an error ("exiting with error: EOF").
That can be confusing and mislead users, and also makes a lot of
traces be marked as errored, when nothing wrong occurred.
So this patch changes the log to not treat it as an error.
When the DATA input is too large, we should keep on reading through it
until we reach the end marker, otherwise there is a security problem:
the remaining data will be interpreted as SMTP commands, so for example
a forwarded message that is too long might end up executing SMTP
commands under an authenticated user.
This patch implements this behaviour, while being careful not to consume
extra memory to avoid opening up the possibility of a DoS.
Note the equivalent logic for single long lines is already implemented.
Currently, there is no limit to incoming line length, so an evil client
could cause a memory exhaustion DoS by issuing very long lines.
This patch fixes the bug by limiting the size of the lines.
To do that, we replace the textproto.Conn with a pair of buffered reader
and writer, which simplify the code and allow for better and cleaner
control.
Thanks to Max Mazurov (fox.cpp@disroot.org) for finding and reporting
this issue.
If we fail to put the message in the queue (e.g. because we're out of
storage space, or the aliases-resolve hook errored out), it should be
considered a transient failure.
Currently we return a permanent error, which is misleading, as we want
clients to retry in these situations.
So this patch changes the error returned accordingly.
The spf library has gained support for macros, but to process them
properly, a new function needs to be called with the full sender
address, spf.CheckHostWithSender.
This patch updates chasquid's calls to the new API.
There are a few context.WithDeadline calls that can be simplified by
using context.WithTimeout.
At the time they were added, WithTimeout was too new so we didn't want
to depend on it. But now that the minimum Go version has been raised to
1.9, we can simplify the calls.
This patch does that simplification, which is purely mechanical, and
does not change the logic itself.
When handling a connection, today we only set a deadline after each
command received.
However, this does not cover our initial greeting, or the initial TLS
handshake (if the socket is TLS), so a connection can hang
indefininitely at that stage.
This patch fixes that by setting a deadline earlier, before we send or
receive anythong on the connection.
Despite its loose appearance, the "Received" header has a reasonably
standarized format.
We were not following the standard format as closely as we should; this
rarely causes problems in this particular case, but there's no need to
deviate from it.
This patch changes the Received header generation as follows:
- The "from" section now uses the remote address as canonical (for
non-authenticated users) which provides more valuable information
than the user-supplied EHLO address (which is also included).
- The remote authenticated user is now hidden, for additional privacy.
- Use the "with" optional clause.
- Use the standard way of printing TLS cipher suite.
- Use the standard way of printing address literals.
This patch adds a missing docstrings for exported identifiers, and
adjust some of the existing ones to match the standard style.
In some cases, the identifiers were un-exported after noticing they had
no external users.
Besides improving documentation, it also reduces the linter noise
significantly.
This patch implements an Authenticator type, which connections use to
do authentication and user existence checks.
It simplifies the abstractions (the server doesn't need to know about
userdb, or keep track of domain-userdb maps), and lays the foundation
for other types of authentication backends which will come in later
patches.
For direct TLS connections, such as submission-over-TLS, we currently
don't get the TLS information so it appears in the headers as "plain
text", which is misleading.
This patch fixes the problem by getting the connection information
early. Note it explicitly triggers the handshake, which would otherwise
happen transparently on the first read/write, so we can use the hostname
(if any) in our hello message.
This patch adds support for TLS-wrapped submission connections.
Instead of clients establishing a connection over plain text and then
using STARTTLS to switch over a TLS connection, this new mode allows the
clients to connect directly over TLS, like it's done in HTTPS.
This is not an official standard yet, but it's reasonably common in
practice, and provides some advantages over the traditional submission
port.
The default port is 465, commonly used for this; chasquid defaults to
systemd file descriptor passing as for the other protocols (for now).
When testing, we don't want the server to do SPF lookups, as those cause
real DNS queries which can be problematic and add a dependency on the
environment.
This patch adds an internal boolean to disable the SPF lookups, which is
only set from the tests.
The loop test can be quite slow, specially on computers without
cryptography-friendly instructions.
This patch introduces a new flag for testing, so that we can bring the
threshold down to 5. The test is just as useful but now runs in a few
seconds, as opposed to a few minutes.
This patch makes the hooks always have a complete set of environment
varuables, set to 0/1 or whatever is appropriate, to make it easier to
write the checks for them.
It is can be convenient for hooks to indicate that an error is
permanent; for example if the anti-virus found something.
This patch makes it so that if the hook exits with code 20, then it's
considered permanent. Otherwise it is considered transient, to help
prevent accidental errors cause final delivery issues.
The default INFO logs are more oriented towards debugging and can be
a bit too verbose when looking for high-level information.
This patch introduces a new "maillog" package, used to log messages of
particular relevance to mail transmission at a higher level.
This patch implements a post-DATA hook, which is run after receiving the
data but before sending a reply.
It can be used to implement content filtering when receiving email, for
example for passing the email through an anti-spam or an anti-virus.
Unknown commands can fill the logs, traces and expvars with a lot of
noise; this patch sanitizes them a bit down to 6 bytes, as a compromise
to maintain some information for troubleshooting.
The submission port is expected to be used only by authenticated
clients, so this patch makes chasquid enforce this, which also helps
to reduce spam.
https://www.rfc-editor.org/rfc/rfc6409.txt
Today, we pick the domain used to send the DSN from based on what we
presented to the client at EHLO time, which itself may be based on the
TLS negotiation (which is not necessarily trusted).
This is complex, not necessarily correct, and involves passing the
domain around through the queue and persisting it in the items.
So this patch simplifies that handling by always using the main domain
as specified by the configuration.
This patch moves chasquid's Server and Conn structures to their own
smtpsrv package, to make chasquid.go a bit more readable. It also helps
clarify the relation between Server and Conn.
There are no functional changes.
Note that git can still track the history across this commit (e.g. git
gui blame shows the right data).
