This patch is the result of running Go 1.19's `gofmt` on the codebase,
which automatically updates all Go doc comments to the new format.
https://tip.golang.org/doc/go1.19#go-doc
We've accumulated a few linter issues around comments and a couple of
variable names.
While none of them is major, this patch cleans them up so it's easier to
go through the linter output, and we can start being more strict about
it.
This patch does a general pass updating Go modules to recent versions, and
regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
This patch implements support for catch-all aliases, where users can add
a `*: destination` alias. Mails sent to unknown users (or other aliases)
will not be rejected, but sent to the indicated destination instead.
Please see https://github.com/albertito/chasquid/issues/23 and
https://github.com/albertito/chasquid/pull/24 for more discussion and
background.
Thanks to Alex Ellwein (aellwein@github) for the alternative patch and
help with testing; and to ThinkChaos (ThinkChaos@github) for help with
testing.
This patch does a general pass updating Go modules to recent versions, and
regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
This patch skips the resolution logic if the address is not local.
Today, the resolution logic handles that case transparently, and returns
the original email address, so this should be a no-op.
However, having an explicit early check makes the resolution logic more
robust, and will simplify future patches.
Note this also means that the `alias-resolve` hook is no longer run for
non-local aliases, which should also help simplify their implementation.
This patch simplifies the internal alias lookup logic, unifying it
across Resolve and Exists.
As part of this, the `alias-exists` hook is removed. It was redundant to
begin with, although it enabled a potential optimization, it isn't worth
the complexity. The timeout for execution of both was the same.
This change should be backwards-compatible because `alias-resolve` is
still used, and the semantics haven't changed.
If the `drop_characters` or `suffix_separators` options are set to "",
currently instead of the empty string, their default value is used instead.
This is a bug, and it also happens on other config options, but because
the others have to be set in order for chasquid to function, it's not a
problem in practice.
Thanks Björn Busse (bbusse@github) for finding and reporting this
problem, on irc and in https://github.com/albertito/chasquid/issues/25.
This patch fixes the problem by marking these fields explicitly
optional, which enables presence testing, as described in the protobuf
documentation:
https://github.com/protocolbuffers/protobuf/blob/master/docs/field_presence.md.
This patch adds tests for STS policy checks in combination with TLS
security levels.
This helps ensure we're detecting mis-matches of TLS status
(plain/insecure/secure) and STS policy enforcement.
When using STARTTLS, the SMTP courier needs to determine whether the
server certificates are valid or not.
Today, that's implemented via connecting once with full certificate
verification, and if that fails, reconnecting with verification
disabled.
This works okay in practice, but it is slower on insecure servers (due
to the reconnection), and some of them even complain because we connect
too frequently, causing delivery problems. The latter has only been
observed once, on the drv-berlin-brandenburg.de MX servers.
To improve on that situation, this patch makes the courier do the TLS
connection only once, and uses the verification results directly.
The behaviour of the server is otherwise unchanged. The only difference
is that when delivering mail to servers that have invalid certificates,
we now connect once instead of twice.
The tests are expanded to increase coverage for this particular case.
When resolving MX records, we need to distinguish between "no such
domain" and other kinds of errors. Before Go 1.13, this was not
possible, so we had a workaround that assumed any permanent error was a
"no such domain", which is not great, but functional.
Now that our minimum supported version is Go 1.15, we can remove the
workaround.
This patch replaces the workaround with proper logic using
DNSError.IsNotFound to identify NXDOMAIN results when resolving MX
records.
This requires to adjust a few tests, that used to work on environments
where resolving unknown domains (used for testing) returned a permanent
error, and now they no longer do so. Instead of relying on this
environmental property, we make the affected tests use our own DNS
server, which should make them more hermetic and reproducible.
tls.BuildNameToCertificate has been deprecated, and calling it is no
longer necessary since Go 1.14.
Now that our minimum supported Go version is 1.15, we can remove it.
By default, golang.org/x/net/trace currently only allows the tracing
pages to be seen from localhost.
This restriction can be confusing for people accessing the monitoring
server remotely, and adds no value in our environment.
The monitoring server already exports very sensitive information, and
must be enabled with care, and is not on by default. This is well
documented.
This patch removes the restriction, making all the monitoring pages
equally accessible.
Some deployments already have users that authenticate without a domain.
Today, we refuse to even consider those, and reject them at parsing time.
However, it is a use-case worth supporting, at least with some
restrictions that make the complexity manageable.
This patch changes the auth package to support authenticating users
without an "@domain" part.
Those requests will always be directly passed on to the fallback
authenticator, if available.
The dovecot fallback authenticator can already handle this case just fine.
The openmetrics proposed standard says we should use the
`application/openmetrics-text` content type when exporting the metrics.
Currently we use `text/plain` for backwards compatibility with
Prometheus, but the new content type is apparently supported since 2018,
so it should be safe to update to match the current proposed standard.
Today, we close the connection after 10 errors. While this is fine for
normal use, it is unnecessarily large.
Lowering it to 3 helps with defense-in-depth for cross-protocol attacks
(e.g. https://alpaca-attack.com/), while still being large enough for
useful troubleshooting and normal operation.
As part of this change, we also remove the AUTH-specific failures limit,
because they're covered by the connection limit.
When we receive unknown commands, we use the first 6 bytes for
troubleshooting (e.g. put them in traces and exported metrics).
While this is safe, since the different places know how to quote them
properly, it makes things more difficult to analyse, since it's not
uncommon to see be binary blobs.
This patch makes us use the ascii-quoted version instead, to make things
easier to analyze.
This patch adds tracing for the auth and domaininfo modules. In the
latter, we replace the long-running event with the short-term request
tracing, which is more practical and useful.
There are no logic changes, it only adds tracing instrumentation to help
troubleshooting.
This patch does a general pass updating Go modules to recent versions,
and regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
We also regenerate protobufs since the newer versions of the liberaries
have a much cleaner dependency tree, which speeds up fetches.
When we fail to check if a user exists, we currently return a permanent
error, which can be misleading and also make things more difficult to
troubleshoot.
This patch makes chasquid return a temporary error in that case.
Thanks to Thor77 (thor77@thor77.org) for suggesting this change.
Currently, chasquid attempts to auto-detect dovecot sockets when
starting up (if needed). If autodetection fails, chasquid emits an
error, continues serving, and never tries again.
This can be problematic if chasquid starts up before dovecot, and at the
time the dovecot sockets are not present (e.g. after a reboot). In that
case, chasquid will not use dovecot for authentication even after
dovecot has started.
This patch changes the autodetect logic, by doing autodetection at
startup and on each request, until we find a working pair of sockets.
Once we do, they're used consistently.
That way, if dovecot is not ready when chasquid starts, it's not a
problem and chasquid will start using dovecot once it becomes available.
Thanks to Thor77 (thor77@thor77.org) for reporting and helping
troubleshoot this issue.
This patch implements support for incoming connections wrapped in the
HAProxy protocol v1.
This is useful when running chasquid behind a HAProxy server, as it
needs the original source IP to perform SPF checks.
This patch is a reimplementation of one originally provided by Denys
Vitali in pull request #15, except the logic for the protocol handling
is moved to a new package, and the smtpsrv.Conn handling of the source
IP is simplified.
It is marked as experimental for now, since we want to give it a bit
more exposure just in case the option/api needs adjustment.
Thanks a lot to Denys Vitali (@denysvitali in github) for sending the
original patch for this, and helping test it!
This patch renames courier.Procmail to courier.MDA, to make it more
obvious that the functionality is not tied to that particular MDA.
It's just for readability, there are no functional changes.
Some utilities might want to access the EHLO/HELO domain in the
post-data hook (for example, to do additional SPF validations).
This patch implements that support, including sanitizing the EHLO domain
on the environment variable to reduce the risk of problems.
The EHLO parameter is generally referred to as "domain", even though it
can take either a domain or an address.
For clarity, rename the variable and comments to match.
This is stylistic only, there are no functional changes.
The queue protobuf definition currently uses the well-known timestamp
protobuf package.
This adds a build-time dependency on it, which is fairly harmless when
building from source (since the golang protobuf compiler includes it
already), but adds overhead for packaging on distributions.
Since this is the only external proto dependency we have, and the
protobuf message itself is trivial, this patch removes it an instead
embeds a compatible definition.
That way we remove the dependency and simplify packaging, with almost
negligible code overhead.
The change is fully backwards compatible and has no functional changes.
This patch makes chasquid's monitoring server expose an OpenMetrics
metrics endpoint.
It adds a new package "expvarom" which implements an HTTP handler that
exports expvar variables in the OpenMetrics text format.
Then, the handler is registered by the monitoring server at /metrics
(where most things expect it to be).
The existing exported variables are also extended with descriptions,
which is optional, but improves the readability of the metrics.
This patch makes protoio use the new protobuf API for
marshalling/unmarshalling text protobufs, as well as extends the tests
to cover marshalling failures.
The protobuf text output is not stable/deterministic and some spaces are
added randomly, so some integration tests have to be adjusted to account
for it.
This patch adds support for writing maillog to stdout and stderr, which
can be desirable in certain environments.
Thanks to Denys Vitali <denys@denv.it> who sent an alternative patch for
this functionality.
When we can't authenticate due to a transient issue, for example if we
rely on Dovecot and it is not responding, we should use a differentiated
error code to avoid confusing users.
However, today we return the same error code as when the user enters the
wrong password, which could confuse users as their MUA might think their
credentials are no longer valid.
This patch fixes the issue by returning a differentiated error code in
that case, as per RFC 4954.
Thanks to Max Mazurov (fox.cpp@disroot.org) for reporting this problem.
This makes it possible to manage chasquid logs using logrotate.
Amended-by: Alberto Bertogli <albertito@blitiri.com.ar>
Added tests, minor style and comment changes.
In preparation for supporting log rotation, this patch makes the maillog
package to use blitiri.com.ar/go/log instead of its own writer.
Some of the tests are made more strict, to better test the log format.
Amended-by: Alberto Bertogli <albertito@blitiri.com.ar>
Fixed build, extended commit message, adjusted to the log options
API, and added tests.
The output of the alias-exists hook is unused, so currently it's
discarded silently.
However, it can be very useful to debug issues when the hook is not
working as expected.
So this patch makes chasquid log the combined output (stdout and stderr)
to the execution trace.
This patch allows the configuration values to be overridden from the
command-line, with a new -config_overrides flag.
There is a fairly specific use case for this, when editing the
configuration file is not feasible or convenient (e.g. running an
user-supplied configuration in a managed environment).
This patch tidies how defaults are handled in the config, using a new
logic to allow "overriding" one config (the default) with another (the
user supplied).
It also improves how the comparisons are done in the tests, using the
more convenient "github.com/google/go-cmp/cmp" package, which also
prints nice diffs on errors.
This is in preparation for a future path where the override mechanism
will be reused.
Currently, the config package logs errors itself, in addition to
returning them.
That is confusing and results in some duplication of logging.
This patch makes config just return errors, and adjusts the callers
to log them properly.
There is a new protobuf library (and corresponding code generator) for
Go: google.golang.org/protobuf.
It is fairly compatible with the previous v1 API
(github.com/golang/protobuf), but there are some changes.
This patch adjusts the code and generated files to the new API.
The on-wire/on-disk format remains unchanged so this should be
transparent to the users.
tls.Config.BuildNameToCertificate was deprecated in Go 1.14, and is no
longer necessary.
However, we support down to 1.11, so we will keep it for now.
This patch adds a TODO to remove it in the future once the minimum
supported version is 1.14; and adjust the CI linter accordingly.
The SMTP courier, which handles outgoing connections, uses the domain of
the envelope's from as the domain in the HELO/EHLO greeting.
This works fine in practice, but ideally the domain used in the greeting
should match the reverse DNS record. This used to be more relevant but
nowadays it is not really enforced; however, it sometimes comes up in
self checks, and might cause some confusion when troubleshooting.
So this patch makes it use the configured hostname instead, which is
under the users' control and more likely to be compliant. It also
simplifies the code.
The documentation of the hostname configuration option is also updated
to mention this behaviour.
Thanks to Jonas Seydel (thor77) for bringing this up.
When creating a new Queue instance, we os.MkdirAll the queue directory.
Currently we don't check if it fails, which will cause us to find out
about problems when the queue is first used, where it is more annoying
to troubleshoot.
This patch adjusts the code so that we check and propagate the error.
That way, problems with the queue directory will be more evident and
easier to handle.
The linter complains that we're not checking for errors, but on some
cases it's on code paths were it is reasonable to do so (e.g. we're
closing the connection and it's a best-effort write).
This patch adjusts the code to make those cases explicit.
