Today, the maximum number of items in the queue, as well as how long we
keep attempting to send each item, is hard-coded and not changed by end
users.
While they are totally adequate for chasquid's main use cases, it can
still be useful for some users to change them.
So this patch adds two new configuration options for those settings.
They're marked experimental for now, so we can adjust them if needed
after they get more exposure.
Thanks to Lewis Ross-Jones <lewis_r_j@hotmail.com> for suggesting this
improvement, and help with testing it.
This patch regenerates the auto-generated files. There are no
significant changes.
- Protobuf files updated the comment formatting to match recent changes
in Go libraries.
- IANA assignment for a AEGIS (currently an IETF draft) has been
updated.
- The link to the human-readable IANA assignment tables from the
generator was manually updated.
When logging the configuration, we currently don't quote the string
values, which can make whitespace-induced problems difficult to identify
and troubleshoot.
This patch changes the formatting to always quote string values when
logging the configuration.
This patch regenerates the auto-generated files.
There are no significant changes, the protobuf just get an updated
comment due to protoc version change, but it is just informational.
Two new TLS ciphers are added, matching the new IANA assignments.
ioutil package was deprecated in Go 1.16, replace all uses with their
respective replacements.
This patch was generated with a combination of `gofmt -r`, `eg`, and
manually (for `ioutil.ReadDir`).
This patch does a general pass updating Go modules to recent versions, and
regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
If the `drop_characters` or `suffix_separators` options are set to "",
currently instead of the empty string, their default value is used instead.
This is a bug, and it also happens on other config options, but because
the others have to be set in order for chasquid to function, it's not a
problem in practice.
Thanks Björn Busse (bbusse@github) for finding and reporting this
problem, on irc and in https://github.com/albertito/chasquid/issues/25.
This patch fixes the problem by marking these fields explicitly
optional, which enables presence testing, as described in the protobuf
documentation:
https://github.com/protocolbuffers/protobuf/blob/master/docs/field_presence.md.
This patch does a general pass updating Go modules to recent versions,
and regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
We also regenerate protobufs since the newer versions of the liberaries
have a much cleaner dependency tree, which speeds up fetches.
This patch implements support for incoming connections wrapped in the
HAProxy protocol v1.
This is useful when running chasquid behind a HAProxy server, as it
needs the original source IP to perform SPF checks.
This patch is a reimplementation of one originally provided by Denys
Vitali in pull request #15, except the logic for the protocol handling
is moved to a new package, and the smtpsrv.Conn handling of the source
IP is simplified.
It is marked as experimental for now, since we want to give it a bit
more exposure just in case the option/api needs adjustment.
Thanks a lot to Denys Vitali (@denysvitali in github) for sending the
original patch for this, and helping test it!
The queue protobuf definition currently uses the well-known timestamp
protobuf package.
This adds a build-time dependency on it, which is fairly harmless when
building from source (since the golang protobuf compiler includes it
already), but adds overhead for packaging on distributions.
Since this is the only external proto dependency we have, and the
protobuf message itself is trivial, this patch removes it an instead
embeds a compatible definition.
That way we remove the dependency and simplify packaging, with almost
negligible code overhead.
The change is fully backwards compatible and has no functional changes.
This patch adds support for writing maillog to stdout and stderr, which
can be desirable in certain environments.
Thanks to Denys Vitali <denys@denv.it> who sent an alternative patch for
this functionality.
This patch allows the configuration values to be overridden from the
command-line, with a new -config_overrides flag.
There is a fairly specific use case for this, when editing the
configuration file is not feasible or convenient (e.g. running an
user-supplied configuration in a managed environment).
This patch tidies how defaults are handled in the config, using a new
logic to allow "overriding" one config (the default) with another (the
user supplied).
It also improves how the comparisons are done in the tests, using the
more convenient "github.com/google/go-cmp/cmp" package, which also
prints nice diffs on errors.
This is in preparation for a future path where the override mechanism
will be reused.
Currently, the config package logs errors itself, in addition to
returning them.
That is confusing and results in some duplication of logging.
This patch makes config just return errors, and adjusts the callers
to log them properly.
There is a new protobuf library (and corresponding code generator) for
Go: google.golang.org/protobuf.
It is fairly compatible with the previous v1 API
(github.com/golang/protobuf), but there are some changes.
This patch adjusts the code and generated files to the new API.
The on-wire/on-disk format remains unchanged so this should be
transparent to the users.
The SMTP courier, which handles outgoing connections, uses the domain of
the envelope's from as the domain in the HELO/EHLO greeting.
This works fine in practice, but ideally the domain used in the greeting
should match the reverse DNS record. This used to be more relevant but
nowadays it is not really enforced; however, it sometimes comes up in
self checks, and might cause some confusion when troubleshooting.
So this patch makes it use the configured hostname instead, which is
under the users' control and more likely to be compliant. It also
simplifies the code.
The documentation of the hostname configuration option is also updated
to mention this behaviour.
Thanks to Jonas Seydel (thor77) for bringing this up.
This patch updates the auto-generated code to match the latest tooling
versions.
In particular, the protobufs are regenerated, and the new version no
longer supports unkeyed literals, so some minor changes are needed.
Other than that, the cipher list is extended with the latest ciphers.
The submission_over_tls_address configuration option has existed for a
long time, but was not properly documented.
This patch adds it to the manpage, as well as printing it in the
configuration output on startup.
This patch adds a missing docstrings for exported identifiers, and
adjust some of the existing ones to match the standard style.
In some cases, the identifiers were un-exported after noticing they had
no external users.
Besides improving documentation, it also reduces the linter noise
significantly.
This patch extends various packages and integration tests, increasing
test coverage. They're small enough that it's not worth splitting them
up, as it would add a lot of noise to the history.
This patch adds dovecot support to the chasquid daemon, using the
internal dovecot library added in previous patches.
Dovecot support is still considered EXPERIMENTAL and may be reverted, or
changed in backwards-incompatible ways.
The patch also adds the corresponding integration test, which brings up
a dovecot server with a custom configuration, and tests chasquid's
authentication against it. If dovecot is not installed, the test is
skipped.
We have many places in our tests where we create temporary directories,
which we later remove (most of the time). We have at least 3 helpers to
do this, and various places where it's done ad-hoc (and the cleanup is
not always present).
To try to reduce the clutter, and make the tests more uniform and
readable, this patch introduces two helpers in a new "testutil" package:
one for creating and one for removing temporary directories.
These new functions are safer, better tested, and make the tests more
consistent. All the tests are updated to use them.
This patch adds support for TLS-wrapped submission connections.
Instead of clients establishing a connection over plain text and then
using STARTTLS to switch over a TLS connection, this new mode allows the
clients to connect directly over TLS, like it's done in HTTPS.
This is not an official standard yet, but it's reasonably common in
practice, and provides some advantages over the traditional submission
port.
The default port is 465, commonly used for this; chasquid defaults to
systemd file descriptor passing as for the other protocols (for now).
The current default is "procmail" for historical reasons, but the
program has been unmaintained for years and its use is not generally
advisable.
This patch changes the default MDA binary to "maildrop", which is a more
modern and reasonable MDA to use.
We keep the courier.Procmail name for now, as that's internal, but it
may be changed later. Its documentation is updated to note that the
name is just for reference but it actually works with almost any binary.
Picking the domain used in the DSN message "From" header is more
complicated than it needs to be, causing confusing code paths and having
different uses for the hostname, which should be purely aesthetic.
This patch makes the queue pick the DSN "From" domain from the message
itself, by looking for a local domain in either the sender or the
original recipients. We should find at least one, otherwise it'd be
relaying.
This allows the code to be simplified, and we can narrow the scope of
the hostname option even further.
glog works fine and has great features, but it does not play along well
with systemd or standard log rotators (as it does the rotation itself).
So this patch replaces glog with a new logging module "log", which by
default logs to stderr, in a systemd-friendly manner.
Logging to files or syslog is still supported.
The default INFO logs are more oriented towards debugging and can be
a bit too verbose when looking for high-level information.
This patch introduces a new "maillog" package, used to log messages of
particular relevance to mail transmission at a higher level.
When we permanently failed to deliver to one or more recipients, send delivery
status notifications back to the sender.
To do this, we need to extend a couple of internal structures, to keep track
of the original destinations (so we can include them in the message, for
reference), and the hostname we're identifying ourselves as (this is arguable
but we're going with it for now, may change later).
This patch removes chasquid-userdb and adds a more generic and extensive
chasquid-util, that supports various operations on user databases as well as
aliases lookups.
The code is not very pretty but for now I took a more practical approach, the
tool is ancillary and can be tidied up later.
This patch tidies up the Procmail courier:
- Move the configuration options to the courier instance, instead of using
global variables.
- Implement more useful string replacement options.
- Use exec.CommandContext for running the command with a timeout.
As a consequence of the first item, the queue now takes the couriers via its
constructor.
This patch adds a "data_dir" option, that chasquid will use to store
persistent data. It defaults to "/var/lib/chasquid" (for now, at least).
Users will come in subsequent patches.
This is just a run of "go generate ./..." which re-generates the protocol
buffers code.
There is no actual change in the logic or APIs, it's just a rebuild with a
newer proto compiler. While it's not strictly necessary, it will help make
subsequent patches cleaner.
We want to be able to distinguish between connections for SMTP and connections
for submission, so we can make different policy decisions.
To do that, we first make the configuration aware of the different kinds of
addresses. This is done in this patch in a backwards-incompatible way, but at
this point in time it is ok to do so.
Then, we extend systemd's socket passing library to support socket naming, so
we can tell the different sockets apart. This is done via the
LISTEN_FDNAMES/FileDescriptorName mechanism.
And finally we make the server and connection types aware of the socket mode.
This patch adds configuration options for the MDA binary and command line
arguments, and changes the (soon to be renamed) procmail courier to make use
of them.
This patch introduces a basic on disk configuration, comprised of a main
configuration file and per-domain directories.
It's still not complete, but will be extended in subsequent patches.
