There are a couple of places in the tests when we attempt to build and
run simultaneously. Here, if the build is slow, there is a race where
"text file busy" can appear.
To fix this, build to a temporary file with a random name, then
atomically rename it to the final binary name.
To send mails, today some tests use msmtp and others our internal smtpc.py.
This works, but msmtp slows down the tests significantly, and smtpc.py
is also not particularly fast, and also has some limitations.
This patch introduces a new SMTP client tool written in Go, and makes
almost all the tests use it.
Some tests still remain on msmtp, mainly for client-check compatibility.
It's likely that this will be moved in later patches to a separate
special-purpose test.
With this patch, integration tests take ~20% less time than before.
Our tests invoke a variety of helpers, some of them are written in Go.
Today, we call "go build" (directly or indirectly via "go run"), which is
a bit wasteful and slows down the tests.
This patch makes the tests only build our Go helpers once every 10s at
most.
The solution is a bit hacky but in the context of these tests, it's
practical.
The generate_cert cache has a bug because it uses the directory's age,
which won't necessarily change, and it was always re-generating
certificates after 10m.
This patch fixes the bug by checking the age of the private key file
instead of the directory.
This patch makes chasquid-util's aliases-resolve and domaininfo-remove
commands talk to the chasquid server (via the new localrpc server).
For aliases-resolve, currently has fairly hacky logic which reimplements
a bunch of the servers', and is also incomplete because it does not
support hooks.
In this patch we fix that by having it talk to the server, where we get
authoritative responses and have no issues with aliases hooks. This
resolves https://github.com/albertito/chasquid/issues/18.
For domaininfo-remove, currently its implementation is also very hacky
since it manipulates files behind the servers' back and without even
using the internal library.
In this patch we fix that by doing the operation through the server,
avoiding the need for those hacks, and also remove the need to manually
reload the server afterwards.
Go 1.20 finally includes proper support for instrumenting binaries for
coverage. This allows us to drop quite a few hacks and workarounds that
we used for it, and we can now also test exiting cases.
The downside is that coverage tests now require Go 1.20, but it is an
acceptable price to pay for the more accurate results.
Normal integration tests are unchanged.
This patch updates the coverage testing infrastructure to make use of
the new Go 1.20 features.
This patch updates the shell scripts with some of the common best
practices, which should make them more resilient to unusual failures and
unexpected environments (in particular, directories with spaces).
Most of these were identified by shellcheck.
The integration tests spend a lot of time on some ancilliary actions,
which slows them down: generating certificates, adding users, and
waiting for things to happen.
To improve the performance of those actions, this patch:
- Makes (most) tests use plain passwords (-20%)
- Adds a certificate cache to reuse certs (-10%)
- Tightens the sleep loops (-5%)
In aggregate, this patch results in a speedup of the integration tests
of ~30-40%.
Note that some of the tests required adjusting the username, because
`chasquid-util user-add` would convert them to lowercase as per PRECIS
mapping.
The current generate_cert helper was originally taken from Go's source,
and is more complex than we need it to be.
This patch replaces it with our own version, rewritten from scratch
independently.
This patch moves the test helper binaries to a "one directory per
helper" layout, and also makes them to be ignored in the coverage build
instead of all builds.
With this change, "go build ./..." will build all binaries including the
test helpers, which helps make sure that module manage automation also
considers them. In particular, this makes "go mod tidy" work fine.
The `which` command isn't guaranteed to be available, it is just
extremely common; `command -v` is the standard way to do find an
executable program. See https://lwn.net/Articles/874049/ for more
details.
This patch replaces the uses of `which` with `command -v`, which only
appears in a couple of tests.
Most integration tests depend on the $HOSTALIASES environment variable
being functional. That variable works on most systems, but not all. In
particular, systems with `systemd-resolved` can cause the variable to be
ignored.
This was reported by Alex Ellwein in
https://github.com/albertito/chasquid/issues/20.
This patch makes the affected tests to be skipped if $HOSTALIASES is not
working properly. It also removes unnecessary hosts files from tests
which don't need it, and documents this behaviour.
Thanks to Alex Ellwein and foxcpp@ for reporting and helping investigate
this issue!
The docopt-go library is quite convenient, but it has been abandoned for
a while :(
Since we only use it for chasquid-util, this patch removes it and
replaces it with a custom small parser, that is a reasonable fit for the
required use cases.
The patch also adds a couple of tests to increase coverage.
NOTE: docopt-go accepted some undocumented behaviour, in particular the
use of "-a b" instead of "-a=b". The new parser does not, so some
user scripts may require updating.
I think this should be rare enough not to be worth the complexity of
adjusting the parser to allow it.
This patch implements support for incoming connections wrapped in the
HAProxy protocol v1.
This is useful when running chasquid behind a HAProxy server, as it
needs the original source IP to perform SPF checks.
This patch is a reimplementation of one originally provided by Denys
Vitali in pull request #15, except the logic for the protocol handling
is moved to a new package, and the smtpsrv.Conn handling of the source
IP is simplified.
It is marked as experimental for now, since we want to give it a bit
more exposure just in case the option/api needs adjustment.
Thanks a lot to Denys Vitali (@denysvitali in github) for sending the
original patch for this, and helping test it!
Allows terminating chasquid via the network. Useful to trigger a restart
(if there is an init system to relaunch chasquid) and thus reload certificates.
Amended-by: Alberto Bertogli <albertito@blitiri.com.ar>
Added tests, and adjusted shutdown sequence.
This patch removes the dependency on wget for fetching content over
http, which was used in one of the tests to do some checking on debug
and metric pages, as well as loop detection.
Instead of wget, we now use a small built-in utility called fexp.
This patch adds a new integration test to cover SPF checks. The main
goal is not to cover the SPF parsing, since that's handled by the
library already, but the higher level aspects: that the mails are indeed
rejected, that the DSN looks reasonable, etc.
In some distributions, including newer Debian versions, msmtp is
installed as setgid.
That prevents $HOSTALIASES from being honoured, which breaks the tests.
This patch works around the problem by creating a copy of the binary,
which will not have the setgid bit set.
This is a mini-DNS server for testing purposes.
This can be used to set up hermetic tests in containers, and work around
glibc's limitation of being unable to create per-process host aliases.
This patch adds a new test, which verifies the TLS tracking.
Because we need to simulate SPF records, and Go does not support fully
intercepting DNS lookups yet, this test relies on dnsmasq to provide a
DNS resolver.
In the future, once Go supports DNS lookup interception, we can get rid
of this additional dependency.
SIGTERM can happen normally in our tests, and the current code has a
trap that makes it trigger a clean exit.
This causes some errors to be masked, as we end up calling "exit 0" when
they occur. The error message will still be displayed, but the caller
script will assume it worked.
This patch fixes the problem by adjusting the bash signal handlers, so
that we ignore SIGTERM (so bash does not get killed by the exit handler)
and exit with error on SIGINT (triggered by ctrl-c).
Note that under some conditions the SIGTERM trap is not necessary, but
this depends on the environment.
This patch introduces stress tests, which put load on the daemon to help
sanity check its behaviour under stress.
They are separate from the existing integration tests, which focus on
correctness.
Two tests are included here: a load test, which sends emails repeatedly;
and a connection test which opens as many conections as possible.
This patch adds some tooling and scripts to generate test coverage
information.
Unfortunately, this involves some hacks as Go does not have support for
generating coverage-enabled binaries, or merging coverage reports; but
overall it's not very intrusive.
This patch adds dovecot support to the chasquid daemon, using the
internal dovecot library added in previous patches.
Dovecot support is still considered EXPERIMENTAL and may be reverted, or
changed in backwards-incompatible ways.
The patch also adds the corresponding integration test, which brings up
a dovecot server with a custom configuration, and tests chasquid's
authentication against it. If dovecot is not installed, the test is
skipped.
The nc.py script is only used in a single test, and for waiting for a
TCP port to be opened for listening.
This patch replaces it entirely, by using chamuyero for the test, and
bash for waiting on a TCP port.
This patch adds "chamuyero", a a tool to test and validate line-oriented
commands and servers.
It can launch and communicate with other processes, and follow a script of
line-oriented request-response, validating the dialog as it goes along.
This can be used to test line-oriented network protocols (such as SMTP) or
interactive command-line tools.
It will be used in follow up patches to test new commands and
functionality.
Netcat's behaviour after seeing EOF from stdin seems to not be very
portable or consistent, even under the same platform.
This has caused t-05-null_address to break recently under some
conditions, for example depending on the particular Debian version of
netcat-openbsd used, and the current situation is unclear.
See https://bugs.debian.org/854292 and https://bugs.debian.org/849192
for more details.
To stop depending on this brittle behaviour, this patch unfortunately
introduces a simple python3-based netcat for our tests to use.
This patch changes chasquid-util's subcommands and parameters to
(hopefully) make them more user friendly and intuitive by default.
The changes include defaulting the configuration to /etc/chasquid, and
using full addresses as usernames.
It also adds some shell tests to cover most of the functionality.
This patch implements the first steps of support for IDNA (Internationalized
Domain Names for Applications).
Internally, we maintain the byte-agnostic representation, including
configuration.
We support receiving IDNA mail over SMTP, which we convert to unicode for
internal handling.
Local deliveries are still kept agnostic.
For sending over SMTP, we use IDNA for DNS resolution, but there are some
corner cases pending in the SMTP courier that are tied with SMTPUTF8 and will
be fixed in subsequent patches.
Having the certificates inside the domain directory may cause some confusion,
as it's possible they're not for the same name (they should be for the MX we
serve as, not the domain itself).
So it's not a problem if we have domains with no certificates (we could be
their MX with another name), and we could have more than one certificate per
"domain" (if we act as MXs with different names).
So this patch moves the certificates out of the domains into a new certs/
directory, where we do a one-level deep lookup for the files.
While at it, change the names of the files to "fullchain.pem" and
"privkey.pem", which match the names generated by the letsencrypt client, to
make it easier to set up. There's no general convention for these names
anyway.
It's often useful to run the tests with the race detector (-race) enabled.
Unfortunately, building with it is too slow to enable unconditionally.
So for now this patch adds an option, in the form of an environment variable,
to enable it manually.
This patch removes chasquid-userdb and adds a more generic and extensive
chasquid-util, that supports various operations on user databases as well as
aliases lookups.
The code is not very pretty but for now I took a more practical approach, the
tool is ancillary and can be tidied up later.
In some cases, it's be useful to have references to directories relative
to the configuration itself.
So this patch makes chasquid do a Chdir into it, so we can assume it in
the rest of the code and config.
This patch introduces a new directory, test/, which contains a simple local
end-to-end test which runs a chasquid binary and uses msmtp to send an email,
which is delivered locally.
As it's the first one, it adds a bunch of common infrastructure to simplify
writing these kinds of tests.
More end-to-end tests will follow, and it's expected that the common
infrastructure will also change significantly to accomodate their needs.
