There's a bug in wget where it logs to files if -q is used:
https://savannah.gnu.org/bugs/?51181
It is harmless to the test, but it clutters the directory and the test
output, so this patch works around the issue by forcing it to log to
/dev/null.
The test starts by removing the previous test certificates, which may
or may not exist.
If they don't, currently "rm" fails, causing the whole test to fail.
I am surprised I did not notice this before :(
This patch fixes the bug by using "rm -f" to remove the test certs.
For direct TLS connections, such as submission-over-TLS, we currently
don't get the TLS information so it appears in the headers as "plain
text", which is misleading.
This patch fixes the problem by getting the connection information
early. Note it explicitly triggers the handshake, which would otherwise
happen transparently on the first read/write, so we can use the hostname
(if any) in our hello message.
Either the recipient or from addresses can be "<>" to indicate the null
address. mda-lmtp does not handle that well, as it sends "<<>>" which is
invalid.
This patch fixes that by special-casing them, which is unfortunate but
reasonably common.
One of the SMTP tests was doing an external DNS lookup for a
non-existing host, which is reasonably harmless but makes the test less
hermetic.
This patch changes the non-existing host for an invalid address, which
has the same effect but avoids the network lookup.
This is a hack; ideally we would be able to override the resolver, but
Go does not implement that yet, and changing the code is not worth the
additional complexity.
We have many places in our tests where we create temporary directories,
which we later remove (most of the time). We have at least 3 helpers to
do this, and various places where it's done ad-hoc (and the cleanup is
not always present).
To try to reduce the clutter, and make the tests more uniform and
readable, this patch introduces two helpers in a new "testutil" package:
one for creating and one for removing temporary directories.
These new functions are safer, better tested, and make the tests more
consistent. All the tests are updated to use them.
The right-hand side addresses of an alias should be normalized, to
maintain the internal invariant that we always deal with normalized
addresses.
Otherwise, strange situations may arise, such as the same domain having
two different domaininfo structures depending on case.
The outgoing security level checks are not being performed, because of a
bug: the courier thinks the "to"'s domain is always empty.
This patch fixes the bug by simplifying the logic, as there's no need
for the conditional (there is always a domain in the "to" address if it
got to the SMTP courier).
The nc.py script is only used in a single test, and for waiting for a
TCP port to be opened for listening.
This patch replaces it entirely, by using chamuyero for the test, and
bash for waiting on a TCP port.
mda-lmtp is a very basic MDA that uses LMTP to do the mail delivery.
It takes command line arguments similar to maildrop or procmail, reads an
email via standard input, and sends it over the given LMTP server.
Supports connecting to LMTP servers over UNIX sockets and TCP.
Since chasquid does not support direct LMTP local delivery, this can be
used as a workaround instead.
Example of use:
$ mda-lmtp --addr localhost:1234 -f juan@casa -d jose < email
This patch adds "chamuyero", a a tool to test and validate line-oriented
commands and servers.
It can launch and communicate with other processes, and follow a script of
line-oriented request-response, validating the dialog as it goes along.
This can be used to test line-oriented network protocols (such as SMTP) or
interactive command-line tools.
It will be used in follow up patches to test new commands and
functionality.
PasswordMatches calculates the proposed derived key, and then compares
it with the actual derived key. That comparison is done using
bytes.Equal, which is not in constant time.
In theory, users with knowledge of the salt could use timing to extract
information about the actual derived key.
In practice, the salt is not being exposed to users, and the caller of
PasswordMatches will add a delay to password checks, so it should not be
easy to exploit via chasquid.
But just to be safe and more future-proof, this patch changes the
comparison to be in constant time.
Currently, chasquid exits if any mode (SMTP/submission/submission+tls)
has no addresses to listen on. This means that chasquid must be given
addresses for all three.
While that's generally the expected configuration, there are cases where
users may not want to have all three.
So this patch replaces that fatal error with a warning, and only makes
chasquid exit if there are no addresses to listen on at all.
This commit removes the experimental MTA-STS (Strict Transport Security)
implementation for now, as it's not up to date with the latest draft.
Development will continue on the "sts" branch, but this way it won't
block releases until it is ready.
Commits reverted:
- cb6500b993
- 0eeb964534
- e66288e4b4
- 216cf47ffa
- d66b06de51
- fe00750e39
- 933ab54cd8
This patch adds support for TLS-wrapped submission connections.
Instead of clients establishing a connection over plain text and then
using STARTTLS to switch over a TLS connection, this new mode allows the
clients to connect directly over TLS, like it's done in HTTPS.
This is not an official standard yet, but it's reasonably common in
practice, and provides some advantages over the traditional submission
port.
The default port is 465, commonly used for this; chasquid defaults to
systemd file descriptor passing as for the other protocols (for now).
The current default is "procmail" for historical reasons, but the
program has been unmaintained for years and its use is not generally
advisable.
This patch changes the default MDA binary to "maildrop", which is a more
modern and reasonable MDA to use.
We keep the courier.Procmail name for now, as that's internal, but it
may be changed later. Its documentation is updated to note that the
name is just for reference but it actually works with almost any binary.
expvar.Int.Value appeared in Go 1.8, but we want to keep compatibility
with Go 1.7 at least until the next release.
So this patch replaces the use of expvar.Int.Value in tests, to make
them compatible with Go 1.7 again.
Currently, if we can't find a mail server for a domain, we consider that
a transient failure (semi-accidentally, as we iterate over the (empty)
list of MXs and fall through the list).
It should be treated as a permanent error, in line with other DNS
issues (which is not ideal but seems to be generally accepted
behaviour).
To avoid accidents/DoS when we are fetching a very very large policy,
this patch limits the size of the reads to 10k, which should be more
than enough for any reasonable policy as per the current draft.
The current tests stop short of fetching over HTTP, but that code is
unfortunately not trivial.
This patch changes the testing strategy to use a testing HTTP server,
which we point our URLs to. That way we can cover much more code with the
same tests.
This patch extends the SMTP courier to (optionally) do STS policy
checking when delivering mail.
As STS support is currently experimental, we gate this behind a flag and
is disabled by default.
This patch adds an on-disk cache for STS policies.
Policies are cached by domain, and stored on files in a single
directory. The files will have as mtime the time when the policy
expires, this makes the store simpler, as it can avoid keeping
additional metadata.
There is no in-memory caching. This may be added in the future, but for
now disk is good enough for our purposes.
This patch extends WriteFile to allow arbitrary operations to be applied
to the file before it is atomically renamed.
This will be used in upcoming patches to change the mtime of the file
before it is atomically renamed.
The "mx" field is required, a policy without it is invalid, so add a
check for it.
See
https://mailarchive.ietf.org/arch/msg/uta/Omqo1Bw6rJbrTMl2Zo69IJr35Qo
for more background, in particular the following paragraph:
> The "mx" field is required, so if it is missing, the policy is invalid
> and should not be honored. (It doesn't make sense to honor the policy
> anyway, I would say, since a policy without allowed MXs is essentially a
> way of saying, "There should be TLS and the server identity should match
> the MX, whatever the MX is." I guess this prevents SSL stripping, but
> doesn't prevent DNS injection, so it's of relatively little value.)
This EXPERIMENTAL patch has a basic implementation of MTA-STS (Strict
Transport Security), based on the current draft at
https://tools.ietf.org/html/draft-ietf-uta-mta-sts-02.
It integrates the policy fetching and checking into the smtp-check tool
for convenience, but not yet in chasquid itself.
This is a proof of concept. Many features and tests are missing; in
particular, there is no caching at all yet.
Currently, we pick the first host in the MX list, and attempt delivery
there. If it fails, we just report the failure to the queue, which will
wait for some time and then try again.
This is not ideal: we should fall back to the other MXs in the list, as
the first host could be having issues for a long time, and not
attempting with the rest just delays delivery.
This patch implements the fallback, so we try all MXs before deciding to
report a failed delivery (unless, of course, an MX returned a permanent
failure).
Netcat's behaviour after seeing EOF from stdin seems to not be very
portable or consistent, even under the same platform.
This has caused t-05-null_address to break recently under some
conditions, for example depending on the particular Debian version of
netcat-openbsd used, and the current situation is unclear.
See https://bugs.debian.org/854292 and https://bugs.debian.org/849192
for more details.
To stop depending on this brittle behaviour, this patch unfortunately
introduces a simple python3-based netcat for our tests to use.
Currently there is no symbol for Fatal-level log entries, so they appear
with "-2", which is distracting.
This patch makes the Fatal log entries have their own symbol, ☠.
The server is written assuming there's at least one valid SSL/TLS
certificate. For example, it unconditionally advertises STARTTLS, and
only supports AUTH over TLS.
This patch makes the server fail to listen if there are no certificates
configured, so the users don't accidentally run an unsupported
configuration.
When testing, we don't want the server to do SPF lookups, as those cause
real DNS queries which can be problematic and add a dependency on the
environment.
This patch adds an internal boolean to disable the SPF lookups, which is
only set from the tests.
chasquid needs at least one certificate in order to start, to prevent
accidental misconfigurations.
This patch documents this in etc/chasquid/README, and while at it fixes
a minor terminology issue.
Thanks to Martin Ferrari for the bug report!
We only care about directories within the certs/, but the code as-is
complains if there are files.
This patch makes the iteration skip non-directories entirely.
Thanks to Martin Ferrari for the bug report!
When adding a user, chasquid-util should create the domain directory if
it doesn't exist, but currently doesn't do that.
This patch fixes this by adding the missing os.MkdirAll call.
Thanks to Martin Ferrari for the bug report!
When building the Debian package, the tests are run in such a way that
test/util/exitcode is not found, which causes them to fail.
This patch is a workaround which skips the test if test/util/exitcode is
not found; for now we should consider it temporary, until either the
Debian packaging is fixed, or we decide that its environment is
reasonably enough to make it permanent.
:
This commit adds a .travis.yml which configures https://travis-ci.org/, a
continuous integration service.
It only builds and runs "go test" for now because their environment is
unfortunately too old to run the integration tests.
One of the command sequences was not indented enough, so it appears as
text instead of code.
This patch fixes that by surrounding it with ``` to mark it explicitly
as a code block.
Picking the domain used in the DSN message "From" header is more
complicated than it needs to be, causing confusing code paths and having
different uses for the hostname, which should be purely aesthetic.
This patch makes the queue pick the DSN "From" domain from the message
itself, by looking for a local domain in either the sender or the
original recipients. We should find at least one, otherwise it'd be
relaying.
This allows the code to be simplified, and we can narrow the scope of
the hostname option even further.
