636 Commits

Author SHA1 Message Date
Alberto Bertogli
008367d320 Update minimum supported Go version 1.15 -> 1.17
The latest version of some packages we depend on (golang.org/x/sys,
indirectly) do not support 1.15 anymore.

We kept 1.15 because it's the version on Debian stable, but that release
is more than 2y old, and newer versions are readily available in Debian
stable backports.

So this patch updates the minimum supported Go version to 1.17.
2022-11-12 20:06:35 +00:00
Alberto Bertogli
6896b221e8 github: Use the latest Go version for govulncheck
govulncheck will complain when run on older Go versions that are known
to have security issues. Github setup-go action has a cache for the
"latest" go version and sometimes the latest isn't really used.

So sometimes, this generates false positives because govulncheck is run
on an older Go version, which is not a problem in chasquid itself.

To prevent those false positives, this patch asks the Github CI to
always check for the latest Go version when running govulncheck.
2022-11-12 18:31:57 +00:00
Alberto Bertogli
f5949a79f1 test: Update expected DSN to accept headers set by latest msmtp
Newer versions of msmtp now set In-Reply-To and References header,
causing t-16-spf test to fail because we expect them to be empty (for no
particular reason).

So this patch changes our expected DSN used for testing to ignore their
values.
2022-11-12 18:21:13 +00:00
Alberto Bertogli
c738c7edf9 courier: Move the test fake server into a separate file
In future patches we will use the test fake server in other tests, so
move it to a separate file for clarity.
2022-11-12 16:34:35 +00:00
Alberto Bertogli
cbb620eec2 modules: Update Go modules
This patch does a general pass updating Go modules to recent versions.

The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
2022-11-12 11:52:26 +00:00
Alberto Bertogli
2bad7a14be etc: Simplify default config by removing systemd sockets
Having systemd open sockets and pass them to chasquid is neat, but also
adds some complexity to the default config, for very little value in
practice.

This patch simplifies the default config by having chasquid open the
sockets instead.

Note that systemd file descriptor passing continues to be supported, and
existing installations will not be affected.
2022-11-12 11:49:20 +00:00
Alberto Bertogli
48da3c1657 etc: Add missing submission_over_tls_address docs to chasquid.conf
This patch adds documentation for submission_over_tls_address in the
default chasquid.conf, which was accidentally missing.
2022-11-12 11:48:46 +00:00
Alberto Bertogli
c3b4cde29a monitoring: Add dark mode CSS to the monitoring home page 2022-10-21 10:12:06 +01:00
Alberto Bertogli
fbb9bed366 ci: Add Github workflows
This patch adds two github workflows that will run on each commit, and
also regularly: CodeQL and govulncheck.
2022-10-21 10:11:37 +01:00
Alberto Bertogli
b9c2ef68f9 Use git describe --tags to include non-annotated tags
By default, `git describe` uses only annotated tags. Since some may not
be annotated (like v1.10) this causes some scripts to pick a confusing
version identifier.

This patch fixes the issue by passing `--tags`, which means all tags
will be considered.
2022-09-04 13:10:07 +01:00
Alberto Bertogli
ddcfacccd0 docs: v1.10 release notes 2022-09-02 16:01:43 +01:00
Alberto Bertogli
776bdc58ab Update Go doc comments to Go 1.19's format
This patch is the result of running Go 1.19's `gofmt` on the codebase,
which automatically updates all Go doc comments to the new format.

https://tip.golang.org/doc/go1.19#go-doc
2022-09-02 11:11:40 +01:00
Alberto Bertogli
7e38a877e8 hooks: Fix dkimpy's diff check
When running a diff for dkimpy's output, we expect that diff to exit with
non-zero code.

Unfortunately, the way we set that expectation (by prefixing the diff
invocation with `!` is incorrect.

Running `! diff ...` will not cause the hook to fail if diff exits with
0, instead `!` will cause the exit code to be ignored.

This patch fixes the problem by running `diff ... && exit 1` instead.

This was caught by shellcheck, https://www.shellcheck.net/wiki/SC2251.
2022-08-27 23:58:26 +01:00
Alberto Bertogli
e85c31782b Fix misc. linter issues (comments, variable naming, etc.)
We've accumulated a few linter issues around comments and a couple of
variable names.

While none of them is major, this patch cleans them up so it's easier to
go through the linter output, and we can start being more strict about
it.
2022-08-27 23:49:33 +01:00
Alberto Bertogli
6dfff9a790 modules: Update Go modules and regenerate protobufs
This patch does a general pass updating Go modules to recent versions, and
regenerates the protobufs accordingly.

The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
2022-08-27 23:39:40 +01:00
Alberto Bertogli
3eed7cd1a9 test: Use our own generate_cert helper
The current generate_cert helper was originally taken from Go's source,
and is more complex than we need it to be.

This patch replaces it with our own version, rewritten from scratch
independently.
2022-08-27 23:39:40 +01:00
Alberto Bertogli
21e8d50df6 test: Improve layout of helper binaries
This patch moves the test helper binaries to a "one directory per
helper" layout, and also makes them to be ignored in the coverage build
instead of all builds.

With this change, "go build ./..." will build all binaries including the
test helpers, which helps make sure that module manage automation also
considers them. In particular, this makes "go mod tidy" work fine.
2022-08-27 18:46:54 +01:00
Alberto Bertogli
567ad35122 docker: Only do setfacl if we issued the certificates
Today, we do setfacl unconditionally; this can be a problem for
user-provided certificates because they may be located somewhere else.

This patch fixes the problem by only doing setfacl after renewing the
certificates.

Externally provided certificates will be untouched, and the user is
responsible for ensuring that chasquid can read them.

Thanks to Alex Ellwein (aellwein@github) for reporting this in
https://github.com/albertito/chasquid/issues/29!
2022-08-23 23:48:35 +01:00
Alberto Bertogli
b779941f90 Remove spf-check debugging tool
The spf-check debugging tool is quite primitive, rarely (if ever) used,
not tested, and has been superceded by the one in the spf library
(https://blitiri.com.ar/git/r/spf/b/master/t/cmd/spf-check/f=spf-check.go.html).

This patch removes it.
2022-08-08 17:56:58 +01:00
Alberto Bertogli
5bb17c7066 Update build tag constraints
This patch updates all build tag constraints to add the new format,
alongside the old one, to maintain backwards compatibility.

This was done by using `go fmt`.

See https://go.dev/doc/go1.17#gofmt and
https://golang.org/design/draft-gobuild for more details.
2022-08-08 17:52:34 +01:00
Alberto Bertogli
d4b89db6c3 modules: Update blitiri.com.ar/go/spf to 1.5.0
This patch updates the dependency on blitiri.com.ar/go/spf from v1.4.0
to v1.5.0, which includes a few bug fixes.

There are no code changes needed.
2022-08-08 17:36:01 +01:00
Alberto Bertogli
735613cdf7 test: Set state_dir in Dovecot config
Dovecot's `state_dir` usually defaults to be at `/var/lib/dovecot`, or a
similar system-wide path.

Under some conditions, our test Dovecot instance can fail, because it's
wanting to write to state_dir, but it is not writeable by us in the test
environment.

This was reported by foxcpp in
https://github.com/albertito/chasquid/issues/28.

This patch fixes the problem by setting a custom state_dir to be within
our testing directory.

Thanks to foxcpp for reporting this problem and suggesting a fix.
2022-07-04 09:46:50 +01:00
Alberto Bertogli
7fe22ef104 docs: Update URL for Arch binary packages
This patch updates the URL for the Arch binary packages, which has moved.

Thanks to foxcpp for maintaining them and letting me know about the
change.
2022-07-04 09:46:11 +01:00
Alberto Bertogli
5c51f74941 modules: Update blitiri.com.ar/go/spf to v1.4.0
This patch updates the dependency on blitiri.com.ar/go/spf from v1.3.0
to v1.4.0.

There are no code changes needed, but the new version contains some
important fixes around lookup limits counting.
2022-06-19 12:11:27 +01:00
Alberto Bertogli
faadae15ca tests: Detect buggy dkimpy versions, and skip the test if needed
Some dkimpy versions have a bug where it can't parse the keys generated
by its own key generator. That causes the dkimpy test to fail.

See https://bugs.launchpad.net/dkimpy/+bug/1978835 for more details.

This patch adds a workaround which detects the buggy version, and skip
the test if needed.
2022-06-19 11:56:30 +01:00
Alberto Bertogli
f303e43082 aliases: Implement catch-all
This patch implements support for catch-all aliases, where users can add
a `*: destination` alias. Mails sent to unknown users (or other aliases)
will not be rejected, but sent to the indicated destination instead.

Please see https://github.com/albertito/chasquid/issues/23 and
https://github.com/albertito/chasquid/pull/24 for more discussion and
background.

Thanks to Alex Ellwein (aellwein@github) for the alternative patch and
help with testing; and to ThinkChaos (ThinkChaos@github) for help with
testing.
2022-03-11 20:51:06 +00:00
Alberto Bertogli
3255ff6801 modules: Update Go modules and regenerate protobufs
This patch does a general pass updating Go modules to recent versions, and
regenerates the protobufs accordingly.

The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
2022-03-11 18:28:11 +00:00
Alberto Bertogli
5878b9d85e docs: v1.9 release notes 2022-03-05 23:26:07 +00:00
Alberto Bertogli
d3c18aa471 modules: Update spf to 1.3.0
This patch updates the dependency on blitiri.com.ar/go/spf from v1.2.0
to v1.3.0, which includes a few bug fixes.

There are no code changes needed, just some minor adjustment to the
tests due to error strings changing.

The go.mod "go" keyword is also bumped up to 1.15 since it's the minimum
supported version since commit e444fe1f (2021-10-05).
2022-02-27 11:30:34 +00:00
Alberto Bertogli
8083e27441 docs: Expand certificate section on the install guide
The install guide mentions that the `certs/` directory can be symlinked,
but doesn't cover permissions much, so it's easy for users to be
confused (like it happened in
https://github.com/albertito/chasquid/issues/26).

This patch adds a bit more details on how to set up certificates, to
hopefully reduce the chances of confusion.
2022-02-27 10:27:23 +00:00
Alberto Bertogli
d7ca50c3e0 aliases: Add tracing to Exists and Resolve
This patch adds tracing to aliases' Exist and Resolve functions, to help
troubleshoot problems with alias resolution.
2022-01-21 12:07:34 +00:00
Alberto Bertogli
feb10299be aliases: Skip resolution logic for non-local addresses
This patch skips the resolution logic if the address is not local.
Today, the resolution logic handles that case transparently, and returns
the original email address, so this should be a no-op.

However, having an explicit early check makes the resolution logic more
robust, and will simplify future patches.

Note this also means that the `alias-resolve` hook is no longer run for
non-local aliases, which should also help simplify their implementation.
2022-01-21 12:07:34 +00:00
Alberto Bertogli
67d0064f57 aliases: Simplify lookup logic, remove alias-exists hook
This patch simplifies the internal alias lookup logic, unifying it
across Resolve and Exists.

As part of this, the `alias-exists` hook is removed. It was redundant to
begin with, although it enabled a potential optimization, it isn't worth
the complexity. The timeout for execution of both was the same.

This change should be backwards-compatible because `alias-resolve` is
still used, and the semantics haven't changed.
2022-01-21 12:07:34 +00:00
Alberto Bertogli
fa1db7d81a config: Support "" values for drop_characters and suffix_separators
If the `drop_characters` or `suffix_separators` options are set to "",
currently instead of the empty string, their default value is used instead.

This is a bug, and it also happens on other config options, but because
the others have to be set in order for chasquid to function, it's not a
problem in practice.

Thanks Björn Busse (bbusse@github) for finding and reporting this
problem, on irc and in https://github.com/albertito/chasquid/issues/25.

This patch fixes the problem by marking these fields explicitly
optional, which enables presence testing, as described in the protobuf
documentation:
https://github.com/protocolbuffers/protobuf/blob/master/docs/field_presence.md.
2022-01-21 12:07:34 +00:00
Alberto Bertogli
4f595ce3d2 docs: Document that only haproxy v1 is supported
Document that only haproxy's PROXY protocol v1 is supported. This can
help users configure their instance and avoid trying to set up other
versions.

Thanks to Björn Busse (bbuse@github) for reporting this!
2021-12-06 17:56:07 +00:00
Alberto Bertogli
02322a74e6 courier: Add tests for STS policy checks
This patch adds tests for STS policy checks in combination with TLS
security levels.

This helps ensure we're detecting mis-matches of TLS status
(plain/insecure/secure) and STS policy enforcement.
2021-11-26 13:25:31 +00:00
Alberto Bertogli
14e270b7f5 test: Replace uses of which with command -v
The `which` command isn't guaranteed to be available, it is just
extremely common; `command -v` is the standard way to do find an
executable program. See https://lwn.net/Articles/874049/ for more
details.

This patch replaces the uses of `which` with `command -v`, which only
appears in a couple of tests.
2021-10-29 11:14:16 +01:00
Alberto Bertogli
643f7576f0 courier: Use explicit certificate validation in the SMTP courier
When using STARTTLS, the SMTP courier needs to determine whether the
server certificates are valid or not.

Today, that's implemented via connecting once with full certificate
verification, and if that fails, reconnecting with verification
disabled.

This works okay in practice, but it is slower on insecure servers (due
to the reconnection), and some of them even complain because we connect
too frequently, causing delivery problems. The latter has only been
observed once, on the drv-berlin-brandenburg.de MX servers.

To improve on that situation, this patch makes the courier do the TLS
connection only once, and uses the verification results directly.

The behaviour of the server is otherwise unchanged. The only difference
is that when delivering mail to servers that have invalid certificates,
we now connect once instead of twice.

The tests are expanded to increase coverage for this particular case.
2021-10-25 12:41:24 +01:00
Alberto Bertogli
90d385556f testlib: Add GenerateCert function
This patch moves the GenerateCert function from the smtpsrv tests to the
common testlib, so it can be used by other tests in the future.
2021-10-25 12:41:24 +01:00
Alberto Bertogli
ed38945fca courier: Use DNSError.IsNotFound to identify NXDOMAIN
When resolving MX records, we need to distinguish between "no such
domain" and other kinds of errors. Before Go 1.13, this was not
possible, so we had a workaround that assumed any permanent error was a
"no such domain", which is not great, but functional.

Now that our minimum supported version is Go 1.15, we can remove the
workaround.

This patch replaces the workaround with proper logic using
DNSError.IsNotFound to identify NXDOMAIN results when resolving MX
records.

This requires to adjust a few tests, that used to work on environments
where resolving unknown domains (used for testing) returned a permanent
error, and now they no longer do so. Instead of relying on this
environmental property, we make the affected tests use our own DNS
server, which should make them more hermetic and reproducible.
2021-10-08 23:11:29 +01:00
Alberto Bertogli
6633f0785c smtpsrv: Remove obsolete call to tls.BuildNameToCertificate
tls.BuildNameToCertificate has been deprecated, and calling it is no
longer necessary since Go 1.14.

Now that our minimum supported Go version is 1.15, we can remove it.
2021-10-08 23:11:29 +01:00
Alberto Bertogli
e1a71105c3 test: Simplify dovecot config in integration test
In the Dovecot integration test, we can now simplify the configuration
as we assume Dovecot 2.3 is the minimum version supported for testing
(as that's the one from Debian stable at the moment).
2021-10-08 23:11:29 +01:00
Alberto Bertogli
fcdc49d742 chasquid-util: Update ReadPassword dependency
Today, we use `golang.org/x/crypto/ssh/terminal` to read passwords. That
package is obsolete, replaced with `golang.org/x/term`.

We couldn't move them because term wasn't compatible with Go 1.11 which
was our oldest supported Go version.

Now that we moved to Go 1.15 as the oldest supported version, we can do
the update.
2021-10-08 23:11:29 +01:00
Alberto Bertogli
07c133fd15 travis: Remove Travis CI configuration
Travis hasn't worked in a while, is shutting down for most projects, and
we have already removed it from all public documentation.

Also all the functionality it provided is now provided by GitLab CI.

This patch removes the obsolete Travis configuration file.
2021-10-05 00:22:25 +01:00
Alberto Bertogli
e444fe1ffd gitlab: Update minimal supported version
With the new Debian stable release, we now support back to Go 1.15.
Update the automated tests to reflect this.
2021-10-05 00:21:14 +01:00
Alberto Bertogli
a5bd8cbc0d docker: Install libcap2-bin binary
The latest Debian stable images don't include the `setcap` binary by
default like they used to.

Our Docker build depends on it, so this patch makes the Dockerfile
install the libcap2-bin package (which contains the `setcap` binary).
2021-09-03 11:41:59 +01:00
Alberto Bertogli
4bd2b9e3c0 gitlab-ci: Push a latest tag to Docker repositories
When doing a `docker pull`, if a tag is not specified, it defaults to
`latest`. We currently don't push such a tag, so it can cause confusion
in some cases (e.g. https://github.com/albertito/chasquid/issues/21).

To help prevent this, make the docker automatic builds for `master` also
apply the `latest` tag.

As a part of this, update the obsolete `$CI_BUILD_REF_NAME` to
`$CI_COMMIT_REF_NAME`.
2021-08-14 14:36:24 +01:00
Alberto Bertogli
a3c06b6ee4 docs: v1.8 release notes 2021-07-30 09:40:10 +01:00
Alberto Bertogli
d53c1d2b88 docs: Add DKIM setup instructions
This patch adds some basic instructions to the documentation on how to
set up DKIM, using the tools supported by the example hook.

It's not meant to be a full DKIM how-to, but to help someone who already
knows enough, or who is complementing it with a more general purpose
DKIM guide.
2021-07-25 14:59:37 +01:00
Alberto Bertogli
270a071c1e hooks: Add dkimpy support
This patch adds support in the default hook for using dkimpy for DKIM
signing.

Unfortunately, dkimpy binaries have the same name as driusan/dkim's, so
we need to use --help to disambiguate. It's not pretty but it should
work, and is quite self contained.

Also, for the integration tests, we still need driusan/dkim because
dkimpy lacks the features needed. Specifically, dkimpy's dkimverify
can't be made to use custom DNS, or override the TXT values in any way,
so we can't verify that the generated signature is reasonable.

Thanks to ne9z@github for suggesting this change and providing an
alternative patch in https://github.com/albertito/chasquid/pull/19.
2021-07-21 02:06:20 +01:00