636 Commits

Author SHA1 Message Date
Alberto Bertogli
607a455b20 docs: Update README links to the documentation
The documentation is now available directly at
https://blitiri.com.ar/p/chasquid/, so remove the /docs/ suffix from the
links in the README.

The /docs/ variant will continue to be functional for a while anyway.
2020-07-30 00:16:14 +01:00
Alberto Bertogli
bee37ef7a4 modules: Update blitiri.com.ar/go/spf to v1.1.1
This patch updates the blitiri.com.ar/go/spf dependency to v1.1.1, which
includes a fix for potential mismatches with a:, mx: and ptr:.
2020-07-28 02:12:29 +01:00
Alberto Bertogli
a6a20fb6e0 monitoring: Add a config dump handler
This patch adds a handler to the monitoring HTTP server which dumps
the parsed config, so it can easily be reviewed for troubleshooting.
2020-07-28 02:12:29 +01:00
Alberto Bertogli
33446fd565 monitoring: Reorganize index structure
This patch improves the organization of the monitoring index page:

- Include the hostname (both OS and configured) for convenience.
- Round the uptime presentation for readability.
- Add a tiny CSS for consistency with the traces.
- Re-arrange the list of links for readability.
2020-07-08 23:23:17 +01:00
Alberto Bertogli
a08b86a663 Move monitoring HTTP server code to a separate file
This patch moves the monitoring HTTP server code to its own file, for
readability purposes only.
2020-07-08 22:15:37 +01:00
Alberto Bertogli
380da0c1a1 dovecot-auth-cli: Add help message
The dovecot-auth-cli binary can be useful to debug and troubleshoot
dovecot auth issues.

This patch adds a small help message to make it easier for users to
understand how to use it.
2020-07-08 00:30:11 +01:00
Alberto Bertogli
35e19dc4a2 protoio: Use new protobuf API for text marshalling
This patch makes protoio use the new protobuf API for
marshalling/unmarshalling text protobufs, as well as extends the tests
to cover marshalling failures.

The protobuf text output is not stable/deterministic and some spaces are
added randomly, so some integration tests have to be adjusted to account
for it.
2020-06-30 11:14:52 +01:00
Alberto Bertogli
4116c046bc hooks: Ignore rspamd's greylist action
The default hook will use rspamc (the command-line client of rspamd) if
it is installed. rspamc will emit one suggested action, and then the
hook will interpret it and return accordingly.

Because the possible actions returned by rspamc are user-configured,
this patch adds a comment to make it clear that the hook will need
adjustment if the configuration uses non-default actions.

In particular, the greylisting module (which usually handles the
"greylist" action) is not run when using rspamc. This can cause
unnecessary rejections and is quite misleading.

This patch removes the "greylist" action handling; now the default hook
will only reject mail once it reaches rspamd's configured threshold for
direct rejection.

In the future, a more custom integration with rspamd might be added to
allow for rspamd-based greylisting, but until then this is a more
reasonable default.

Thanks to Jonas Seydel (thor77) and Max Mazurov (fox.cpp@disroot.org)
for noticing this issue, helping investigate, and discussing the course
of action.
2020-06-01 21:05:02 +01:00
Thor77
b6248f3089 hooks: Fix output of command -v dkimsign added to message
When checking if the dkimsign command exists, the default hook doesn't
redirect the output to /dev/null, so if the command is present it will
emit unwanted output (interpreted as message headers, as expected).

This patch adds the missing redirection.

Amended-by: Alberto Bertogli <albertito@blitiri.com.ar>
  Extended commit message.
2020-06-01 14:18:11 +01:00
Alberto Bertogli
d781b3f403 Update blitiri.com.ar/go/systemd to v1.1.0
This patch updates the modules to blitiri.com.ar/go/systemd v1.1.0.

There are no significant changes, but it's the first version which
supports go modules.
2020-05-29 02:04:07 +01:00
Alberto Bertogli
d9d56552f3 maillog: Support logging to stdout and stderr
This patch adds support for writing maillog to stdout and stderr, which
can be desirable in certain environments.

Thanks to Denys Vitali <denys@denv.it> who sent an alternative patch for
this functionality.
2020-05-24 02:26:18 +01:00
Alberto Bertogli
d83c1dc591 smtpsrv: Fix error code on transient authentication issues
When we can't authenticate due to a transient issue, for example if we
rely on Dovecot and it is not responding, we should use a differentiated
error code to avoid confusing users.

However, today we return the same error code as when the user enters the
wrong password, which could confuse users as their MUA might think their
credentials are no longer valid.

This patch fixes the issue by returning a differentiated error code in
that case, as per RFC 4954.

Thanks to Max Mazurov (fox.cpp@disroot.org) for reporting this problem.
2020-05-23 01:05:12 +01:00
Alberto Bertogli
b4c8244e48 modules: Update blitiri.com.ar/go/spf to v1.1.0
This patch updates the modules to blitiri.com.ar/go/spf v1.1.0.

There are no significant changes, but it's the first version which
includes go modules.
2020-05-22 22:41:39 +01:00
ThinkChaos
db810084a0 Reopen logs on SIGHUP
This makes it possible to manage chasquid logs using logrotate.

Amended-by: Alberto Bertogli <albertito@blitiri.com.ar>
  Added tests, minor style and comment changes.
2020-05-22 20:34:42 +01:00
ThinkChaos
ade107f62e maillog: Use blitiri.com.ar/go/log for mail log
In preparation for supporting log rotation, this patch makes the maillog
package to use blitiri.com.ar/go/log instead of its own writer.

Some of the tests are made more strict, to better test the log format.

Amended-by: Alberto Bertogli <albertito@blitiri.com.ar>
  Fixed build, extended commit message, adjusted to the log options
  API, and added tests.
2020-05-22 20:09:19 +01:00
Alberto Bertogli
0f686399b6 modules: Update blitiri.com.ar/go/log to v1.1.0
This patch updates the modules to blitiri.com.ar/go/log v1.1.0.

That version includes new APIs we will rely on in subsequent patches.
They are experimental but it should be fine for our case, and we can
adjust if they change.
2020-05-22 20:08:14 +01:00
Alberto Bertogli
9fe790d7c6 aliases: Log the "alias-exists" hook output, for debugging
The output of the alias-exists hook is unused, so currently it's
discarded silently.

However, it can be very useful to debug issues when the hook is not
working as expected.

So this patch makes chasquid log the combined output (stdout and stderr)
to the execution trace.
2020-05-22 14:43:28 +01:00
Alberto Bertogli
bee7a9f193 docs: v1.4 release notes 2020-05-22 12:59:29 +01:00
Alberto Bertogli
aacd50e87a modules: Update Go module versions
This patch updates the Go module versions to the latest available.
2020-05-22 12:59:29 +01:00
Alberto Bertogli
4367c9e059 travis: Remove references in documentation, allow to fail
The Travis tests don't work due to a Travis networking issue reaching
some external hosts.

Until it is fixed, remove references from the documentation, since its
output is misleading.

We also allow it to fail, which makes the GitHub UI not show commits as
having failed tests because of Travis.

Since the previous commit, GitLab CI does almost all the same tests, so
there should be no significant loss of coverage anyway.
2020-05-22 12:11:27 +01:00
Alberto Bertogli
a48af00162 gitlab-ci: Run go tests on multiple versions
This patch makes the GitLab CI config run go tests on Go 1.11 and the
latest Go release.

This is in addition to existing tests (which are renamed for clarity).

This replaces the main use case of Travis CI, which was to test on
multiple Go versions.
2020-05-17 11:49:03 +01:00
Alberto Bertogli
f296bc31a8 cirrus: Allow the testing user to write to $GOPATH
Cirrus CI caches the module directory inside $GOPATH so it can be shared
between test runs, to speed them up.

However, it is currently created as root, which causes new entries to
fail since they run under the "testing" user.

This patch fixes that problem by making the "testing" user own the
entire $GOPATH, which includes the module directory.
2020-05-17 10:02:49 +01:00
Alberto Bertogli
4c28efcb20 config: Allow overrides from the command line
This patch allows the configuration values to be overridden from the
command-line, with a new -config_overrides flag.

There is a fairly specific use case for this, when editing the
configuration file is not feasible or convenient (e.g. running an
user-supplied configuration in a managed environment).
2020-05-17 00:10:06 +01:00
Alberto Bertogli
7909b479eb config: Tidy default handling and comparisons in tests
This patch tidies how defaults are handled in the config, using a new
logic to allow "overriding" one config (the default) with another (the
user supplied).

It also improves how the comparisons are done in the tests, using the
more convenient "github.com/google/go-cmp/cmp" package, which also
prints nice diffs on errors.

This is in preparation for a future path where the override mechanism
will be reused.
2020-05-16 23:48:09 +01:00
Alberto Bertogli
b1fe4f81f9 config: Improve logging of errors
Currently, the config package logs errors itself, in addition to
returning them.

That is confusing and results in some duplication of logging.

This patch makes config just return errors, and adjusts the callers
to log them properly.
2020-05-16 23:46:43 +01:00
Alberto Bertogli
50986a7b7e Update protobuf library to v2
There is a new protobuf library (and corresponding code generator) for
Go: google.golang.org/protobuf.

It is fairly compatible with the previous v1 API
(github.com/golang/protobuf), but there are some changes.

This patch adjusts the code and generated files to the new API.

The on-wire/on-disk format remains unchanged so this should be
transparent to the users.
2020-05-16 10:12:51 +01:00
Alberto Bertogli
5ec567b27e travis: Only notify IRC about failures on change
There's no need to notify IRC on every failure, it can get spammy when
we're iterating trying to fix something.

This patch changes the config to notify IRC on change instead of always.
2020-05-13 23:50:01 +01:00
Alberto Bertogli
7fa564f822 smtpsrv: Add comment on BuildNameToCertificate being deprecated
tls.Config.BuildNameToCertificate was deprecated in Go 1.14, and is no
longer necessary.

However, we support down to 1.11, so we will keep it for now.

This patch adds a TODO to remove it in the future once the minimum
supported version is 1.14; and adjust the CI linter accordingly.
2020-05-13 23:42:37 +01:00
Alberto Bertogli
70387207f2 man: Update generated versions
This patch just re-runs the pod generator using a newer version of it,
going from 4.09 to 4.11.

There are no content changes.
2020-05-13 20:42:09 +01:00
Alberto Bertogli
13ee3ba482 courier: Use the hostname in SMTP HELO
The SMTP courier, which handles outgoing connections, uses the domain of
the envelope's from as the domain in the HELO/EHLO greeting.

This works fine in practice, but ideally the domain used in the greeting
should match the reverse DNS record. This used to be more relevant but
nowadays it is not really enforced; however, it sometimes comes up in
self checks, and might cause some confusion when troubleshooting.

So this patch makes it use the configured hostname instead, which is
under the users' control and more likely to be compliant. It also
simplifies the code.

The documentation of the hostname configuration option is also updated
to mention this behaviour.

Thanks to Jonas Seydel (thor77) for bringing this up.
2020-05-13 20:27:17 +01:00
Alberto Bertogli
7b0703eaa0 queue: Check that we can create the queue directory
When creating a new Queue instance, we os.MkdirAll the queue directory.

Currently we don't check if it fails, which will cause us to find out
about problems when the queue is first used, where it is more annoying
to troubleshoot.

This patch adjusts the code so that we check and propagate the error.
That way, problems with the queue directory will be more evident and
easier to handle.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
25ebc4f2e2 Avoid unnecessary calls to fmt.Sprintf
The linter caught some unnecessary calls to fmt.Sprintf. This patch
removes them.

There are no functional changes.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
d6b512166b Make it explicit when we are intentionally not checking errors
The linter complains that we're not checking for errors, but on some
cases it's on code paths were it is reasonable to do so (e.g. we're
closing the connection and it's a best-effort write).

This patch adjusts the code to make those cases explicit.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
aed0156579 chasquid: Check result of initial os.Chdir
The daemon attempts to change to the config directory on startup, for
security and convenience.

We currently don't check if this works, which is not a big deal since it
will just fail later on when it can't find the files. However, it makes
things more awkward to debug, so this patch adds an explicit check.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
4802e2f3e4 smtpsrv: Check TLS Handshake result
When receiving a message on a TLS socket, we currently don't check the
Handshake result, so connections often fail in a way that is not easy to
troubleshoot.

This patch fixes that by checking the result and emitting a nicer error
message before closing the connection.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
0fd3941cf0 courier: Don't call testing.T.Fatalf from a goroutine
Calling testing.T.Fatalf from a new goroutine is not supported; since
this should be quite exceptional, we just panic instead.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
15b79d9de9 chasquid-util: Check that we can create the database directory
When creating a database directory, we were missing the check to see if
it had succeeded, which would make issues more difficult to troubleshoot.

This patch adds the missing check.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
929a9947bf chasquid-util: Use docopt.ParseDoc
docopt.Parse is deprecated. This patch updates the code to the newer
variant, ParseDoc, since the default options are what we want.

There are no functional changes.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
d60c03f4d2 tests: Add Cirrus CI integration
This patch adds a configuration for https://cirrus-ci.org/, which
includes running the Go tests on FreeBSD.
2020-04-14 12:01:01 +01:00
Alberto Bertogli
2e1e188387 travis: Enable Go modules in 1.11
Currently the modules are ignored in the Go 1.11 build, because the
files are within $GOPATH.

This causes problems when some dependencies are updated in
backwards-incompatible ways, and assuming that Go modules are being
used. In particular, the new protobuf release caused this problem which
was caught by the automated builds:
https://travis-ci.org/github/albertito/chasquid/jobs/674701956.

This patch enables Go modules in 1.11 builds.

Thanks to Jonas Seydel (thor77) for the help investigating and finding a
fix for this problem.
2020-04-14 12:00:57 +01:00
Alberto Bertogli
2b9c2df9d3 docs: v1.3 release notes 2020-04-12 11:42:00 +01:00
Alberto Bertogli
9fbd1fe786 tlsconst: Update list of ciphers
This patch updates the list of ciphers in tlsconst, using the latest
list from IANA.
2020-04-12 11:25:27 +01:00
Alberto Bertogli
6815cf1d5c docs: Add Arch installation instructions
There is an AUR package for chasquid, so this patch adds references to
it in the documentation.

Thanks to Max Mazurov (fox.cpp@disroot.org) for adding the package.
2020-03-28 17:59:46 +00:00
Alberto Bertogli
4357379737 test: Implement retries on the load generator
If the load generator is sending emails too fast, chasquid queue might
hit the maximum size and fail the test.

This patch makes it sleep and retry, to give the server some time to
catch up.

Thanks to Max Mazurov (fox.cpp@disroot.org) for reporting this problem.
2020-03-28 16:42:04 +00:00
Alberto Bertogli
35c2d1d485 modules: Update go module versions
This patch updates the Go module versions to the latest available.
2020-03-27 08:55:34 +00:00
Alberto Bertogli
18716c25bb docker: Add dockerhub to the docs, and soften the warning
This patch adds a link to the dockerhub image mirror, as well as
softening the warning against the image, since by now it is reasonably
functional.
2020-03-23 01:39:39 +00:00
Alberto Bertogli
7814c64cfb gitlab: Upload images to dockerhub
It can be convenient to upload images to dockerhub for redundancy and
visibility, so this patch updates the gitlab CI configuration to do
that.

While at it, rename the stages for readability.
2020-03-23 01:28:46 +00:00
Alberto Bertogli
29512f7e4f testlib: Add tests for testlib.WaitFor 2020-03-21 23:56:31 +00:00
Alberto Bertogli
fdae72f275 testlib: Add comments and unexport unnecessary structs
This patch contains some readability improvements to testlib: it
adds/reformats some comments for exported functions for consistency, and
unexports some structs that are not used outside the library.
2020-03-21 23:32:28 +00:00
Alberto Bertogli
44140220b9 test: Improve DATA handling in the smtpsrv fuzzer
The smtpsrv fuzzer doesn't handle DATA commands particularly well:
it will continue to read but will skip lines that have STARTTLS as
content, and only really care for the first line due to a bug.

This patch fixes the handling, and moves the logic to a separate
function for readability.
2020-03-21 23:27:19 +00:00