caddy: Caddy does not support http+https on standard ports if certs are supplied

By “weird” I simply mean, not orthogonal. On the surface, Caddy’s configuration language appears to be orthogonal throughout: stuff inside a server block appears not to affect stuff outside, and vice versa… except for when it does.

If you’re not clear on what I mean by orthogonal, I mean that changing something in one place should not require you to change something somewhere else, when the things are not obviously related or dependent.

So for example, if someone is using Caddy-generated certs to start with, and then later adds a tls directive to specify certs, it is not obvious that changing the labels of the previously-working configuration is required to keep the site behavior the same. Simply adding the tls directive forces other things to change, or else your site becomes unreachable on the ports that it was reachable on ten minutes ago. A busy sysadmin should not be expected to magically know that “foo.bar.com” is no longer a valid address just because they added a tls line.

So, if that’s the behavior Caddy is going to have, I would expect at minimum these non-orthogonal bits to be documented for any part of the syntax where there is non-local influence.

For example, the Site Addresses section on this page should explain what happens when automatic HTTPS is not enabled, rather than only giving examples for when it is, and the tls directive page should have a prominent warning explaining that many Caddy features work differently or do not work at all if you use your own certificates – and list which ones those are.

In addition, each directive should indicate where it falls in the middleware stack: it should not be necessary to reference the code in order to know what directives take precedence over what.

These kinds of documentation, IMO, are the minimum needed to avoid the kind of hassles I spent hours on, to do what seemed to me like pretty basic things to do with a webserver.

If you want to say, “well, it’s not really meant to be a general-purpose webserver,” sure, but then you also seem to be saying it should work fine for everyone… so I’m not really sure what position you’re actually taking.

In any case, I’ve now told you what documentation changes would’ve helped me avoid wasting time and giving up on Caddy before I could even really get started with it, as well as what behavior changes (e.g. making user-supplied and Caddy-obtained certs have no effect on how addresses map to ports) would make those documentation changes unnecessary.

FWIW I totally agree about the non-orthogonality. I ran into the same issues and find it really annoying that because I only changed a server from having a LE cert to using self_signed, it stopped working, because I didn’t specify the port or https:// in the label.

@pjeby

If Caddy wants to serve http on 2015 – and only 2015 – that’d be fine… except that if you specify http://, then it goes to :80… so you can’t get things to be consistent without explicitly supplying ports everywhere. Conversely, if you set -port 80, TLS is disabled, instead of having http served on 80 and https on 443.

The -port flag changes the default port - in other words, the port to use if one is not specified. I don’t see what’s so hard about specifying a different port if you want a different port.

From where I’m standing, Caddy configuration is already failing at orthogonality because it is special-casing all over the place, and if you fall outside the boundaries of those special cases, you just get weird results instead of clear explanations or things just working orthogonally.

There is not that much “special casing” – the requirements for automatic HTTPS are clearly outlined in the docs. It’s either on, or it’s not. What do you mean by “weird results”?

That you have to have automatic TLS in order to get dual http/https behavior is just weird

This is not true either. If you specify your own certificates, you can certainly have both HTTP and HTTPS. Caddy can’t read your mind to know what you want to do.

there is no reason for a user to expect this requirement, and Caddy doesn’t even issue a warning for it.

A warning for what? It prints the list of sites it’s serving, what are you looking for exactly?

nginx, to name one example, doesn’t require you to duplicate a server block or even the server name in order to serve the same site for both http and https. (It applies any https config to the https listener, and ignores it for the http.) Since nginx doesn’t do automatic certs, clearly the feature is possible without it.

Remember, Caddy’s goal is to NOT serve on HTTP as often as possible. Caddy is designed to serve HTTPS by default. Caddy is recommend by experts for easy, secure website setups. Nginx doesn’t have this as a goal. If you want HTTP by default and not HTTPS, then use another web server.

(Notice, by the way, that I’m not saying anything about redirecting http to https. I’m talking about serving the site under both – which is what I was originally trying to do here.)

http://example.com, https://example.com
tls cert.pem key.pem
...

That’s all it takes. HTTP and HTTPS with your own cert. Easy. And Caddy does print a warning for the site served over HTTP in this case.

All that being said, I personally consider the issue moot. At this point I’ve given up on Caddy for at least this use case (an ingress controller mapping multiple subpaths under multiple domains to specific docker containers), as in practice the configuration language has other issues besides this and the case-sensitivity one. (The more I’ve played around with it, the more issues I find, like inconsistencies between how paths are handled in the body of a server config vs. the key.)

We’ll get the bugs fixed – you’re welcome to help contribute. Can you elaborate on the other things you’d like to see changed? That would be more helpful than “this doesn’t work the way I want”.

And as I’ve skimmed through the issues and codebase looking for answers, I mostly keep finding indications that Caddy’s design consists of patching and repatching special cases (e.g. moving things up and down the middleware stack) until certain scenarios Just Work… which pretty much makes it something I can’t trust for this type of infrastructure, because who knows what will happen if I change some parameter down the road and I suddenly fall outside the Just Working scenarios?

All software is patching and repatching. Caddy has almost 200 separate contributors to this repo alone, so of course there will be lots of patching. I think you’re using the term “special cases” to mean “edge cases” or “bugs”. 😃 As for “moving things up and down the middleware stack”, how is this not a legitimate fix? Code has to run in a certain order for it to be correct. How do you propose that order is defined?

(And yeah, I get that it was originally supposed to be a single-user developer thing, but the configuration language at first blush looked like a readable and highly-usable alternative to traefik and nginx’s configuration languages.)

Indeed, Caddy is designed to adapt well to home development use as well as corporate production use. It sees a variety of use across all these spectrums.

Looking forward to your response.

but I literally don’t know what feature that is.

“The feature” I’m referring to is automatic HTTPS.

Not having to supply a certificate is merely an added bonus on top of that feature, in my view.

Not having to supply a certificate is a necessary part of the feature, as it doesn’t make sense without that part of it.

Hope that clears things up.

And that’s the heart of the disagreement, right there: to me, “the feature” is that specifying an address without a scheme gets you both http and https on the standard ports, with http redirecting to https. Apparently you have something else in mind for the definition of “this feature”, but I literally don’t know what feature that is.

If “the feature” is automatic Let’s Encrypt, then that strikes me as saying the feature is a drill, when what I want is a hole. How the certificate is obtained is not the (end user) feature, it’s merely a means to the end of serving http and https on standard ports.

Not having to supply a certificate is merely an added bonus on top of that feature, in my view.

But I guess we’ll have to agree to disagree, because we clearly disagree. 😄

Well, if the feature’s that important, why does it stop working when you provide your own certs?

Because that literally is the opposite of the feature, you can’t do both. I’m not going to introduce hybrid “sorta-auto-HTTPS” modes: it’s either fully automatic or you do things the classic, manual way.

Well, if the feature’s that important, why does it stop working when you provide your own certs? I mean, fixing that also makes it orthogonal and avoids documentation changes. And it is what I suggested first. 😉

@mholt: note that you could also make the behavior orthogonal by always requiring explicit addresses, and deprecating the implicit interpretation of scheme-free addresses. This has the advantage of also making the documentation simpler, since you would only need to remove a feature rather than add new documentation for the exception cases. 😄

@Whitestrake: the word “orthogonal” literally means “at right angles to each other”, but the metaphorical meaning used in IT and compsci is that things are on independent dimensions or axes. That is, if two axes or dimensions are at right angles, then moving something along one dimension does not move it on the other dimensions: changing the X position doesn’t change the Y position and vice versa.

Thus, language or application features are said to be orthogonal when they can be adjusted independently, without needing to think deeply about how things on “other dimensions” might be affected.

(The idea is closely related to separation of concerns, at least in the sense that a lack of orthogonality is often a symptom or “code smell” indicating an insufficient separation of concerns in the underlying architecture. In the case at hand, it points to the lack of separation between the concerns of address label interpretation, and automated certificate handling.)

I don’t think I’m a fan of the term orthogonal in this context; I don’t really understand how its meaning maps to what you’re using it for, but that could be my own lack of exposure to its use in this manner elsewhere. But you’ve explained it well enough, which I appreciate.

First, if it’s not unwelcome, a short opinion:

The headlining feature of Caddy, as I see it, is Automatic HTTPS. There’s a large amount of hand-holding involved in this feature, such as setting your HTTP redirection behaviour for you so you don’t need to worry about it. It’s the product differentiator, and a primary design concern.

If a user doesn’t want to use Automatic HTTPS, a lot of that hand-holding must go away, and the user needs to tell Caddy explicitly how they want to handle things. It’s an almost-complete, take-it-or-leave-it full TLS configuration. This makes sense to me, as long as the concept of Automatic HTTPS and its effects are well understood.

Now, outside of that opinion, there’s a few things addressed that I’m in full agreement with:

Documentation of features that affect, or are affected by, the Automatic HTTPS feature explicitly outlining the extent of those effects in either mode of operation. 👍

Having the ordering of directives to be exposed somewhere prominent in the documentation. It’s not infrequent that I have to check it, so I like this idea. 👍

Maybe even changing tls directive behaviour. I’m not sure about that, though - having manually supplied certificates not break Automatic HTTPS while other changes do break it is even more of an edge case.

The Caddy website documentation is also available here on Github. Outside of bringing the documentation shortcomings to the attention of the contributors, I have no doubt that any additional input on what the documentation should look like, in the form of a PR or an issue, will be welcomed as well.

We won’t be using 80 or 443 as default ports because they’re privileged. For spinning up Caddy for quick test or dev sites, that’s annoying. (Also, people don’t usually want ports 80 or 443 actually open and responding on their home network.) If it can’t bind to 80 or 443, then erroring out is a hassle, and changing to a higher port automatically sometimes is confusing.

I appreciate the feedback here but I’m not convinced this is a better idea either on technical grounds or usability ones. If we start splicing out the automatic HTTPS behavior into bits here and pieces there, depending on configuration, how Caddy works is going to get confusing really fast.

When you provide your own certificates, Caddy doesn’t change the ports for you (because automatic HTTPS is disabled) – so all you’ll have to do is https://test4.pjeby.com or test4.pjeby.com:443 and it will serve on the ports you designate. I know you want to use just a “plain” hostname for some reason but that’s simply not how Caddy works. (Thanks for filling out the issue template, though, it made this really easy to handle.)