caddy: Build error regarding h2quic

Nothing needs to be done here. Caddy has its vendored version of quic-go, so the current state of the quic-go repository is irrelevant.

I have the same issue when building from source. I don’t see the vendored code in caddy. Could you please re-open this issue?

I’m following the build instructions from the README.md

depends on the state of the cyrrent master branch of the caddy repo. And there was the case of this setup breaking the build, and there’s no evidence that there are any measures taken to prevent that in the future.

Frankly, you shouldn’t be relying on master for any kind of stability. That’s effectively the development trunk of this repo. If you want stability you should always be using a tagged release or specific commits. It feels like you’re asking for the Caddy repo to stifle progress in the name of stability in the master branch, but that’s unrealistic.

I can definitely vouch for @elcore and that he’s doing what he can to help you. He’s not trolling.

We’ve also learned a lot here when it comes to go get and go mod. See the issue that @mholt opened on the Go upstream describing some of your complaints: https://github.com/golang/go/issues/31529. go mod is still a new tool, and everyone’s just trying to figure out how to use it properly.

With all that said, your feedback about this is definitely appreciated, it’s helping uncover edge cases of the build tooling. Thanks for that.

I’ve been playing with modules + Caddy all day to try to understand what is going on.

A few things that are not obvious:

• Caddy has a tagged release with modules. However, it’s a -beta release, and go tools don’t honor pre-releases unless they’re the only tag. This is buried here in the go tool docs:

By default, get chooses the latest tagged release version, such as v0.4.5 or v1.2.3. If there are no tagged release versions, get chooses the latest tagged prerelease version, such as v0.0.1-pre1. If there are no tagged versions at all, get chooses the latest known commit.

• Because of that design decision, go get github.com/mholt/caddy/caddy pulls down v0.11.5, the latest non-prerelease tag, which is not a module.

• Because Caddy v0.11.5 is not a module, running go get -u has no constraints on which version of russross/blackfriday is pulled down, so it gets the latest, which is v2, which is not compatible with Caddy’s current implementation which uses v1. Hence the errors undefined: blackfriday.HtmlRenderer etc.

• go get will always pull into a GOPATH:

When using modules, GOPATH is no longer used for resolving imports. However, it is still used to store downloaded source code (in GOPATH/pkg/mod) and compiled commands (in GOPATH/bin).

I haven’t yet figured out why everyone saw the errors they described here, but I suspect it has to do with a combination of these things, and lots of hidden magic in the Go tooling:

• The removal of our vendor folder for the beta release
• The presence of a prerelease value in the latest tag
• The value of the GO111MODULE env variable
• The state/version of the existing Caddy code in the GOPATH, if any
• Whether commands are run in or out of GOPATH
• Whether git clone was used first

In my testing, I’ve seen wildly different, difficult-to-explain behavior when any and all of these variables change the slightest bit.

It’s very sensitive.

However, it appears that starting from scratch, and once we have a stable 1.0 release, we can finally test the transition to Go modules… in production! AFTER tagging the stable release. (Edit: Someone suggested to me that using replace in go.mod might help.)

Sigh.

Hi,

I hope I can help you. Caddy v0.11.5 does not support modules. Because of this, these problems occur.

If you want to build Caddy v0.11.5, you will have to use these instructions. (GO111MODULES should be set to auto - export GO111MODULES=auto)

1. go get github.com/mholt/caddy
2. go get github.com/caddyserver/builds
3. cd $GOPATH/src/github.com/mholt/caddy/caddy
4. git checkout tags/v0.11.5
5. go run build.go

If you want to build Caddy v1.0.0-beta1, you will have to use these instructions.

1. export GO111MODULES=on This will not be necessary with Go 1.13 (it should be the default)
2. go get github.com/mholt/caddy
3. go get github.com/caddyserver/builds
4. cd $GOPATH/src/github.com/mholt/caddy/caddy
5. go run build.go

or

1. git clone https://github.com/mholt/caddy
2. cd caddy/caddy
3. go run build.go

Good evening,

I am running into this issue as well. Setting GO111MODULE to on does allow the go get call to continue, but I then run into the error shown below:

# github.com/mholt/caddy/caddyhttp/markdown/summary
../pkg/mod/github.com/mholt/caddy@v0.11.5/caddyhttp/markdown/summary/render.go:24:5: cannot use (*renderer)(nil) (type *renderer) as type blackfriday.Renderer in assignment:
	*renderer does not implement blackfriday.Renderer (missing RenderFooter method)
../pkg/mod/github.com/mholt/caddy@v0.11.5/caddyhttp/markdown/summary/summary.go:26:44: too many arguments to conversion to blackfriday.Markdown: blackfriday.Markdown(input, renderer literal, 0)
# github.com/mholt/caddy/caddytls
../pkg/mod/github.com/mholt/caddy@v0.11.5/caddytls/setup.go:174:28: cannot use value (type "github.com/xenolf/lego/certcrypto".KeyType) as type "github.com/go-acme/lego/certcrypto".KeyType in assignment
../pkg/mod/github.com/mholt/caddy@v0.11.5/caddytls/setup.go:354:4: cannot use config.Manager.KeyType (type "github.com/go-acme/lego/certcrypto".KeyType) as type "github.com/xenolf/lego/certcrypto".KeyType in field value

Is there something else I should be doing or setting?

Ok, let’s go through what go get does currently with GO111MODULE=off. First of all, it doesn’t crash - that’s good

Great to hear that from you.

It does load extra dependencies from the thirdparty repos, which is undesireable for the following reasons

Your reasons are totally valid, it does pull some thirdparty repos over the network.

Speaking of evidence, I don’t think just providing and example of the command running successfully now proves that it’s the right approach.

I was trying to show you, that the build will be successful, if you follow instructions I have provided.

At this point it looks like some kind of trolling from you, @elcore. I know you have something that works for you […].

I am very sorry, if you felt like I was trolling you! I was just trying to be helpful, the transition to modules is not as easy as it should have been. I do not expect anything in return, I am using my resources available to help this project and its community.

I even created a simple Docker container as a proof of concept a few minutes ago. https://github.com/elcore/caddy-docker

For us it still won’t work, as we tried that, and now we know better, and will be using much more strightforward git cloning.

It is great that you have found a solution that works for you.

In my testing, I’ve seen wildly different, difficult-to-explain behavior when any and all of these variables change the slightest bit.

I feel much less stupid now.

So, if you want v0.11.5, you will have to disable modules or leave it in auto mode

Disabling modules will not help. You still have to do git clone, as go get (even with -d) will fail.

😕 The whole point of me writing all that text is to explain the behavior there. It’s pretty easy to grasp if you know the rules of old vendoring and new modules system, however not everyone reads the release notes thoroughly.

I’m not using modules because I need a very particular version of the caddy (commit hashes are for an older version in my real use case) - and used the vendoring setup back then.

That is okay, you can use an older version of Caddy.

I’m also donig git clone instead of a go get -d (or even go get as is the example), because it doesn’t work if the latest version is “broken” (which is the case after switch to go modules for all the older Go versions or disabled GO111MODULES).

It is not broken, go get github.com/mholt/caddy works

Of course, moving to the latest version could solve some of the issues - but it has other problems (with dependencies) that are blocking me from doing just that. I’ll look into it later.

What is the issue?

I’m just saying that for now I’ve came up with a workaround that’s good enough for us - but this was not trivial. Please reconsider making caddy simply go-get-able.

Caddy is go-get-able…

We no longer use vendor, starting with 1.0 beta 1.

Are you in a GOPATH? Try setting export GO111MODULE=on and try again.

Alright, thanks. Closing this then.

Ok, let’s go through what go get does currently with GO111MODULE=off. First of all, it doesn’t crash - that’s good, but it means something have changed since I last lloked into it. Nontheless. It does load extra dependencies from the thirdparty repos, which is undesireable for the following reasons:

1. The versions of those dependencies are not pinned.
2. It’s a waste of reources, since they’re not gonig to be used (after we checkout to a tag with vendored versions).
3. They can break the build. In our current case it’s not going to happen, but in another other it will due to stricter netwrok access rules and vendoring policy.

The worst this is what exactly is being attemped to be downloaded depends on the state of the current master branch of the caddy repo. And there was the case of this setup breaking the build, and there’s no evidence that there are any measures taken to prevent that in the future. Speaking of evidence, I don’t think just providing and example of the command running successfully now proves that it’s the right approach. At this point it looks like some kind of trolling from you, @elcore. I know you have something that works for you, but have to try and explain why it works and what are the potential downsides of the solution. For us it still won’t work, as we tried that, and now we know better, and will be using much more strightforward git cloning.

Regarding Go 1.12.4 - definetly a good idea. We already got our toolchain updated after we encountered this issue.

Disabling modules will not help. You still have to do git clone

This works pretty well, you don’t have to use git clone, just git checkout

If you want to build Caddy v0.11.5, you will have to use these instructions. (GO111MODULE should be set to auto - export GO111MODULE=auto)

1. go get github.com/mholt/caddy
2. go get github.com/caddyserver/builds
3. cd $GOPATH/src/github.com/mholt/caddy/caddy
4. git checkout tags/v0.11.5
5. go run build.go

In my case the behavior is pretty obvious. Remember, I need an older version of caddy. With modules enabled usual go get will pull v0.11.5 that has vendor dir, which will, however, be ignored, and go get will try to download the dependencies from the respective sources. And those won’t be pinned by the mod file (and not from vendor dir). This obviously fails. The behavior is the same even if I do go get -d. That’s why I ended up doing git clone instead of go get -d, and then the good old git checkout <rev-i-want> to enforce proper version. This works like a charm because older caddy version still provides vendor directory - and everything that’s needed for it to work is actually vendored, as opposed to just version-pinned. Personally I like vendoring more than version pinning, and mostly because it’s much more easier to build some very old app version from just the app git repo. With pinning a lot of external services have to be up and provide old package versions - which is currently problematic for go.

By the way, it would be great if somebody notify go team about the issues we encounter here - as this is the very feedback they look on the new mod system.

I am here to help 😄. I wish you a wonderful day.

Hi,

None of the instructions in the thread above are working for me in the current (go 1.12) docker container.

The reason is that go run build.go in the README doesn’t work, as go modules are not actually git repositories, and are stored in $GOPATH/pkg/mod.

Here’s the procedure that is actually working for v1.0.0-beta1, starting from an empty GOPATH:

export GO111MODULE=on
go get -v -ldflags "-X github.com/mholt/caddy/caddy/caddymain.appVersion=1.0.0-beta1" github.com/mholt/caddy/caddy@v1.0.0-beta1
$GOPATH/bin/caddy -version

Not sure what the correct fix is for the documentation @mholt ?

Hi @elcore,

Thank you for the clear instructions, this helped me get it working! Much appreciated.