go-restful: Tokenizer change in 3.10 breaks many URLs

I’m using https://github.com/emicklei/go-restful/tree/67c9f7e97871832a5773f8a7c66230c7e3322e20/examples/openapi as the test case.

This has the following content in go.mod:

module github.com/emicklei/go-restful/examples/openapi

go 1.14

require (
        github.com/emicklei/go-restful-openapi/v2 v2.8.0
        github.com/emicklei/go-restful/v3 v3.8.0
        github.com/go-openapi/spec v0.20.4
)

Running this:

❯ go install

❯ go run ./restful-openapi.go
2022/11/13 21:18:05 Get the API using http://localhost:8080/apidocs.json
2022/11/13 21:18:05 Open Swagger UI using http://localhost:8080/apidocs/?url=http://localhost:8080/apidocs.json

And in a second window:

❯ curl --silent -D - http://localhost:8080/apidocs.json | head -10
HTTP/1.1 200 OK
Content-Type: application/json
Date: Sun, 13 Nov 2022 20:20:57 GMT
Transfer-Encoding: chunked

{
 "swagger": "2.0",
 "info": {
  "description": "Resource for managing Users",
  "title": "UserService",

Upgraded to github.com/emicklei/go-restful/v3 v3.9.0:

❯ go get github.com/emicklei/go-restful/v3@v3.9.0
go: upgraded github.com/emicklei/go-restful/v3 v3.8.0 => v3.9.0

❯ go run ./restful-openapi.go
2022/11/13 21:21:25 Get the API using http://localhost:8080/apidocs.json
2022/11/13 21:21:25 Open Swagger UI using http://localhost:8080/apidocs/?url=http://localhost:8080/apidocs.json

Still good:

❯ curl --silent -D - http://localhost:8080/apidocs.json | head -10
HTTP/1.1 200 OK
Content-Type: application/json
Date: Sun, 13 Nov 2022 20:21:48 GMT
Transfer-Encoding: chunked

{
 "swagger": "2.0",
 "info": {
  "description": "Resource for managing Users",
  "title": "UserService",

But with 3.10:

❯ go get github.com/emicklei/go-restful/v3@v3.10.0
go: upgraded github.com/emicklei/go-restful/v3 v3.9.0 => v3.10.0

❯ go run ./restful-openapi.go
2022/11/13 21:22:09 Get the API using http://localhost:8080/apidocs.json
2022/11/13 21:22:09 Open Swagger UI using http://localhost:8080/apidocs/?url=http://localhost:8080/apidocs.json

It fails:

❯ curl --silent -D - http://localhost:8080/apidocs.json | head -10
HTTP/1.1 404 Not Found
Date: Sun, 13 Nov 2022 20:22:20 GMT
Content-Length: 19
Content-Type: text/plain; charset=utf-8

404: Page Not Found

Instead it suddenly requires a trailing slash:

❯ curl --silent -D - http://localhost:8080/apidocs.json/ | head -10
HTTP/1.1 200 OK
Content-Type: application/json
Date: Sun, 13 Nov 2022 20:22:50 GMT
Transfer-Encoding: chunked

{
 "swagger": "2.0",
 "info": {
  "description": "Resource for managing Users",
  "title": "UserService",

This is a breaking change that I did not expect in a stable release… And besides, the URL became ugly 😃