scion: Runtime panic (invalid memory address or nil pointer dereference) when setting up registry for hidden paths in local AS

When trying to set up a registry for hidden segments locally (see config below), the control service has a runtime panic.

Tested version: SCIONLab package 4.6.2 on Ubuntu 22.04

Stack trace:

github.com/scionproto/scion/go/lib/log.HandlePanic
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/lib/log/log.go:132
runtime.gopanic
  /usr/local/go/src/runtime/panic.go:1038
runtime.panicmem
  /usr/local/go/src/runtime/panic.go:221
runtime.sigpanic
  /usr/local/go/src/runtime/signal_unix.go:735
github.com/scionproto/scion/go/lib/infra/messenger.AddressRewriter.buildFullAddress
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/lib/infra/messenger/addr.go:178
github.com/scionproto/scion/go/lib/infra/messenger.AddressRewriter.redirectToQUIC
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/lib/infra/messenger/addr.go:108
github.com/scionproto/scion/go/lib/infra/messenger.AddressRewriter.RedirectToQUIC
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/lib/infra/messenger/addr.go:76
github.com/scionproto/scion/go/cs/onehop.(*AddressRewriter).RedirectToQUIC
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/cs/onehop/addr.go:67
github.com/scionproto/scion/go/pkg/grpc.(*QUICDialer).Dial
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/pkg/grpc/dialer.go:159
github.com/scionproto/scion/go/pkg/hiddenpath/grpc.(*Discoverer).Discover
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/pkg/hiddenpath/grpc/discovery.go:37
github.com/scionproto/scion/go/pkg/hiddenpath.resolve
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/pkg/hiddenpath/discovery.go:96
github.com/scionproto/scion/go/pkg/hiddenpath.RegistrationResolver.Resolve
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/pkg/hiddenpath/discovery.go:55
github.com/scionproto/scion/go/pkg/hiddenpath.(*BeaconWriter).Write.func1
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/pkg/hiddenpath/beaconwriter.go:108
github.com/scionproto/scion/go/pkg/hiddenpath.(*remoteWriter).run
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/pkg/hiddenpath/beaconwriter.go:155
github.com/scionproto/scion/go/pkg/hiddenpath.(*BeaconWriter).Write.func3
  /builds/PRV-PERRIG/scionlab/scion-builder/scion/go/pkg/hiddenpath/beaconwriter.go:123

The problematic line seems to be

	if len(p.Metadata().Interfaces) == 0 { //when local AS

so I assume that for the concrete path Metadata() returns nil.

HP config:

---
groups:
  ffaa:1:1099-000a:
    owner: 17-ffaa:1:1099
    readers:
    - 17-ffaa:1:1099
    registries:
    - 17-ffaa:1:1099
    writers:
    - 17-ffaa:1:1099

registration_policy_per_interface:
  1:
    - "ffaa:1:1099-000a"

I haven’t tested it with the up-to-date version in master yet, but the relevant code seems to be unchanged.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 21 (21 by maintainers)

Commits related to this issue

Most upvoted comments

Another thing I tried succesfully (after fixing a bug in the dispatcher), is to let the resolver get a dynamic port number by default. This has the advantage of not requiring any change in the configuration schema since there’s no need to specify a separate address:port for the resolver.

That works for now, and can be our preferred option. We will need to revisit during dispatcher removal though.

Yeah, but then, I doubt that having kept the resolver port static will make the removal critically simpler. We can design a new scheme at that point.

To summarize the options that I have at this point. I can do either or both of:

  1. Allow the explicit configuration of an address for the resolver (as a separate service).
  2. Give a dynamic port to the resolver service if not otherwise configured.

I can also allow the configuration of a separate resolver address for each service, but I am not sure I understand what this would be for. It would also be worth reflecting a bit on the interactions between the server config files and the topology file but if we go for number 1 we can make it a separate conversation.

I’ll assume we’re happy with just number 1. Anyone thinks it’s wrong, speak up!

Thanks for the clarification. I agree that communicating with the local registry shouldn’t itself be a problem. Since then I have continued down the path of resolving the connection issues until I reached the point where the hidden paths get successfully registered. I can tell whether it worked or not based on the logs, so I don’t really need some end-2-end proof of the pudding just yet.

Resolving the nil reference was just the tip of the iceberg. It seems that resolving global addresses to local (even internal to the same server) connections wasn’t an existing use case until now because it doesn’t work. I had to fix a couple of issues where a nil path was used where an empty path was expected, and even after that, I ran into deadline exceeded errors that I haven’t been able to root-cause yet.

I’ll update the issue when I have something more tangible to report.

Best, J-C

On Fri, Aug 4, 2023 at 12:11 PM Markus Legner @.***> wrote:

Markus, based on the rather contrived configuration you gave (which Matthias and I duplicated in our minimal test environment) it is hard to figure what result you expect. If you do expect some kind of successful end-to-end behavior, could you help me set things up correctly so I can verify that my fix addresses the real problem and not an irrelevant symptom?

Hi @jiceatscion https://github.com/jiceatscion. Thank you very much for all your debugging efforts.

I actually don’t believe that the configuration is that contrived: For example, this could be used to hide a path from any other AS but allow local end hosts to use it. So the local control service should be able to register segments there and other local services should be able to retrieve them.

Specifically, I’m experimenting with an additional service in the AS that retrieves and further distributes the hidden segments. I would expect this service to be able to communicate with the hidden-path registry.

— Reply to this email directly, view it on GitHub https://github.com/scionproto/scion/issues/4364#issuecomment-1665366757, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBLEYOAQGUWHSPO2PPECHILXTTDDRANCNFSM6AAAAAA2ET4HWY . You are receiving this because you were mentioned.Message ID: @.***>

Ah. NM. Matthias figured how to produce it.