aspire: AddContainer does not have support for docker network

Hi,

Is there some roadmap for adding more docker container capabilities? Right now, I see that there is support for Volumes and Replicas using the ResourceAnnotations.

builder.AddContainer(..)
    .WithAnnotation(new VolumeMountAnnotation(..))
    .WithAnnotation(new ReplicaAnnotation(..))

I was looking to create a set of containers that use a docker network. If it’s something that is not meant to be done using ResourceAnnotations, could you guide me on how we should do it? I see no option to add arguments to the ‘docker run’ command?

Greetings!

P.S. I’m loving this project already 😃

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 1
  • Comments: 31 (18 by maintainers)

Most upvoted comments

I agree with @davidfowl, we should be using a dedicated Docker network for running an Aspire app (and also use the Aspire component names for container names, instead of Docker auto-generated names).

@Blind-Striker thank you for pointing out the Testcontainers project, looks interesting, I will take a closer look.

We shouldn’t add support for docker networks but we should make a default network so that containers can reference each other by host name.

Moving this up the priority list

We’ve made a change to the upcoming version of aspire to support container to container networking with the default aspire-network (https://github.com/dotnet/aspire/pull/5628). This isn’t extensible as we’re working towards a bigger network modeling change.

This should help with migration from docker-compose and it will also improve building resources where multiple containers talk to each other.

We shouldn’t add support for docker networks but we should make a default network so that containers can reference each other by host name.

Default networks in docker work’s some different.

We can see on Docker Docs “Containers that attach to a custom network use Docker’s embedded DNS server. The embedded DNS server forwards external DNS lookups to the DNS servers configured on the host.” at https://docs.docker.com/network/#dns-services

Service discovery is part of custom docker networks, and when we connect some containers at this network, we’re expecting that some containers talk to each other.

Kafka and zookeper, etcd and other service tools, elk stack or anything else, like a kong, nginx envoy etc need a custom docker network to use a docker service discovery.

I think we need to start getting concrete about what features we want here and what the benefits are going to be both in terms of local development, and what it means to deployment scenarios.

Are we just looking to provide a mechanism to fine tune the Docker network configuration? Or do we want to have a fully fletched network resource that we put in the manifest which deployment tools then need to interpret.

I think we are going to run into a translation problem between the network topology that we can represent in Aspire locally and what the deployment tools can represent.

For example, lets say that we want to have multiple Docker networks defined and used in a single Aspire. API TBD, but it might look something like this:

var net1 = builder.AddNetwork("net1");
var net2 = builder.AddNetwork("net2");

var app1 = builder.AddContainer("app1", "image1")
                  .AttachTo(net1);

var app2 = builder.AddContainer("app2", "image2")
                  .AttachTo(net1)
                  .AttachTo(net2);

var app2 = builder.AddContainer("app3", "image3")
                  .AttachTo(net2);

This would result in DCP creating two networks, and when we attach the containers to them. But what is the benefit here for local development?

The above is also problematic for deployment scenarios because target deployment environments won’t always be able to faithfully represent a complex network topology like this. For example when we deploy with AZD we deploy all the containers on the same container app environment. Would using networks cause us to create multiple container app environments to ensure network isolation?

… we couldn’t represent a container being on two networks so I guess that would have to be an error.

We’re not sure what to implement exactly. If you go directly through the docker network, then you circumvent the networking of our orchestrator which means it isn’t modeled and doesn’t show up anywhere. Projects don’t run as containers so you can’t attach projects to a container network. Right now, container to container networking is via the host and the orchestrator (dcp) manages proxies to make things like replicas work (see https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/networking-overview for more details). Service discovery might just work but we haven’t considered what it means to build the end to end such that if you turned your project into a container and added it to the app model that service discovery would continue to work as is (the follow on implications of those changes etc).

I think we will need to support this, but there has to be experimentation in how to expose this in the app model. I believe there’s support for it in the underlying orchestrator, but it’s not exposed.

TL;DR: We want to do this, but have decided this isn’t a priority for GA so no time has been spent exploring or fleshing out potential solutions.

Another potential approach could be to leverage Dotnet Test containers and integrate it to Aspire. Testcontainers is a widely-used and well-established library within the community, offering robust and tested functionalities.

If this aligns with the project goals, I’m ready to open an issue to explore integrating Testcontainers. Thoughts?

Dotnet Testcontainers GitHub

We’ve made a change to the upcoming version of aspire to support container to container networking with the default aspire-network (#5628). This isn’t extensible as we’re working towards a bigger network modeling change.

This should help with migration from docker-compose and it will also improve building resources where multiple containers talk to each other.

that’s fantastic to here; is there any visibility of the bigger network modeling change? IMHO, it would be very interesting to have the team do rounds of open standup/discussion like the early day of .NET CORE 😉 always keen to here background discussion about direction and choice that need to be agreed/disagreed etc.

Wanted to chime in as an engineer researching what migration would look like for our team,

Our developers have been utilizing docker-compose for the last 5-ish years but we’re desiring to migrate to Aspire. We use an external docker-compose network so that our docker-compose files that are specific to each of our services that reside in individual service-specific repositories can all communicate to each other.

Migration for us from docker-compose to Aspire would require the ability to spin up containers in Aspire with the ability to overload the network until we have all of our services on Aspire, only then could we switch to a theoretical “global” Aspire docker network.

We’re very interested in Aspire as it’d give us much easier control over how we spin up our services locally for development purposes.

I’m fine with that. I’m just trying to draw the perimeter on what me mean when we say that we support Docker networks 😉

Can we just create a singlet network for all containers? We want to go through the dcp proxy for them as well for container -> host comms

@ejsmith better support for integration testing is something we’re tracking in #1704

IMHO Network configuration would be good as it would allow true infrastructure as code and follow what best practices with regards layer separation and firewall/port configuration testing in the developer environments.