portainer: Portainer to Portainer Agent container console integration stopped working

I run a Docker Swarm of 7 nodes. 3 are managers. I have been using a portainer and portainer-agent deployment successfully up until version 1.20.

The Swarm is set up to use an Apache as a front door (using virtual hosting) to the services running in the Swarm - including portainer.

ssl --> Apache --> reverse proxy to various services based on servername.

Here is the stack file I use to deploy the services: All nodes are Linux Ubuntu 18.10 running dockerd 18.09.0

version: "3.7"

services:
  # Portainer Swarm Management
  portainer-agent:
    image: portainer/agent:latest
    environment:
      AGENT_CLUSTER_ADDR: tasks.portainer-agent
      AGENT_PORT: 9001
      LOG_LEVEL: debug
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /data01/docker/volumes
        target: /var/lib/docker/volumes
    deploy:
      mode: global
      placement:
        constraints: [node.platform.os == linux]

  portainer:
    image: portainer/portainer:latest
    command: -H tcp://tasks.portainer-agent:9001 --tlsskipverify
    volumes:
      - type: bind
        source: /mnt/docker/core/portainer
        target: /data
    deploy:
      placement:
        constraints:
          - node.role == manager

networks:
  default:
    name: "prod"
    external: true

Example portainer agent err file:

portainer_portainer-agent.0.pl4smj566kjv@ucfo-wgmn01    | 2018/12/18 11:05:35 http error: Missing request signature headers (err=Unauthorized) (code=403)
portainer_portainer-agent.0.yjmxe744e2ic@ucfo-wgwk02    | 2018/12/18 11:05:35 websocketproxy: couldn't dial to remote backend url websocket: bad handshake

Corresponding error in portainer log:

portainer_portainer.1.u9ge6bxok8sf@ucfo-wgwk01    | 2018/12/18 11:07:38 http error: Unable to query endpoint (err=Endpoint is down) (code=503)

portainer_portainer.1.u9ge6bxok8sf@ucfo-wgwk01    | 2018/12/18 11:05:35 websocketproxy: couldn't dial to remote backend url websocket: bad handshake

Here are some other spurious errors from portainer indicating connection problems.

portainer_portainer.1.u9ge6bxok8sf@ucfo-wgwk01    | 2018/12/18 11:07:31 http: proxy error: Docker container identifier not found
portainer_portainer.1.u9ge6bxok8sf@ucfo-wgwk01    | 2018/12/18 11:07:38 http error: Unable to query endpoint (err=Endpoint is down) (code=503)

I have a second swarm installation that is exhibiting the same problems.