rancher: Unable to run rancher host/agent on same machine that rancher/server container on

On my Ubuntu 16.04 it’s iptables issue, I flush any rules and set policy to ACCEPT, restart docker daemon - problem solved with default agent cmd from webUI, with CATTLE_AGENT_IP as real external IP.

UPD kernel: iptables: blocked: IN=docker0 OUT= PHYSIN=veth72d701a MAC=... SRC=172.17.0.3 DST=[PUBLIC-IP] LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=16008 DF PROTO=TCP SPT=38082 DPT=8080 WINDOW=29200 RES=0x00 SYN URGP=0

So, i just add this docker subnet to my rules

...
-A INPUT -s 138.XX.XX.YY/32 -j ACCEPT
-A INPUT -s 172.17.0.0/16 -j ACCEPT
-A INPUT -j LOG --log-prefix "iptables: blocked: "
-A INPUT -j DROP
...

UPD2 and another one, without this scheduler/healthcheck are broken -A INPUT-s 10.0.0.0/8 -j ACCEPT

UPD3 ALLinONE fix 4 my issue: iptables -I INPUT -i docker0 -j ACCEPT

I still have this issue?

hholst@fb-hholst3:~/test5$ sudo docker run -e CATTLE_AGENT_IP="10.46.161.106"  --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.0.2 http://localhost:8080/v1/scripts/777703ACD0B87294B61E:1478167200000:gofAQVGmSArBZlQ2BiqUVnRIA

INFO: Running Agent Registration Process, CATTLE_URL=http://10.46.161.106:8080/v1
INFO: Checking for Docker version >= 1.6.0
INFO: Found Server version: 1.12.3
INFO: docker version: Client version: 1.6.0
INFO: docker version: Client API version: 1.18
INFO: docker version: Go version (client): go1.4.2
INFO: docker version: Git commit (client): 4749651
INFO: docker version: OS/Arch (client): linux/amd64
INFO: docker version: Server version: 1.12.3
INFO: docker version: Server API version: 1.24
INFO: docker version: Go version (server): go1.6.3
INFO: docker version: Git commit (server): 6b644ec
INFO: docker version: OS/Arch (server): linux/amd64
INFO: docker info: Containers: 375
INFO: docker info: Images: 1186
INFO: docker info: Storage Driver: aufs
INFO: docker info: Root Dir: /var/lib/docker/aufs
INFO: docker info: Backing Filesystem: extfs
INFO: docker info: Dirs: 1677
INFO: docker info: Dirperm1 Supported: true
INFO: docker info: Execution Driver:
INFO: docker info: Kernel Version: 4.4.0-43-generic
INFO: docker info: Operating System: Ubuntu 16.04.1 LTS
INFO: docker info: CPUs: 56
INFO: docker info: Total Memory: 62.84 GiB
INFO: docker info: Name: fb-hholst3
INFO: docker info: ID: OCTE:L424:L5TD:4NQ3:ZCS2:AQFZ:HP64:HJBP:MNEB:6PSP:5YTE:A3G3
INFO: docker info: Http Proxy:
INFO: docker info: Https Proxy:
INFO: docker info: No Proxy:
INFO: docker info: Labels:
WARNING: No swap limit support
INFO: docker info: provider=generic
INFO: Attempting to connect to: http://10.46.161.106:8080/v1
INFO: http://10.46.161.106:8080/v1 is accessible
INFO: Inspecting host capabilities
INFO: Systemd: true
INFO: Boot2Docker: false
INFO: Host writable: true
INFO: Token: xxxxxxxx
INFO: Running registration
Traceback (most recent call last):
  File "./register.py", line 11, in <module>
    secret_key=os.environ['CATTLE_REGISTRATION_SECRET_KEY'])
  File "/usr/local/lib/python2.7/site-packages/cattle.py", line 45, in from_env
    return gdapi.from_env(prefix=prefix, factory=Client, **kw)
  File "/usr/local/lib/python2.7/site-packages/gdapi.py", line 608, in from_env
    return _from_env(prefix=prefix, factory=factory, **args)
  File "/usr/local/lib/python2.7/site-packages/gdapi.py", line 627, in _from_env
    return factory(**result)
  File "/usr/local/lib/python2.7/site-packages/cattle.py", line 12, in __init__
    super(Client, self).__init__(*args, **kw)
  File "/usr/local/lib/python2.7/site-packages/gdapi.py", line 195, in __init__
    self._load_schemas()
  File "/usr/local/lib/python2.7/site-packages/gdapi.py", line 313, in _load_schemas
    response = self._get_response(self._url)
  File "/usr/local/lib/python2.7/site-packages/gdapi.py", line 262, in _get_response
    headers=self._headers)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 437, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /v1 (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7faa31258e50>: Failed to establish a new connection: [Errno 111] Connection refused',))
hholst@fb-hholst3:~/test5$ 

I had the same issue. Fixed that by deleting the files under /var/lib/rancher (only the files, not the folders).

In case your server uses a domain name (i.e. rancher.server.com), edit the /etc/hosts file and add the server address to it. I prefer to use the internal IP address instead of the public IP because I had fewer problems this way.

Then, rerun the agent registration:

sudo docker run -e CATTLE_AGENT_IP=[PRIV-IP-ADDRESS] --add-host rancher.server.com:[PRIV-IP-ADDRESS] -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.0.2 https://rancher.server.com/v1/scripts/753155FC97D6375C1F3A:1468022400000:JWzjpNACU9EjJ3iRyGueT8ooCxQ

Hope it helps.