jaeger: jaeger agent can't connect to collector

jaeger-agent deployment command: docker run -d --name agent --rm -p5775:5775/udp -p6831:6831/udp -p6832:6832/udp -p5778:5778/tcp jaegertracing/jaeger-agent /go/bin/agent-linux --collector.host-port=localhost:14267

jaeger-collector deployment command: docker run -d --rm -p16686:16686/tcp jaegertracing/jaeger-query /go/bin/query-linux --es.server-urls=http://es-server:9200 --span-storage.type=elasticsearch --query.static-files=/go/jaeger-ui/

logs of jaeger-agent container ako p xs lc 1t_u 2 q0

The firewalld is inactive and SELINUX is disabled.

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 25 (8 by maintainers)

Most upvoted comments

@jpkrohling This the thing. The jager agent has own machine, collector and ui in the another machine. First I try to connect to es backend , but I cant catch any trace. So I try to use collector with memory backend. In the same situation I cant catch any trace. Here is my docker compose file:

version: "3"

services:
  jaeger-collector:
    image: jaegertracing/jaeger-collector
    ports:
      - "14269:14269"
      - "14268:14268"
      - "14267:14267"
      - "9411:9411"
    networks:
      - default
    restart: on-failure
    environment:
      - SPAN_STORAGE_TYPE=memory
    command: [
      #"--es.server-urls=http://172.31.60.138:9200",
      "--memory.max-traces=1000",
      "--log-level=debug"
    ]
    #depends_on:
    #  - elasticsearch

  #jaeger-agent:
  #  image: jaegertracing/jaeger-agent
  #  hostname: jaeger-agent
  #  command: ["--collector.host-port=jaeger-collector:14267"]
  #  ports:
  #    - "5775:5775/udp"
  #    - "6831:6831/udp"
  #    - "6832:6832/udp"
  #    - "5778:5778"
  #  networks:
  #    - elastic-jaeger
  #  networks:
  #    - elastic-jaeger
  #  restart: on-failure
  #  environment:
  #    - SPAN_STORAGE_TYPE=elasticsearch
  #  depends_on:
  #    - jaeger-collector

  jaeger-query:
    image: jaegertracing/jaeger-query
    environment:
      - SPAN_STORAGE_TYPE=memory
      - no_proxy=localhost
    ports:
      - "16686:16686"
      - "16687:16687"
    networks:
      - default
    restart: on-failure
    command: [
      #"--es.server-urls=http://172.31.60.138:9200",
      #"--span-storage.type=elasticsearch",
      "--memory.max-traces=1000",
      "--log-level=debug"
      #"--query.static-files=/go/jaeger-ui/"
    ]
    depends_on:
      - jaeger-collector

log show:

jaeger-collector_1  | 2018-08-09T08:52:31.737071735Z {"level":"info","ts":1533804751.736927,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"}
jaeger-collector_1  | 2018-08-09T08:52:31.737223952Z {"level":"info","ts":1533804751.7371488,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":1000}}
jaeger-collector_1  | 2018-08-09T08:52:31.737855280Z {"level":"info","ts":1533804751.737793,"caller":"static/strategy_store.go:77","msg":"No sampling strategies provided, using defaults"}
jaeger-collector_1  | 2018-08-09T08:52:31.737918697Z {"level":"info","ts":1533804751.7378654,"caller":"collector/main.go:141","msg":"Registering metrics handler with HTTP server","route":"/metrics"}
jaeger-collector_1  | 2018-08-09T08:52:31.737952308Z {"level":"info","ts":1533804751.7379005,"caller":"collector/main.go:149","msg":"Starting Jaeger Collector HTTP server","http-port":14268}
jaeger-collector_1  | 2018-08-09T08:52:31.737958490Z {"level":"info","ts":1533804751.737913,"caller":"healthcheck/handler.go:133","msg":"Health Check state change","status":"ready"}
jaeger-query_1      | 2018-08-09T08:52:32.321901870Z {"level":"info","ts":1533804752.321113,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":16687,"status":"unavailable"}
jaeger-query_1      | 2018-08-09T08:52:32.323285311Z {"level":"info","ts":1533804752.3231862,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":1000}}
jaeger-query_1      | 2018-08-09T08:52:32.323299988Z {"level":"info","ts":1533804752.3232355,"caller":"query/main.go:180","msg":"Archive storage not created","reason":"Archive storage not supported"}
jaeger-query_1      | 2018-08-09T08:52:32.551971761Z {"level":"info","ts":1533804752.5518212,"caller":"query/main.go:125","msg":"Registering metrics handler with HTTP server","route":"/metrics"}
jaeger-query_1      | 2018-08-09T08:52:32.552059577Z {"level":"info","ts":1533804752.5519016,"caller":"healthcheck/handler.go:133","msg":"Health Check state change","status":"ready"}
jaeger-query_1      | 2018-08-09T08:52:32.552112344Z {"level":"info","ts":1533804752.5520308,"caller":"query/main.go:134","msg":"Starting jaeger-query HTTP server","port":16686}

The agent log show:

jaeger-agent_1  | 2018-08-09T09:16:57.134805643Z {"level":"info","ts":1533806217.134292,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"}
jaeger-agent_1  | 2018-08-09T09:16:57.134845921Z {"level":"info","ts":1533806217.1343672,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"172.31.43.67:14267"}
jaeger-agent_1  | 2018-08-09T09:16:57.135284852Z {"level":"info","ts":1533806217.1351295,"caller":"agent/main.go:62","msg":"Starting agent"}
jaeger-agent_1  | 2018-08-09T09:16:58.135179196Z {"level":"info","ts":1533806218.1348429,"caller":"peerlistmgr/peer_list_mgr.go:157","msg":"Not enough connected peers","connected":0,"required":1}
jaeger-agent_1  | 2018-08-09T09:16:58.135204972Z {"level":"info","ts":1533806218.1349018,"caller":"peerlistmgr/peer_list_mgr.go:166","msg":"Trying to connect to peer","host:port":"172.31.43.67:14267"}
jaeger-agent_1  | 2018-08-09T09:16:58.139015123Z {"level":"info","ts":1533806218.138778,"caller":"peerlistmgr/peer_list_mgr.go:176","msg":"Connected to peer","host:port":"[::]:14267"}

thanks in advance !

lxkaka on Aug 9, 2018

--collector.host-port=127.0.0.1:14267

On Docker, 127.0.0.1 is the container itself, not the host where it’s running. Please, refer to Docker documentation about networking and/or linking containers.

https://docs.docker.com/network/ https://docs.docker.com/network/links/

jpkrohling on Jun 19, 2018

@jpkrohling thanks for the reply

I tried exposing the ports in the host. I got the below output

From Collector docker run --rm --env SPAN_STORAGE_TYPE=elasticsearch -p14267:14267/tcp -p14268:14268 -p9411:9411 jaegertracing/jaeger-collector /go/bin/collector-linux --es.server-urls=http://es-ip:9200 --log-level=debug

{“level”:“info”,“ts”:1529300740.434398,“caller”:“healthcheck/handler.go:99”,“msg”:“Health Check server started”,“http-port”:14269,“status”:“unavailable”} {“level”:“info”,“ts”:1529300740.6546476,“caller”:“static/strategy_store.go:77”,“msg”:“No sampling strategies provided, using defaults”} {“level”:“info”,“ts”:1529300740.6550906,“caller”:“collector/main.go:141”,“msg”:“Registering metrics handler with HTTP server”,“route”:“/metrics”} {“level”:“info”,“ts”:1529300740.655314,“caller”:“collector/main.go:149”,“msg”:“Starting Jaeger Collector HTTP server”,“http-port”:14268} {“level”:“info”,“ts”:1529300740.6554735,“caller”:“healthcheck/handler.go:133”,“msg”:“Health Check state change”,“status”:“ready”}

Form Agent {“level”:“info”,“ts”:1529300741.6044064,“caller”:“peerlistmgr/peer_list_mgr.go:166”,“msg”:“Trying to connect to peer”,“host:port”:“54.174.65.124:14267”} {“level”:“info”,“ts”:1529300741.6142902,“caller”:“peerlistmgr/peer_list_mgr.go:176”,“msg”:“Connected to peer”,“host:port”:“[::]:14267”}

It is showing that both Collector and Agent are connected to each other.

But once i run run the application, I can’t see any logs in Jaeger UI , It is not creating the service also. How to fix this?

How to find collector are collecting the logs.

roopagowda on Jun 18, 2018