ansible-kafka-admin: Unable to get Topic Information when connecting to a Cluster using SSL

Hello,

first of all, I would like to thank you for this brilliant project. We are starting to implement Kafka and want to manage topics through Ansible. That said, this project fits perfectly. However, I just faced an issue, which might already be referenced in #123, but my error is slightly different. Here are the details:

Expected Behavior

run task kafka_info with resource: “topic” and get a list of all available topics

Actual Behavior

Ansible playbook stops with an error:

Additional information

  1. The same playbooks can successfully retrieve the broker and the topic-config through the kafka_info module. It only fails when I want to retrieve the topic information.
  2. The CN of the certificate is a super user of the Kafka Cluster.

Play to Reproduce the Problem

---
- hosts: kafka11t
  remote_user: root
  vars:
    # see roles/cert/default/main.yml for all available
    # parameters
    my_bootstrap_servers: "[kafka11t.example.com:9093](http://kafka11t.example.com:9093/),[kafka12t.example.com:9093](http://kafka12t.example.com:9093/),[kafka13t.example.com:9093](http://kafka13t.example.com:9093/),[kafka14t.example.com:9093](http://kafka14t.example.com:9093/)"
    my_zookeeper_servers: "[kafka11t.example.com:2182](http://kafka11t.example.com:2182/)"
    kafka_api_version: "2.6.0"

  tasks:
    # get information about topics
    - name: get topic information
      StephenSorriaux.ansible_kafka_admin.kafka_info:
        resource: "topic"
        bootstrap_servers: "{{ my_bootstrap_servers }}"
        api_version: "{{ kafka_api_version }}"
        security_protocol: "SSL"
        ssl_cafile: "/etc/pki/ca-trust/source/anchors/MMW-CA-SHA256.cer.crt"
        ssl_certfile: "/etc/pki/tls/certs/{{ ansible_fqdn }}.crt"
        ssl_keyfile: "/etc/pki/tls/private/{{ ansible_fqdn }}.pem"
      changed_when: false
      register: topics_info

    - name: "Display information for topic jh03"
      debug:
        var: topics_info['ansible_module_results']['jh03']
...

## Logs from the play with Ansible in debug mode

`ANSIBLE_DEBUG=true ansible-playbook my-awesome-playbook.yml`

[WARNING]: Module invocation had junk after the JSON data: <BrokerConnection node_id=2 host=kafka12t.example.com:9093 <connected> [IPv4 (‘10.100.210.63’, 9093)]>: Closing connection. <BrokerConnection node_id=4 host=kafka14t.example.com:9093 <connecting> [IPv4 (‘10.100.210.72’, 9093)]>: Closing connection. fatal: [kafka11t]: FAILED! => {“changed”: false, “msg”: "Error while getting topic from Kafka: KafkaManagerError: Connection is not ready, please check your client and server configurations. "}


## Specifications

  - Library version: 0.15.3
  - Result of `pip list` command: 
  - Kafka version:  Confluent Platform 7.0.1
  - Python version: 3.6.8
  - OS: Linux - RHEL 8.4 on Ansible Node

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 20 (8 by maintainers)

Most upvoted comments

Thanks for looking into it. I try to do some tests tomorrow or on Friday.

Thank you for the feedback @teebee, I just released the 0.20.0 version.

@jheinitz if by any chance you can give it a try, it would be great!