netbox-docker: LDAP + TLS not working after update to v3.2.6

Current Behavior

After the update from 3.2.5-ldap to 3.2.6, LDAP with ldaps://, and ldap:// + StartTLS aren’t working anymore. When using ldap:// + StartTLS, it throws this error: Caught LDAPError while authenticating testuser: CONNECT_ERROR({'result': -11, 'desc': 'Connect error', 'ctrls': [], 'info': '(unknown error code)'}). When using ldaps://, it throws this error: Caught LDAPError while authenticating testuser: SERVER_DOWN({'result': -1, 'desc': "Can't contact LDAP server", 'ctrls': [], 'info': '(unknown error code)'})

Using plaintext LDAP works, so the server is not down, and the error messages are misleading. I thought maybe there’s something wrong with the CA, or server certificate, and ran openssl verify <(openssl s_client -showcerts -connect contoso.com:636 </dev/null) inside the netbox container. This turned out to work as expected:

depth=1 C = XX, ST = YYY, L = Example City, O = The Company, CN = contoso AD CA, emailAddress = admin@contoso.com
verify return:1
depth=0 CN = dc2.contoso.com
verify return:1
/dev/fd/63: OK
DONE

My Dockerfile to bundle the CA certificate:

FROM netboxcommunity/netbox:v3.2.6

COPY contoso-ca-ad.crt /usr/local/share/ca-certificates/
RUN chmod a+r /usr/local/share/ca-certificates/contoso-ca-ad.crt \
    && update-ca-certificates

Expected Behavior

I expect to be able to log in.

Docker Compose Version

docker-compose version 1.25.0, build unknown (it’s the one packaged for Ubuntu 20.04)

Docker Version

Client: Docker Engine - Community
 Version:           20.10.15
 API version:       1.41
 Go version:        go1.17.9
 Git commit:        fd82621
 Built:             Thu May  5 13:19:23 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.15
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.9
  Git commit:       4433bf6
  Built:            Thu May  5 13:17:28 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.4
  GitCommit:        212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc:
  Version:          1.1.1
  GitCommit:        v1.1.1-0-g52de29d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

The git Revision

b45934cd9e3f5f8ee2dad181c778f2ab3712b4e9

The git Status

On branch release
Your branch is up to date with 'origin/release'.

nothing to commit, working tree clean

Startup Command

docker-compose up

NetBox Logs

⚙️ Applying database migrations
🧬 loaded config '/etc/netbox/config/configuration.py'                                                                                                                          🧬 loaded config '/etc/netbox/config/extra.py'                                          
🧬 loaded config '/etc/netbox/config/logging.py'                                                                                                                                🧬 loaded config '/etc/netbox/config/plugins.py'                                                                                                                                
Operations to perform:                                                                                                                                                            Apply all migrations: admin, auth, circuits, contenttypes, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, wireless
Running migrations:                                                                                                                                                               Applying contenttypes.0001_initial... OK                                              
  Applying auth.0001_initial... OK                                                                                                                                              
  Applying admin.0001_initial... OK                                                                                                                                               Applying admin.0002_logentry_remove_auto_add... OK                     
  Applying admin.0003_logentry_add_action_flag_choices... OK              
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying extras.0001_squashed... OK
  Applying tenancy.0001_squashed_0012... OK
  Applying tenancy.0002_tenant_ordering... OK
  Applying dcim.0001_squashed... OK
  Applying dcim.0002_squashed... OK
  Applying ipam.0001_squashed... OK
  Applying virtualization.0001_squashed_0022... OK
  Applying extras.0002_squashed_0059... OK
  Applying extras.0060_customlink_button_class... OK
  Applying extras.0061_extras_change_logging... OK
  Applying extras.0062_clear_secrets_changelog... OK
  Applying tenancy.0003_contacts... OK
  Applying tenancy.0004_extend_tag_support... OK
  Applying dcim.0003_squashed_0130... OK
  Applying ipam.0002_squashed_0046... OK
  Applying ipam.0047_prefix_depth_children... OK
  Applying ipam.0048_prefix_populate_depth_children...
Updating 0 prefixes...
 OK
  Applying ipam.0049_prefix_mark_utilized... OK
  Applying ipam.0050_iprange... OK
  Applying ipam.0051_extend_tag_support... OK
  Applying extras.0063_webhook_conditions... OK
  Applying extras.0064_configrevision... OK
  Applying ipam.0052_fhrpgroup... OK
  Applying ipam.0053_asn_model... OK
  Applying dcim.0131_consoleport_speed... OK 
  Applying dcim.0132_cable_length... OK
  Applying dcim.0133_port_colors... OK
  Applying dcim.0134_interface_wwn_bridge... OK
  Applying dcim.0135_tenancy_extensions... OK
  Applying dcim.0136_device_airflow... OK
  Applying dcim.0137_relax_uniqueness_constraints... OK
  Applying dcim.0138_extend_tag_support... OK
  Applying dcim.0139_rename_cable_peer... OK 
  Applying wireless.0001_wireless... OK
  Applying dcim.0140_wireless... OK
  Applying dcim.0141_asn_model... OK
  Applying dcim.0142_rename_128gfc_qsfp28... OK
  Applying dcim.0143_remove_primary_for_related_name... OK
  Applying dcim.0144_fix_cable_abs_length... OK
  Applying dcim.0145_site_remove_deprecated_fields... OK
  Applying ipam.0054_vlangroup_min_max_vids... OK
  Applying virtualization.0023_virtualmachine_natural_ordering... OK
  Applying virtualization.0024_cluster_relax_uniqueness... OK
  Applying virtualization.0025_extend_tag_support... OK
  Applying virtualization.0026_vminterface_bridge... OK
  Applying extras.0065_imageattachment_change_logging... OK
  Applying extras.0066_customfield_name_validation... OK
  Applying extras.0067_customfield_min_max_values... OK
  Applying extras.0068_configcontext_cluster_types... OK
  Applying extras.0069_custom_object_field... OK
  Applying extras.0070_customlink_enabled... OK
  Applying ipam.0055_servicetemplate... OK
  Applying ipam.0056_standardize_id_fields... OK
  Applying ipam.0057_created_datetimefield... OK
  Applying circuits.0001_squashed... OK
  Applying circuits.0002_squashed_0029... OK 
  Applying circuits.0003_extend_tag_support... OK
  Applying circuits.0004_rename_cable_peer... OK
  Applying circuits.0032_provider_service_id... OK
  Applying circuits.0033_standardize_id_fields... OK
  Applying circuits.0034_created_datetimefield... OK
  Applying circuits.0035_provider_asns... OK 
  Applying dcim.0146_modules... OK
  Applying dcim.0147_inventoryitemrole... OK 
  Applying dcim.0148_inventoryitem_component... OK
  Applying dcim.0149_inventoryitem_templates... OK
  Applying dcim.0150_interface_vrf... OK
  Applying dcim.0151_interface_speed_duplex... OK
  Applying dcim.0152_standardize_id_fields... OK
  Applying dcim.0153_created_datetimefield... OK
  Applying django_rq.0001_initial... OK
  Applying extras.0071_standardize_id_fields... OK
  Applying extras.0072_created_datetimefield... OK
  Applying extras.0073_journalentry_tags_custom_fields... OK
  Applying sessions.0001_initial... OK
  Applying social_django.0001_initial... OK
  Applying social_django.0002_add_related_name... OK
  Applying social_django.0003_alter_email_max_length... OK
  Applying social_django.0004_auto_20160423_0400... OK
  Applying social_django.0005_auto_20160727_2333... OK
  Applying social_django.0006_partial... OK
  Applying social_django.0007_code_timestamp... OK
  Applying social_django.0008_partial_timestamp... OK
  Applying social_django.0009_auto_20191118_0520... OK
  Applying social_django.0010_uid_db_index... OK
  Applying taggit.0001_initial... OK
  Applying taggit.0002_auto_20150616_2121... OK
  Applying taggit.0003_taggeditem_add_unique_index... OK
  Applying taggit.0004_alter_taggeditem_content_type_alter_taggeditem_tag... OK
  Applying tenancy.0005_standardize_id_fields... OK
  Applying tenancy.0006_created_datetimefield... OK
  Applying tenancy.0007_contact_link... OK
  Applying users.0001_squashed_0011... OK
  Applying users.0002_standardize_id_fields... OK
  Applying virtualization.0027_standardize_id_fields... OK
  Applying virtualization.0028_vminterface_vrf... OK
  Applying virtualization.0029_created_datetimefield... OK
  Applying wireless.0002_standardize_id_fields... OK
  Applying wireless.0003_created_datetimefield... OK
⚙️ Running trace_paths
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
Found no missing console port paths; skipping
Found no missing console server port paths; skipping
Found no missing interface paths; skipping
Found no missing power feed paths; skipping
Found no missing power outlet paths; skipping
Found no missing power port paths; skipping
Finished.
⚙️ Removing stale content types
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
⚙️ Removing expired user sessions
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
💡 Superuser Username: admin, E-Mail: admin@example.com
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
▶️  Running the startup script /opt/netbox/startup_scripts/000_users.py
▶️  Running the startup script /opt/netbox/startup_scripts/010_groups.py
▶️  Running the startup script /opt/netbox/startup_scripts/020_object_permissions.py
▶️  Running the startup script /opt/netbox/startup_scripts/030_custom_fields.py
▶️  Running the startup script /opt/netbox/startup_scripts/040_custom_links.py
▶️  Running the startup script /opt/netbox/startup_scripts/050_tags.py
▶️  Running the startup script /opt/netbox/startup_scripts/060_webhooks.py
▶️  Running the startup script /opt/netbox/startup_scripts/070_tenant_groups.py
▶️  Running the startup script /opt/netbox/startup_scripts/080_tenants.py
▶️  Running the startup script /opt/netbox/startup_scripts/090_regions.py
▶️  Running the startup script /opt/netbox/startup_scripts/110_sites.py
▶️  Running the startup script /opt/netbox/startup_scripts/120_locations.py
▶️  Running the startup script /opt/netbox/startup_scripts/130_rack_roles.py
▶️  Running the startup script /opt/netbox/startup_scripts/140_racks.py
▶️  Running the startup script /opt/netbox/startup_scripts/150_power_panels.py
▶️  Running the startup script /opt/netbox/startup_scripts/160_power_feeds.py
▶️  Running the startup script /opt/netbox/startup_scripts/170_manufacturers.py
▶️  Running the startup script /opt/netbox/startup_scripts/180_device_roles.py
▶️  Running the startup script /opt/netbox/startup_scripts/190_device_types.py
▶️  Running the startup script /opt/netbox/startup_scripts/200_devices.py
▶️  Running the startup script /opt/netbox/startup_scripts/210_dcim_interfaces.py
▶️  Running the startup script /opt/netbox/startup_scripts/220_platforms.py
▶️  Running the startup script /opt/netbox/startup_scripts/230_route_targets.py
▶️  Running the startup script /opt/netbox/startup_scripts/240_vrfs.py
▶️  Running the startup script /opt/netbox/startup_scripts/250_rirs.py
▶️  Running the startup script /opt/netbox/startup_scripts/260_asns.py
▶️  Running the startup script /opt/netbox/startup_scripts/270_aggregates.py
▶️  Running the startup script /opt/netbox/startup_scripts/280_prefix_vlan_roles.py
▶️  Running the startup script /opt/netbox/startup_scripts/290_cluster_types.py
▶️  Running the startup script /opt/netbox/startup_scripts/300_cluster_groups.py
▶️  Running the startup script /opt/netbox/startup_scripts/310_clusters.py
▶️  Running the startup script /opt/netbox/startup_scripts/320_vlan_groups.py
▶️  Running the startup script /opt/netbox/startup_scripts/330_vlans.py
▶️  Running the startup script /opt/netbox/startup_scripts/340_virtual_machines.py
▶️  Running the startup script /opt/netbox/startup_scripts/350_virtualization_interfaces.py
▶️  Running the startup script /opt/netbox/startup_scripts/360_prefixes.py
▶️  Running the startup script /opt/netbox/startup_scripts/370_ip_addresses.py
▶️  Running the startup script /opt/netbox/startup_scripts/380_primary_ips.py
▶️  Running the startup script /opt/netbox/startup_scripts/400_services.py
▶️  Running the startup script /opt/netbox/startup_scripts/420_providers.py
▶️  Running the startup script /opt/netbox/startup_scripts/440_circuit_types.py
▶️  Running the startup script /opt/netbox/startup_scripts/450_circuits.py
▶️  Running the startup script /opt/netbox/startup_scripts/460_cables.py
▶️  Running the startup script /opt/netbox/startup_scripts/470_contact_groups.py
▶️  Running the startup script /opt/netbox/startup_scripts/480_contact_roles.py
▶️  Running the startup script /opt/netbox/startup_scripts/490_contacts.py
✅ Initialisation is done.
⏳ Waiting for control socket to be created... (1/10)
2022/07/13 15:00:25 [info] 19#19 discovery started
2022/07/13 15:00:25 [notice] 19#19 module: python 3.9.2 "/usr/lib/unit/modules/python3.9.unit.so"
2022/07/13 15:00:25 [info] 6#6 controller started
2022/07/13 15:00:25 [notice] 6#6 process 19 exited with code 0
2022/07/13 15:00:25 [info] 21#21 router started
2022/07/13 15:00:25 [info] 21#21 OpenSSL 1.1.1n  15 Mar 2022, 101010ef
⚙️ Applying configuration from /etc/unit/nginx-unit.json
2022/07/13 15:00:26 [info] 25#25 "netbox" prototype started
2022/07/13 15:00:26 [info] 26#26 "netbox" application started
✅ Unit configuration loaded successfully
2022/07/13 15:00:28 [notice] 6#6 process 17 exited with code 0
2022/07/13 15:00:34 [info] 28#28 "netbox" application started
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET / HTTP/1.1" 200 91127 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safa
ri/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox-external.css?v=3.2.6 HTTP/1.1" 200 286568 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) Appl
eWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox-light.css?v=3.2.6 HTTP/1.1" 200 232175 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWe
bKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox.js?v=3.2.6 HTTP/1.1" 200 376180 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/53
7.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox-dark.css?v=3.2.6 HTTP/1.1" 200 374410 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWeb
Kit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox_logo.svg HTTP/1.1" 200 4719 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36
 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox_icon.svg HTTP/1.1" 200 835 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 
(KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox-print.css?v=3.2.6 HTTP/1.1" 200 727867 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWe
bKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/materialdesignicons-webfont-KSYPMDN6.woff2?v=5.9.55 HTTP/1.1" 200 325244 "https://netbox.contoso.com/static/netbox-e
xternal.css?v=3.2.6" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:40 +0000] "GET /static/netbox.ico HTTP/1.1" 200 1174 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHT
ML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
192.168.144.2 - - [13/Jul/2022:15:00:41 +0000] "GET /login/?next=/ HTTP/1.1" 200 8559 "https://netbox.contoso.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, 
like Gecko) Chrome/103.0.5060.114 Safari/537.36"
Caught LDAPError while authenticating testuser: SERVER_DOWN({'result': -1, 'desc': "Can't contact LDAP server", 'ctrls': [], 'info': '(unknown error code)'})
192.168.144.2 - - [13/Jul/2022:15:00:53 +0000] "POST /login/ HTTP/1.1" 200 9474 "https://netbox.contoso.com/login/?next=/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (
KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"

Content of docker-compose.override.yml

version: '3.4'
services:
  netbox:
    expose:
      - 8080
    networks:
      - default
      - traefik_default
    environment:
      REMOTE_AUTH_ENABLED: "true"
      REMOTE_AUTH_BACKEND: "netbox.authentication.LDAPBackend"
      AUTH_LDAP_SERVER_URI: "ldaps://contoso.com"
      #AUTH_LDAP_START_TLS: "True"
      AUTH_LDAP_BIND_DN: "ldapauth"
      AUTH_LDAP_BIND_PASSWORD: "totallytherealpassword"
      AUTH_LDAP_USER_SEARCH_BASEDN: "OU=employees,DC=contoso,DC=com"
      AUTH_LDAP_GROUP_SEARCH_BASEDN: "OU=groups,DC=contoso,DC=com"
      AUTH_LDAP_REQUIRE_GROUP_DN: "CN=Netbox_Login,OU=groups,Dc=contose,DC=com"
      AUTH_LDAP_IS_ADMIN_DN: "CN=Netbox_Admin,OU=groups,Dc=contoso,DC=com"
      AUTH_LDAP_IS_SUPERUSER_DN: "CN=Netbox_SuperUser,OU=groups,Dc=contoso,DC=com"
      LDAP_IGNORE_CERT_ERRORS: "false"
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_default"
      - "traefik.http.routers.netbox.entrypoints=websecure"
      - "traefik.http.routers.netbox.rule=Host(`netbox.contoso.com`)"
      - "traefik.http.routers.netbox.tls=true"
      - "traefik.http.services.netbox.loadbalancer.server.port=8080"

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 28 (3 by maintainers)

Most upvoted comments

As a workaround its a good solution. But I think if we get the option into Netbox it will benefit more users. To be consistent I choose the same way as was used for LDAP_IGNORE_CERT_ERRORS.

Will you amend the issue to request the option for specifying a directory also? Otherwise I don’t see it solving the issue that the debian build broke LDAPS handling. That is not respecting the system certificate store. We really should be setting that by default to match the previous (sane) behavior.

+4
kkthxbye-code on Jul 14, 2022

I spent some time debugging this and found the solution.

The image does not have file /etc/ldap/ldap.conf which contains

# TLS certificates (needed for GnuTLS)
TLS_CACERT	/etc/ssl/certs/ca-certificates.crt

The file is provided by package libldap-common which is NOT installed.

To fix this, just add installation of libldap-common package to the Dockerfile

I would create a merge request but the change is so trivial, that I hope this comment will suffice.

+2
jiri-lunacek on Jul 20, 2022

The Netbox LDAP Backend (based on django-auth-ldap) has at the moment no way to set the TLS_CACERTDIR option (which refers to that one)

I have opened an issue for Netbox that addresses the problem. I have prepared a fix from which I will open a PR when (or if) the issue is accepted.

+1
tobiasge on Jul 14, 2022

Setting LDAP_IGNORE_CERT_ERRORS: "true" works for me. But this is strange because I had before with image 3.2.5-ldap never problems with the certificate validation.

The new image uses debian as a base instead of alpine. Not sure if there’s actual differences between openldap-dev (alpine) and libldap-dev (debian) as they are the same product, but it seems that the debian version doesn’t use the system cacerts by default.

I think it might make sense for us to set TLS_CACERTDIR to /etc/ssl/certs by default. This would still allow the user to overwrite TLS_CACERTFILE to point to a specific cacert file.

+1
kkthxbye-code on Jul 14, 2022

Setting LDAP_IGNORE_CERT_ERRORS: "true" works for me. But this is strange because I had before with image 3.2.5-ldap never problems with the certificate validation.

+1
netsandbox on Jul 14, 2022
Read more comments on GitHub
← netbox-docker: Configuring BASE_PATH does not works
netbox-docker: LDAP broken after upgrade to 1.x →