circuitpython: certificate chain verification bug in esp-idf

the following request works OK with and ESP32 with the NINA firmware, but not with the esp32s2 native wifi

response = requests.get("https://api.thingspeak.com/channels/1417/feeds.json?results=1")
 fails with 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "wifi_cheer.py", line 39, in <module>
  File "adafruit_requests.py", line 507, in get
  File "adafruit_requests.py", line 456, in request
  File "adafruit_requests.py", line 405, in _get_socket
  File "adafruit_requests.py", line 401, in _get_socket
OSError: Failed SSL handshake
>>>

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 20 (9 by maintainers)

Commits related to this issue

esp32s2: Update esp-idf submodule to include fix for #3424 This re-points the submodule to my personal fork of esp-idf. Users may need to `git submodule sync` in their existing trees when this change... — committed to jepler/circuitpython by jepler 4 years ago
Merge pull request #3691 from jepler/issue-3424 esp32s2: Update esp-idf submodule to include fix for #3424 — committed to adafruit/circuitpython by tannewt 4 years ago

Most upvoted comments

Hi, I think I’m still getting this bug today, even after updating my board (Feather S2) to the newest firmware from S3 and the 20201117 library bundle. Issue happens even if I use the firmware directly referenced in the commit for this fix (f8eed1f).

Interestingly, the examples from this bug (thingspeak.com, io.adafruit.com) do appear to work properly now, but my own domain does not. I’m suspecting that it’s an issue with my SSL certificate chain based on the SSL Labs report, but I’m not entirely sure there’s anything I can do on my end to fix it.

My code:

import wifi
import socketpool
import adafruit_requests
import ssl

for network in wifi.radio.start_scanning_networks():
    print(network, network.ssid, network.rssi, network.channel)

wifi.radio.stop_scanning_networks()

try:
    wifi.radio.connect("SSID_here", "psk_here")
except Exception as e:
    print("Unable to connect to wifi! Error: " + str(e))

pool = socketpool.SocketPool(wifi.radio)
request = adafruit_requests.Session(pool, ssl.create_default_context())
response = request.get("https://toke.haus")
print(response.status_code)
print(response.text)

print("done")

Result:

Traceback (most recent call last):
  File "code.py", line 18, in <module>
  File "adafruit_requests.py", line 594, in get
  File "adafruit_requests.py", line 572, in request
  File "adafruit_requests.py", line 441, in _get_socket
RuntimeError: Sending request failed

Edit: I updated my cert to a LetsEncrypt cert and it now works fine. I’m assuming my SSL cert provider was the cause of the issue and won’t be using them again. Might still be worth looking into as a potential blocker for others? FWIW, my cert was a PositiveSSL cert purchased from namecheap.com (issuer was sectigo).

krittick on Nov 17, 2020

@jepler Want to switch circuitpython to your IDF branch until it is integrated upstream? I think this is the right fix.

tannewt on Nov 14, 2020

This looks site-specific, so I think cert-related. I had never encountered the SSL handshake issue with my own server (HTTPS/TLS), but I can replicate with the link above. Doesn’t matter if the RTC is 2000 or 2020.

Saola 1 w/Wrover with ESP32S2
6.0.0-rc.0 on 2020-10-16
adafruit-circuitpython-bundle-6.x-mpy-20201101
Requests 1.7.2

anecdata on Nov 3, 2020

I ran that python script from this folder circuitpython/ports/esp32s2/esp-idf/components/mbedtls/esp_crt_bundle

esp-idf/components/mbedtls/esp_crt_bundle on  master via 🐍 system took 2s
➜ python gen_crt_bundle.py --input cacrt_all.pem


gen_crt_bundle.py: Parsing certificates from cacrt_all.pem
gen_crt_bundle.py: Successfully added 135 certificates
gen_crt_bundle.py: Successfully added 135 certificates in total

And then you end up with a new x509_crt_bundle

➜ gst
HEAD detached at 8bc19ba89
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	x509_crt_bundle

askpatrickw on Oct 28, 2020

Slowly trying to make some sense of this the way certificates are configured in the sdconfig is quite different for the ESP32S2 https://github.com/adafruit/circuitpython/blob/main/ports/esp32s2/esp-idf-config/sdkconfig.defaults#L548 vs NINA https://github.com/adafruit/nina-fw/blob/master/sdkconfig#L563

It is also not clear to me how to update the certificates used by the ids in the esp32s2 in the Docs it says the certificates used are quite old - Jan 2019 – and the ones in the NINA build are much newer

The bundle comes with the complete list of root certificates from Mozilla’s NSS root certificate store. Using the gen_crt_bundle.py python utility the certificates’ subject name and public key are stored in a file and embedded in the ESP32 binary.

When generating the bundle you may choose between:

The full root certificate bundle from Mozilla, containing more than 130 certificates. The current bundle was updated Wed Jan 23 04:12:09 2019 GMT.
A pre-selected filter list of the name of the most commonly used root certificates, reducing the amount of certificates to around 35 while still having around 90 % coverage according to market share statistics.
In addition it is possible to specify a path to a certificate file or a directory containing certificates which then will be added to the generated bundle.

It also indicates that one can specify a file when building so I think that is the route to examine …

jerryneedell on Oct 28, 2020

These are the API docs where I would start: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/protocols/esp_crt_bundle.html

@brentru might have experience with it for nina.

tannewt on Oct 13, 2020