ccxt: SSL verify fail using ccxt asyncio, works using toy aiohttp script

Long story short - catch 22.

We wrap our bots within and behind a varnish cache to prevent api blacklisting and outboud leaks (keys) to unwanted destinations. We SSL terminate at our internal hitch to do this.

For synchronous CCXT calls using requests everything is perfectly. We set an environment variables to help requests along its way:

REQUESTS_CA_BUNDLE=/home/creslin/freqcache/cert/ca.pem
export REQUESTS_CA_BUNDLE

Using CCXT asyncio the error is ran into: ssl:None [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)]

Our CA root certificate is added to our host OS key-store. Creating a toy aiohttp script and executing within the same environment/shell/directory works, and gets the time from binance.
Removing the cert from the key store, predictably breaks the toy, but demonstrates the keystore is being used by aiothttp.

toy script working - a.py:

import aiohttp
import asyncio

async def main():
    session = aiohttp.ClientSession()
    async with session.get('https://api.binance.com/api/v1/time') as response:
        print(await response.text())
    await session.close()

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
python3 a.py 
{"serverTime":1533729699098}

however when using aiohttp via ccxt i get SSL errors: This is true is the root cert is in our keystore, or environment variables are set or not.

Are you aware of any other locations I should be looking for placing our root certificate so CCXT async methods can make us of it?

CCXT asyncio script - 1.py

import ccxt
import asyncio
import ccxt.async_support as ccxt_async

exchange = ccxt.binance({
    "apiKey": "123",
    "secret": "abc",
})

exchange_async = ccxt_async.binance({ 'verbose': True,
    "apiKey": "123",
    "secret": "abc",
})


async def fetch_closed_order(exchange_async, symbol):
    closedOrder = await exchange_async.fetchClosedOrders(symbol=symbol, since="", limit=500 , params={})
    return closedOrder

async def get_closed_orders(exchange_async, symbols):
    input_coroutines = [fetch_closed_order(exchange_async, symbol) for symbol in symbols]
    closedOrders = await asyncio.gather(*input_coroutines, return_exceptions=True)

    for closedOrder, symbol in zip(closedOrders, symbols):
        if not isinstance(closedOrder, list):
            print(exchange.id, symbol, 'error')
        else:
            print(exchange.id, symbol, 'ok')

    await exchange_async.close()
    return closedOrders


symbols=['ETH/BTC']

closedOrders = asyncio.get_event_loop().run_until_complete\
    (get_closed_orders(exchange_async, symbols))

for order in closedOrders:
	print(order)
Request: GET https://api.binance.com/api/v1/exchangeInfo {'User-Agent': 'python-requests/2.19.1', 'Accept-Encoding': 'gzip, deflate'} None
binance ETH/BTC error
binance https://api.binance.com/api/v1/exchangeInfo GET Cannot connect to host api.binance.com:443 ssl:None [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)]

https://github.com/ccxt/ccxt/blob/master/CONTRIBUTING.md#how-to-submit-an-issue

  • OS: ubuntu 18.04
  • Programming Language version: Python 3.6
  • CCXT version: Version: 1.17.45
  • Exchange: binance
  • Method: Any

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 17 (17 by maintainers)

Most upvoted comments

Hi Kroitor,

Thanks for the confirmation, always good to know its not environmental issue on my side. It would be really handy to be able to use an ENV to point to a custom CA.root file.

I know requests.adaptor is only used on ccxt sync calls, those guys are also on certifi.

requests adapters supports the ENV REQUESTS_CA_BUNDLE for using a pem outside of certifi; a similar such as CCXT_CA_BUNDLE would be a big help to those like myself with custom CAs.

My work-around at the moment is to append into certifi/cacert.pem. This is not as portable as flipping an ENV var as move ccxt-dockers between environments and production. We’re trying to backtest/dry test many hundreds of instances - thus the need to our api-cache to not hit api-limits.

To add to this, through process of elimination i added my ca-root into requests bundled ca-store.

At this point asyncio/aiohttp via ccxt started working

The following *seems true

  • Requests will use a custom CA if pointed to via REQUESTS_CA_BUNDLE= env variable
  • Natural aiohttp/asyncio will only use the hosts system store, per python does, no env vars honoured - so add custom root CA to OS keystore works
  • ccxt aiohttp/asyncio will ignore system store and REQUESTS_CA_BUNDLE, but will use the bundled requests certificate store, so append custom root CA into requests certifi cert.pem

thats perfect, thank you!

sorry thats my fault - i missed the import

id tried that, no joy

    "session": aiohttp.ClientSession (),
NameError: name 'aiohttp' is not defined

this one didnt work out, returns an error on start

"session": asyncio.ClientSession (),
AttributeError: module 'asyncio' has no attribute 'ClientSession'

@creslinux got it, will add support for env vars to asyncio certs, hopefully soon.