ccxt: SSL verify fail using ccxt asyncio, works using toy aiohttp script
Long story short - catch 22.
We wrap our bots within and behind a varnish cache to prevent api blacklisting and outboud leaks (keys) to unwanted destinations. We SSL terminate at our internal hitch to do this.
For synchronous CCXT calls using requests everything is perfectly.
We set an environment variables to help requests along its way:
REQUESTS_CA_BUNDLE=/home/creslin/freqcache/cert/ca.pem
export REQUESTS_CA_BUNDLE
Using CCXT asyncio the error is ran into:
ssl:None [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)]
Our CA root certificate is added to our host OS key-store.
Creating a toy aiohttp script and executing within the same environment/shell/directory works, and gets the time from binance.
Removing the cert from the key store, predictably breaks the toy, but demonstrates the keystore is being used by aiothttp.
toy script working - a.py:
import aiohttp
import asyncio
async def main():
session = aiohttp.ClientSession()
async with session.get('https://api.binance.com/api/v1/time') as response:
print(await response.text())
await session.close()
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
python3 a.py
{"serverTime":1533729699098}
however when using aiohttp via ccxt i get SSL errors: This is true is the root cert is in our keystore, or environment variables are set or not.
Are you aware of any other locations I should be looking for placing our root certificate so CCXT async methods can make us of it?
CCXT asyncio script - 1.py
import ccxt
import asyncio
import ccxt.async_support as ccxt_async
exchange = ccxt.binance({
"apiKey": "123",
"secret": "abc",
})
exchange_async = ccxt_async.binance({ 'verbose': True,
"apiKey": "123",
"secret": "abc",
})
async def fetch_closed_order(exchange_async, symbol):
closedOrder = await exchange_async.fetchClosedOrders(symbol=symbol, since="", limit=500 , params={})
return closedOrder
async def get_closed_orders(exchange_async, symbols):
input_coroutines = [fetch_closed_order(exchange_async, symbol) for symbol in symbols]
closedOrders = await asyncio.gather(*input_coroutines, return_exceptions=True)
for closedOrder, symbol in zip(closedOrders, symbols):
if not isinstance(closedOrder, list):
print(exchange.id, symbol, 'error')
else:
print(exchange.id, symbol, 'ok')
await exchange_async.close()
return closedOrders
symbols=['ETH/BTC']
closedOrders = asyncio.get_event_loop().run_until_complete\
(get_closed_orders(exchange_async, symbols))
for order in closedOrders:
print(order)
Request: GET https://api.binance.com/api/v1/exchangeInfo {'User-Agent': 'python-requests/2.19.1', 'Accept-Encoding': 'gzip, deflate'} None
binance ETH/BTC error
binance https://api.binance.com/api/v1/exchangeInfo GET Cannot connect to host api.binance.com:443 ssl:None [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)]
https://github.com/ccxt/ccxt/blob/master/CONTRIBUTING.md#how-to-submit-an-issue
- OS: ubuntu 18.04
- Programming Language version: Python 3.6
- CCXT version: Version: 1.17.45
- Exchange: binance
- Method: Any
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 17 (17 by maintainers)
Hi Kroitor,
Thanks for the confirmation, always good to know its not environmental issue on my side. It would be really handy to be able to use an ENV to point to a custom CA.root file.
I know requests.adaptor is only used on ccxt sync calls, those guys are also on certifi.
requestsadapters supports the ENVREQUESTS_CA_BUNDLEfor using a pem outside of certifi; a similar such asCCXT_CA_BUNDLEwould be a big help to those like myself with custom CAs.My work-around at the moment is to append into certifi/cacert.pem. This is not as portable as flipping an ENV var as move ccxt-dockers between environments and production. We’re trying to backtest/dry test many hundreds of instances - thus the need to our api-cache to not hit api-limits.
To add to this, through process of elimination i added my ca-root into requests bundled ca-store.
At this point asyncio/aiohttp via ccxt started working
The following *seems true
REQUESTS_CA_BUNDLE=env variablethats perfect, thank you!
sorry thats my fault - i missed the import
id tried that, no joy
this one didnt work out, returns an error on start
@creslinux got it, will add support for env vars to asyncio certs, hopefully soon.