scrapy: sslv3 alert handshake failure when making a request

Hi there, I recently upgraded to the latest scrapy and on some sites SSL enabled sites I get an exception when trying to make requests to it, while on previous scrapy versions I didn’t have this issue. The issue can be seen by making a request with scrapy shell:

scrapy shell "https://www.gohastings.com/"

The error I get is: Retrying <GET https://www.gohastings.com/> (failed 1 times): <twisted.python.failure.Failure OpenSSL.SSL.Error: ('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert handshake failure'), ('SSL routines', 'SSL3_WRITE_BYTES', 'ssl handshake failure')>

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 36 (18 by maintainers)

Commits related to this issue

Switched to scrapy 1.1.0rc3 to fix SSL bug Scrapy prior to 1.1.0rc3 had an issue where it would fail to connect to certain servers with an error stating ```<twisted.python.failure.Failure OpenSSL.SSL... — committed to ddworken/xsscrapy by ddworken 8 years ago

Most upvoted comments

Well I have no idea what’s going on now.

I’ve uninstalled cryptography and pyopenssl and reinstalled them with and without brew prefix flags and pOpenSSL is linking to the newer homebrew ssl versions and everything is working.

So, to anybody who’s hitting this bug:

scrapy version -v | grep pyOpenSSL

will return one of the following:

pyOpenSSL : 16.0.0 (OpenSSL 1.0.2g  1 Mar 2016)
pyOpenSSL : 16.0.0 (OpenSSL 0.9.8zg 14 July 2015)

If it’s OpenSSL version 0.9.8 then you need to try again.

rm -rf ~/Library/Caches/pip
pip uninstall cryptography pyopenssl
env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install cryptography pyopenssl

So basically you can probably ignore my previous two posts.

kneufeld on Apr 6, 2016

Here ya go: https://github.com/scrapy/scrapy/pull/1909

kneufeld on Apr 7, 2016