puffing-billy: Chrome 72 breaking stubs

To overcome below error

routines:ssl3_read_bytes:sslv3 alert certificate unknown:ssl/record/rec_layer_s3.c:1544:SSL alert number 46

I’ve added following option

options.add_argument('--ignore-certificate-errors')

I’ve figured out how to install two different version of Google Chrome. I can use Chrome 72 for my main browser, and I have a separate installation of Chrome 71 that I use to run my tests (until this issue is sorted out.)

I downloaded Chrome 71 from the link above, and I dragged the “Google Chrome” application into /Applications. When Mac OS asked about the duplicate file, I chose “Keep Both” (which copied it to “Google Chrome 2”.) Then I renamed “Google Chrome 2” to “Google Chrome 71”.

I have the os gem in my Gemfile (gem 'os'), which I use to detect Mac (development) or Linux (CI and prod).

In my Capybara configuration, I set the binary in the chrome options:

options = Selenium::WebDriver::Chrome::Options.new
# ...
if OS.mac?
  # Capybara / puffing-billing proxy is broken on Chrome 72
  options.binary = '/Applications/Google Chrome 71.app/Contents/MacOS/Google Chrome'
end

This option makes chromedriver use the Google Chrome at /Applications/Google Chrome 71.app.

Here’s the complete driver configuration that I’m using with Capybara:

# From: https://github.com/oesmith/puffing-billy/issues/193
Capybara.register_driver :headless_chrome_billy do |app|
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    acceptInsecureCerts: true,
    loggingPrefs: { browser: 'ALL' }
  )
  options = Selenium::WebDriver::Chrome::Options.new
  # options.headless!
  options.add_argument('--disable-gpu')
  # options.add_argument('--disable-web-security')

  if OS.mac?
    # Capybara / puffing-billing proxy is broken on Chrome 72
    options.binary = '/Applications/Google Chrome 71.app/Contents/MacOS/Google Chrome'
  end

  # See: https://developers.google.com/web/updates/2017/04/headless-chrome
  options.add_argument('--no-sandbox')

  options.add_argument('--window-size=1280,1000')
  options.add_argument("--proxy-server=#{Billy.proxy.host}:#{Billy.proxy.port}")
  proxy_bypass_list = [
    '127.0.0.1',
    'localhost',
  ].join(';')
  options.add_argument("--proxy-bypass-list=#{proxy_bypass_list}")

  Capybara::Selenium::Driver.new app,
    browser: :chrome,
    options: options,
    desired_capabilities: capabilities,
    driver_opts: {
      log_path: Rails.root.join('log/chromedriver.log').to_s,
      verbose: true,
    }
end
Capybara.javascript_driver = :headless_chrome_billy

Hope that helps someone, and I look forward to getting this fixed in Chrome 72.

@ndbroadbent agreed, the --tls13-variant=disabled switch doesn’t seem to have any effect in headless mode.

I’ve done some more tests on this and managed to get something working with headless, but it requires code changes in puffing-billy and pointing to the eventmachine master branch.

eventmachine pushed some changes to deal with TLS 1.3 a few weeks ago, so I’ve first tried updating it and hoped that everything would work fine, but the Change Cipher TLS_AES_128_GCM_SHA256 still happens and lead to the same infinite loop.

This being said, it is possible for puffing-billy to be more restrictive when it initiates the TLS handshake and explicitly disable TLS 1.3 by passing an extra ssl_version: ['TLSv1_2'] argument to this call https://github.com/oesmith/puffing-billy/blob/088bbc94ec91076df7f28fa9630d4b84ddc761e5/lib/billy/proxy_connection.rb#L56

This unfortunately only works in combination with the eventmachine TLS1.3 changes because unless specifically disabled, OpenSSL will default to having it in its list of supported protocols.

TL; DR; There seems to be two options in fixing this: explicitly disabling TLS1.3 support in puffing-billy once eventmachine gets released or figuring out why the Change Cipher triggers this infinite loop. I unfortunately don’t have time at this point to dig deeper in how eventmachine works, but maybe previous contributors to puffing-billy could pitch in.