scipy: BUG maybe: fmin_bfgs possibly broken in 1.0

Kevin just started to test statsmodels against scipy 1.0 https://github.com/statsmodels/statsmodels/issues/3926

The main problem (besides a few deprecations that we missed) is that we get a large number of convergence failures mainly with fmin_bfgs

I’m just starting to look into it, so have nothing specific yet.

However, and old test case see below fails to converge to the correct parameters, that should be relatively straightforward. (I don’t know what the returned convergence status is, and it is possible that the current maxiter in statsmodels is too low now in this case.)

(Many other cases are difficult to optimize, but I just managed last week to clean up the unittest so they pass with all bfgs since scipy 0.14)

self = <statsmodels.discrete.tests.test_discrete.TestProbitBFGS object at 0x7f0b074db198>
    def test_params(self):
>       assert_almost_equal(self.res1.params, self.res2.params, DECIMAL_4)
E       AssertionError: 
E       Arrays are not almost equal to 4 decimals
E       
E       (mismatch 100.0%)
E        x: array([ 0.4412,  0.1027,  1.4389, -4.8224])
E        y: array([ 1.6258,  0.0517,  1.4263, -7.4523])

class TestProbitBFGS(CheckBinaryResults):

    @classmethod
    def setup_class(cls):
        data = sm.datasets.spector.load()
        data.exog = sm.add_constant(data.exog, prepend=False)
        cls.res1 = Probit(data.endog, data.exog).fit(method="bfgs",
            disp=0)
        res2 = Spector()
        res2.probit()
        cls.res2 = res2