qiskit-machine-learning: test_change_kernel and test_qsvr fail in clean stable 0.7 environment

Environment

Qiskit Machine Learning version: 0.7.1
Python version: 3.11.6
Operating system: Arch Linux

What is happening?

I’m trying to improve the PKGBUILD for AUR and run the python tests in the check() function.

The following tests fail in a clean chroot environment:

test/algorithms/regressors/test_qsvr.py::TestQSVR::test_change_kernel
test/algorithms/regressors/test_qsvr.py::TestQSVR::test_qsvr
test/algorithms/classifiers/test_fidelity_quantum_kernel_pegasos_qsvc.py::TestPegasosQSVC::test_save_load (core dumped)

_________________________ TestQSVR.test_change_kernel __________________________

self = <test.algorithms.regressors.test_qsvr.TestQSVR testMethod=test_change_kernel>

    def test_change_kernel(self):
        """Test QSVR with QuantumKernel later"""
        qkernel = FidelityQuantumKernel(feature_map=self.feature_map)
    
        qsvr = QSVR()
        qsvr.quantum_kernel = qkernel
        qsvr.fit(self.sample_train, self.label_train)
        score = qsvr.score(self.sample_test, self.label_test)
    
>       self.assertAlmostEqual(score, 0.38359, places=4)
E       AssertionError: 0.38411965819305205 != 0.38359 within 4 places (0.0005296581930520627 difference)

test/algorithms/regressors/test_qsvr.py:71: AssertionError

______________________________ TestQSVR.test_qsvr ______________________________

self = <test.algorithms.regressors.test_qsvr.TestQSVR testMethod=test_qsvr>

    def test_qsvr(self):
        """Test QSVR"""
        qkernel = FidelityQuantumKernel(feature_map=self.feature_map)
    
        qsvr = QSVR(quantum_kernel=qkernel)
        qsvr.fit(self.sample_train, self.label_train)
        score = qsvr.score(self.sample_test, self.label_test)
    
>       self.assertAlmostEqual(score, 0.38359, places=4)
E       AssertionError: 0.38411965819305205 != 0.38359 within 4 places (0.0005296581930520627 difference)

test/algorithms/regressors/test_qsvr.py:60: AssertionError

test/algorithms/classifiers/test_fidelity_quantum_kernel_pegasos_qsvc.py::TestPegasosQSVC::test_save_load Fatal Python error: Aborted

Current thread 0x00007f424fe64740 (most recent call first):
  File "/usr/lib/python3.11/pickle.py", line 578 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 902 in save_tuple
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 1002 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 887 in save_tuple
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 959 in _batch_appends
  File "/usr/lib/python3.11/pickle.py", line 932 in save_list
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 887 in save_tuple
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 956 in _batch_appends
  File "/usr/lib/python3.11/pickle.py", line 932 in save_list
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 887 in save_tuple
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 959 in _batch_appends
  File "/usr/lib/python3.11/pickle.py", line 932 in save_list
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 1212 in save_module_dict
  File "/usr/lib/python3.11/pickle.py", line 560 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 717 in save_reduce
  File "/usr/lib/python3.11/pickle.py", line 603 in save
  File "/usr/lib/python3.11/site-packages/dill/_dill.py", line 412 in save
  File "/usr/lib/python3.11/pickle.py", line 998 in _batch_setitems
  File "/usr/lib/python3.11/pickle.py", line 972 in save_dict
  ...

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, psutil._psutil_linux, psutil._psutil_posix, symengine.lib.symengine_wrapper, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._isolve._iterative, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.linalg._flinalg, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.optimize._minpack2, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize.__nnls, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.special._ellip_harm_2, scipy.spatial.transform._rotation, scipy.optimize._direct, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.special.cython_special, scipy.stats._stats, scipy.stats.beta_ufunc, scipy.stats._boost.beta_ufunc, scipy.stats.binom_ufunc, scipy.stats._boost.binom_ufunc, scipy.stats.nbinom_ufunc, scipy.stats._boost.nbinom_ufunc, scipy.stats.hypergeom_ufunc, scipy.stats._boost.hypergeom_ufunc, scipy.stats.ncf_ufunc, scipy.stats._boost.ncf_ufunc, scipy.stats.ncx2_ufunc, scipy.stats._boost.ncx2_ufunc, scipy.stats.nct_ufunc, scipy.stats._boost.nct_ufunc, scipy.stats.skewnorm_ufunc, scipy.stats._boost.skewnorm_ufunc, scipy.stats.invgauss_ufunc, scipy.stats._boost.invgauss_ufunc, scipy.interpolate._fitpack, scipy.interpolate.dfitpack, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy._lib._uarray._uarray, scipy.stats._statlib, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._mvn, scipy.stats._rcont.rcont, sklearn.__check_build._check_build, sklearn.utils._isfinite, sklearn.utils.murmurhash, sklearn.utils._openmp_helpers, sklearn.utils._logistic_sigmoid, sklearn.utils.sparsefuncs_fast, sklearn.preprocessing._csr_polynomial_expansion, sklearn.preprocessing._target_encoder_fast, scipy.io.matlab._mio_utils, scipy.io.matlab._streams, scipy.io.matlab._mio5_utils, sklearn.datasets._svmlight_format_fast, sklearn.utils._random, sklearn.utils._vector_sentinel, sklearn.feature_extraction._hashing_fast, sklearn.utils._seq_dataset, sklearn.metrics.cluster._expected_mutual_info_fast, sklearn.metrics._dist_metrics, sklearn.metrics._pairwise_distances_reduction._datasets_pair, sklearn.utils._cython_blas, sklearn.metrics._pairwise_distances_reduction._base, sklearn.metrics._pairwise_distances_reduction._middle_term_computer, sklearn.utils._heap, sklearn.utils._sorting, sklearn.metrics._pairwise_distances_reduction._argkmin, sklearn.metrics._pairwise_distances_reduction._argkmin_classmode, sklearn.metrics._pairwise_distances_reduction._radius_neighbors, sklearn.metrics._pairwise_fast, sklearn.linear_model._cd_fast, sklearn._loss._loss, sklearn.utils.arrayfuncs, sklearn.svm._liblinear, sklearn.svm._libsvm, sklearn.svm._libsvm_sparse, sklearn.utils._weight_vector, sklearn.linear_model._sgd_fast, sklearn.linear_model._sag_fast (total: 154)
/startdir/PKGBUILD: line 38:   741 Aborted                 (core dumped) PYTHONPATH="test_dir/$_site_packages:$PYTHONPATH" pytest -v test/algorithms/classifiers/test_fidelity_quantum_kernel_pegasos_qsvc.py

About this issue

Original URL
State: open
Created 7 months ago
Comments: 20 (3 by maintainers)

Most upvoted comments

@oscar-wallis Thanks for the detailed analysis! I still didn’t have time to look to this in more detail. I think I will wait for the next release and I will run the tests again with qiskit 1.0.1.

iyanmv on Feb 28, 2024

Environment

Qiskit Machine Learning version: 0.7.1
Qiskit version: 0.46.0
Python version: 3.12.1
Operating system: MacOS, M2 chip

Investigation Update

Hi @iyanmv, @woodsp-ibm and I went and tested the test failing errors on my device where I can replicate the failures. We found we could pass the tests if we set enforce_psd = False in FidelityQuantumKernel as shown below.

    def test_qsvr(self):
        """Test QSVR"""
        qkernel = FidelityQuantumKernel(feature_map=self.feature_map, enforce_psd=False)
        qsvr = QSVR(quantum_kernel=qkernel)
        qsvr.fit(self.sample_train, self.label_train)
        score = qsvr.score(self.sample_test, self.label_test)

        self.assertAlmostEqual(score, 0.38359, places=4)

    def test_change_kernel(self):
        """Test QSVR with QuantumKernel later"""
        qkernel = FidelityQuantumKernel(feature_map=self.feature_map, enforce_psd=False)

        qsvr = QSVR()
        qsvr.quantum_kernel = qkernel
        qsvr.fit(self.sample_train, self.label_train)
        score = qsvr.score(self.sample_test, self.label_test)

        self.assertAlmostEqual(score, 0.38359, places=4)

enforce_psd is a variable (by default set to True) added previously to the BaseKernel which, if true, will mathematically “find the closest positive semi-definite approximation to a symmetric kernel matrix. The (symmetric) matrix should always be positive semi-definite by construction, but this can be violated in case of noise, such as sampling noise.” - that’s lifted verbatim from the documentation. It does this simply using numpy arrays and mathematics.

There is a follow on question of why enforce_psd is actually causing an issue. FidelityQuantumKernel uses qiskit.primitive.Sampler by default, this Sampler uses a simulator by default. This means the kernel matrix should always be positive semi-definite by default. Additionally, if we run using qiskit_aer.primitives.Sampler with shots = None, ie the ideal case, no error is thrown, whether enforce_psd is set to True or False. That code is shown below too.

    def test_qsvr(self):
        """Test QSVR"""
        from qiskit.algorithms.state_fidelities import ComputeUncompute
        from qiskit_aer.primitives import Sampler

        qkernel = FidelityQuantumKernel(
            fidelity=ComputeUncompute(sampler=Sampler(run_options={"shots": None})),
            feature_map=self.feature_map,
            enforce_psd=True,
        )
        qsvr = QSVR(quantum_kernel=qkernel)
        qsvr.fit(self.sample_train, self.label_train)
        score = qsvr.score(self.sample_test, self.label_test)

        self.assertAlmostEqual(score, 0.38359, places=4)

This points to qiskit.primitives.Sampler being the issue, although may be fixed with Qiskit 1.0, I’ll try to further isolate the issue and leave an issue over on the Qiskit repo. When I do, I’ll link it to this issue so you can follow up if you wish.

Action Taken

As this is an issue which seems to be pretty rare and doesn’t occur with the CI on the latest versions of ubuntu, mac and windows VMs, across a range of Python versions as mentioned by @woodsp-ibm, we don’t feel the repo needs changing, so no direct fix will be implemented. As for a local fix, in all honesty, I would recommend the qiskit_aer.primitives.Sampler fix. From my testing on this and other issues, qiskit_aer seems to be the most robust, however, this could change with Qiskit 1,0. If you don’t want that dependency added to this test or just want a simpler fix use the enforce_psd = False.
Issue to be isolated and issue posted to Qiskit repo

P.S. I really liked your issue format and copied it here, I’ll probably continue to use it so thanks!

oscar-wallis on Feb 28, 2024

I will try to investigate a little bit more the crash issue in the next few days and also check the CI pipeline you are using. I will comment if I figure something out.

iyanmv on Feb 22, 2024