sktime: [BUG] test failures in deep learning classifiers, regressors

Opening this issue to track known test failures with deep learning classifiers, regressors. Some of these seem to be due to interface or dependency degradation as tests were turned off.

Failures, with checkboxes (checked = fixed) and links to PR:

  • LSTMFCNRegressor - fails for multivariate output scenario after broadcasting
  • LSTMFCNRegressor - fails during build_network with x = AttentionLSTM(self.lstm_size)(x)
  • MACNNClassifier, MACNNRegressor - fail all tests with ValueError: A KerasTensor cannot be used as input to a TensorFlow function.
  • SimpleRNNClassifier, SimpleRNNRegressor - fail all tests as passed argument is lr, not learning_rate, as the network expects - noticed by @vandit98 and fixed in https://github.com/sktime/sktime/pull/6154
  • ResNetClassifier, ResNetRegressor fail test case for serialization when attempting to pickle a lambda
  • test_deep_estimator_full fails for regressors

Suggested procedure for debugging: pick one of these and ensure to run check_estimator locally until tests pass.

Test failures can be inspected here: https://github.com/sktime/sktime/pull/6138

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 15 (9 by maintainers)

Commits related to this issue

Most upvoted comments

Hi @fkiraly I have fixed the MACNNClassifier, MACNNRegressor - fail all tests with ValueError: A KerasTensor cannot be used as input to a TensorFlow function. Please review it.

Hi @fkiraly I just finished with check_estimator for all listed, please find them below.

LSTMFCNRegressor: FAILED: test_multioutput[LSTMFCNRegressor-1]. MACNNClassifier:FAILED: test_classifier_on_unit_test_data[MACNNClassifier], FAILED: test_multioutput[MACNNClassifier-0], FAILED: test_multioutput[MACNNClassifier-1]. MACNNRegressor: FAILED: test_multioutput[MACNNRegressor-0], FAILED: test_multioutput[MACNNRegressor-1]. SimpleRNNClassifier: All tests PASSED! SimpleRNNRegressor: All tests PASSED! ResNetClassifier: All tests PASSED! ResNetRegressor: All tests PASSED!

currently working on resolving the multi-variate output issue test_multioutput, please feel free to drop any guidelines/suggestions.

Thanks for your response @fkiraly, I am looking into it!

hi @fkiraly I have opened a new pr for the fix. Please review it and let me know if it needs any changes.