pytest: BUG: --lf running only a subset of last-failed tests

After running the pandas test suite and getting e.g. 20 test failures, running the same command with " --lf" appended should run those 20 tests. In practice I’m seeing only fraction of these run on the subsequent run.

$ pytest pandas/tests --skip-slow --skip-db
[...]
=================================================================== short test summary info ====================================================================
FAILED pandas/tests/resample/test_resampler_grouper.py::test_apply_columns_multilevel - KeyError: Timestamp('2017-01-01 00:00:00', freq='H')
FAILED pandas/tests/groupby/test_function.py::test_median_empty_bins[True] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="0") are different
FAILED pandas/tests/groupby/test_function.py::test_ops_general[count-size] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="0") are different
FAILED pandas/tests/groupby/test_grouping.py::TestGrouping::test_evaluate_with_empty_groups[agg-expected1] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_aggregate_item_by_item - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>0] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column ...
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>1] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column ...
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>2] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column ...
FAILED pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-mean-mean] - AssertionError: Attributes of DataFrame.iloc[:, 0] (col...
FAILED pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-median-<lambda>] - AssertionError: Attributes of DataFrame.iloc[:, 0...
FAILED pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-var-<lambda>] - AssertionError: Attributes of DataFrame.iloc[:, 0] (...
FAILED pandas/tests/groupby/aggregate/test_other.py::test_agg_lambda_with_timezone - ValueError: Must produce aggregated value
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[NotImplementedError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[RuntimeError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[KeyError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[IndexError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[OSError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[ValueError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[ArithmeticError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[AttributeError] - AssertionError: Attributes of Series are different
====================================== 20 failed, 5034 passed, 1652 skipped, 374 xfailed, 1 warning in 208.51s (0:03:28) =======================================

$ pytest pandas/tests/resample pandas/tests/groupby pandas/tests/generic/ --skip-slow --lf
[...]
============================================================== 1 failed, 1455 deselected in 0.68s ==============================================================

Running pytest 6.0.2, py3.8.5, OSX 10.15.5