stable-baselines3-contrib: a bug with MultiDiscrete action space when actions are not same size

when the environment has a action space that each action has different size, like this: self.action_space = MultiDiscrete([3,2])

and the action masker is like this for example: a = [[True, False, True],[Flase, True]]

following error happen since each “a” 's row is not same size:

`File ~\AppData\Roaming\Python\Python39\site-packages\sb3_contrib\common\maskable\distributions.py:228, in MaskableMultiCategoricalDistribution.apply_masking(self, masks)
    226 split_masks = [None] * len(self.distributions)
    227 if masks is not None:
--> 228     masks = th.as_tensor(masks)
    230     # Restructure shape to align with logits
    231     masks = masks.view(-1, sum(self.action_dims))

TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool.`

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 25 (9 by maintainers)

Most upvoted comments

maybe this is more clear, however, it’s not sequentially exactly. it’s better to say that the masking of the second action depends on which first action has been chosen.

Conditional action masking depending on a chosen action is not developed in the repo. It could be done, however.

Let’s say you have multi discrete space of [2, 3], where the masks for the 2nd part depends on which action do you take in the 1st part. When you take 0 in the 1st part, then masking for the 2nd would be: [True, False, True] and if you take 1, it would be: [True, True, False]. Current action mask would be a list of 5 elements (2+3), so it’s not sufficient. Instead you could pass action mask in form of: ([True, True], [[True, False, True], [True, True, False]]), where the first list has masking for the 1st part and the 2nd list has masking when you take 0 and when you take 1, stacked. Then you would also need to override functions in MaskableCategorical so it does apply this conditional masking correctly.