stable-baselines3-contrib: a bug with MultiDiscrete action space when actions are not same size
when the environment has a action space that each action has different size, like this:
self.action_space = MultiDiscrete([3,2])
and the action masker is like this for example:
a = [[True, False, True],[Flase, True]]
following error happen since each “a” 's row is not same size:
`File ~\AppData\Roaming\Python\Python39\site-packages\sb3_contrib\common\maskable\distributions.py:228, in MaskableMultiCategoricalDistribution.apply_masking(self, masks)
226 split_masks = [None] * len(self.distributions)
227 if masks is not None:
--> 228 masks = th.as_tensor(masks)
230 # Restructure shape to align with logits
231 masks = masks.view(-1, sum(self.action_dims))
TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool.`
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 1
- Comments: 25 (9 by maintainers)
Conditional action masking depending on a chosen action is not developed in the repo. It could be done, however.
Let’s say you have multi discrete space of [2, 3], where the masks for the 2nd part depends on which action do you take in the 1st part. When you take 0 in the 1st part, then masking for the 2nd would be: [True, False, True] and if you take 1, it would be: [True, True, False]. Current action mask would be a list of 5 elements (2+3), so it’s not sufficient. Instead you could pass action mask in form of: ([True, True], [[True, False, True], [True, True, False]]), where the first list has masking for the 1st part and the 2nd list has masking when you take 0 and when you take 1, stacked. Then you would also need to override functions in MaskableCategorical so it does apply this conditional masking correctly.