FinRL-Meta: [DEBUGGING HELP] ValueError: could not broadcast input array from shape (14,6) into shape (22,14)

In refernce to https://github.com/AI4Finance-Foundation/FinRL-Meta/blob/master/tutorials/1-Introduction/FinRL_PortfolioAllocation_NeurIPS_2020.ipynb

The train data is shaped as (7812, 19)

Passing the data to the env runs without any errors

cryptoEnv = cryptoPortfolioAllocationEnvironment(dataFrame=trainData, **envKwargs)

And when I call cryptoEnv.observation_space, the shape is (22, 14), which assume is a combination of the price and indicators:

14 tickers, 8 indicators

running activeEnv, _ = cryptoEnv.stableBaselineEnv()

returns


ValueError                                Traceback (most recent call last)
[<ipython-input-68-f822f4852cfe>](https://localhost:8080/#) in <module>
----> 1 activeEnv, _ = cryptoEnv.stableBaselineEnv()

2 frames
[<ipython-input-63-fd656b920b2f>](https://localhost:8080/#) in stableBaselineEnv(self)
    189     def stableBaselineEnv(self):
    190       sb = DummyVecEnv([lambda: self])
--> 191       obs = sb.reset()
    192       return sb, obs
    193 

[/usr/local/lib/python3.7/dist-packages/stable_baselines3/common/vec_env/dummy_vec_env.py](https://localhost:8080/#) in reset(self)
     62         for env_idx in range(self.num_envs):
     63             obs = self.envs[env_idx].reset()
---> 64             self._save_obs(env_idx, obs)
     65         return self._obs_from_buf()
     66 

[/usr/local/lib/python3.7/dist-packages/stable_baselines3/common/vec_env/dummy_vec_env.py](https://localhost:8080/#) in _save_obs(self, env_idx, obs)
     92         for key in self.keys:
     93             if key is None:
---> 94                 self.buf_obs[key][env_idx] = obs
     95             else:
     96                 self.buf_obs[key][env_idx] = obs[key]

ValueError: could not broadcast input array from shape (14,6) into shape (22,14)


What am I missing ?

Please let me know if you require any additional info.The function to generate the env is as follows

def stableBaselineEnv(self):
      sb = DummyVecEnv([lambda: self])
      obs = sb.reset()
      return sb, obs

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 48 (12 by maintainers)

Most upvoted comments

Right,.

closing this as it’s clearly not a finRL issue.

Cheers @XiaoYangLiu-FinRL. valuable learning experience.

#Opensourceisthefuture lol

After weeks of testing different solutions, I Finally got the problem sorted.

Before I assumed that it was something to do with the DataPreprocessor so I tried using the Yahhoo preprocessor to download the data. (NOT)

Then tried using my env in the tutorial notebook and everything checked out, the model training occurs. so I concluded that it wasn’t the issues

As noted earlier, in my project, the model training and data acquisition are done in seperate notebooks

pd.to_csv() was used to export the data and pd.read_csv(df,index_col =False )

so it seemed as though the issue is as a result of the one of the two.

The shape of the imported df was verified, but when passed to the env, a different shape is produced by df.loc[self.day,:]

What could some potential causes/solutions?

@XiaoYangLiu-FinRL