tensorforce: InvalidArgumentError on terminal observe call

Perhaps something is wrong with my code, but almost half the time when the episode ends, I get an assertion error when I run observe on my PPO agent:

Traceback (most recent call last):
  File "ll.py", line 208, in <module>
    main()
  File "ll.py", line 181, in main
    agent.give_reward(reward, terminal)
  File "ll.py", line 123, in give_reward
    self.agent.observe(reward=reward, terminal=terminal)
  File "c:\users\connor\desktop\tensorforce\tensorforce\agents\agent.py", line 534, in observe
    terminal=terminal, reward=reward, parallel=[parallel], **kwargs
  File "c:\users\connor\desktop\tensorforce\tensorforce\core\module.py", line 578, in fn
    fetched = self.monitored_session.run(fetches=fetches, feed_dict=feed_dict)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\training\monitored_session.py", line 754, in run
    run_metadata=run_metadata)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\training\monitored_session.py", line 1360, in run
    raise six.reraise(*original_exc_info)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\six.py", line 696, in reraise
    raise value
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\training\monitored_session.py", line 1345, in run
    return self._sess.run(*args, **kwargs)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\training\monitored_session.py", line 1418, in run
    run_metadata=run_metadata)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\training\monitored_session.py", line 1176, in run
    return self._sess.run(*args, **kwargs)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\client\session.py", line 956, in run
    run_metadata_ptr)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\client\session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\client\session.py", line 1359, in _do_run
    run_metadata)
  File "C:\Users\Connor\Anaconda3\envs\ll\lib\site-packages\tensorflow_core\python\client\session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [] [Condition x <= y did not hold element-wise:x (baseline-network-state.observe/baseline-network-state.core_observe/baseline-network-state.core_experience/memory.enqueue/strided_slice:0) = ] [18243] [y (baseline-network-state.observe/baseline-network-state.core_observe/baseline-network-state.core_experience/memory.enqueue/sub_2:0) = ] [17999]
         [[{{node Assert}}]]

My original theory was that I was accidentally calling observe again after setting terminal=True and before resetting the agent, or some other abuse of observe, but I prevented that from happening in my code, so I don’t believe that’s the case. Also, the episode runs completely fine, and I get through thousands of calls to observe without ever running to any issues. It’s only when terminal=True that it seems to occur.

Running on Windows 10 x64, with tensorflow-gpu v2.0.0 on an RTX2070, Tensorforce installed from the Github at commit 827febcf8ffda851e5e4f0c9d12d4a8e8502b282

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 21 (8 by maintainers)

Most upvoted comments

Make sure your memory is greater than (batch_size + 1) * max_episode_timesteps. If you increased the batch size without increasing memory=10000, you may have gone over. You could remove the memory=10000 line and have the agent set its own memory, and memory will be set automatically according to batch_size.