godot_rl_agents: Code runs for a bit then crashes. (again)
Unfortunately, Itās happening again š¦. It sometimes works fine for like an hour, and sometimes I canāt for the life of me get it to go more than 5 secondsā¦
Here is the error I get:
(gdrl_conda) PS C:\Users\Jetpackjules\Documents\Godot Gamez\ANTv0.61> gdrl
run_name PPO/editor
SETTING WORKERS TO 1
{'algorithm': 'PPO', 'RAY_IGNORE_UNHANDLED_ERRORS': 1, 'reuse_actors': True, 'stop': {'episode_reward_mean': 50000, 'training_iteration': 1000000, 'timesteps_total': 2000000000}, 'config': {'env': 'godot', 'env_config': {'framerate': None, 'action_repeat': None, 'show_window': True, 'seed': 0, 'env_path': None}, 'framework': 'torch', 'lambda': 0.95, 'gamma': 0.95, 'ignore_worker_failures': True, 'vf_clip_param': 200.0, 'clip_param': 0.2, 'entropy_coeff': 0.001, 'entropy_coeff_schedule': None, 'train_batch_size': 1024, 'sgd_minibatch_size': 128, 'num_sgd_iter': 16, 'num_workers': 1, 'lr': 0.0003, 'num_envs_per_worker': 1, 'batch_mode': 'truncate_episodes', 'rollout_fragment_length': 16, 'num_gpus': 0, 'model': {'fcnet_hiddens': [256, 256], 'use_lstm': False, 'lstm_cell_size': 32, 'framestack': 4}, 'no_done_at_end': False, 'soft_horizon': False}}
2023-02-06 15:31:28,321 ERROR syncer.py:111 -- Log sync requires rsync to be installed.
pid=3084) 2023-02-06 15:31:31,471 INFO ppo.py:249 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.
pid=3084) 2023-02-06 15:31:31,471 INFO trainer.py:779 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
pid=13392) No game binary has been provided, please press PLAY in the Godot editor
pid=13392) waiting for remote GODOT connection on port 11008
== Status ==
Current time: 2023-02-06 15:32:19 (running for 00:00:51.37)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name | status | loc |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING | 127.0.0.1:3084 |
+-----------------------+----------+----------------+
pid=3084) 2023-02-06 15:32:19,531 INFO trainable.py:127 -- Trainable.setup took 48.063 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
pid=3084) 2023-02-06 15:32:19,532 WARNING util.py:55 -- Install gputil for GPU system monitoring.
pid=13392) connection established
pid=13392) action space {'move': {'action_type': 'continuous', 'size': 2}}
pid=13392) observation space {'obs': {'size': [37], 'space': 'box'}}
== Status ==
Current time: 2023-02-06 15:32:20 (running for 00:00:52.39)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name | status | loc |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING | 127.0.0.1:3084 |
+-----------------------+----------+----------------+
== Status ==
Current time: 2023-02-06 15:32:25 (running for 00:00:57.44)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name | status | loc |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING | 127.0.0.1:3084 |
+-----------------------+----------+----------------+
== Status ==
Current time: 2023-02-06 15:32:30 (running for 00:01:02.50)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 2.0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 RUNNING)
+-----------------------+----------+----------------+
| Trial name | status | loc |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | RUNNING | 127.0.0.1:3084 |
+-----------------------+----------+----------------+
2023-02-06 15:32:31,191 ERROR trial_runner.py:920 -- Trial PPO_godot_60f66_00000: Error processing event.
Traceback (most recent call last):
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trial_runner.py", line 886, in _process_trial
results = self.trial_executor.fetch_result(trial)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\ray_trial_executor.py", line 675, in fetch_result
result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(RuntimeError): ray::PPOTrainer.train() (pid=3084, ip=127.0.0.1, repr=PPOTrainer)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
return method(self, *_args, **_kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
step_results = self._exec_plan_or_training_iteration_fn()
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
return method(self, *_args, **_kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
results = next(self.train_exec_impl)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
return next(self.built_iterator)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
yield ray.get(futures, timeout=timeout)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
ValueError: The two structures don't have the same nested structure.
First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434, 0.95693845, -0.6448113 , -0.9968011 , 0.70489717,
0.48645383, 0.81601405, 0.68891144, 0.8622593 , -0.4577487 ,
0.16517176, 0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
0.2706621 , -0.50107604, -0.80227774, 0.58026075, -0.35255522,
0.76724803, 0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
-0.05977842, 0.529509 , -0.28611887, 0.62858206, 0.73975134,
0.1269484 , 0.52392966, -0.88686603, -0.0761585 , 0.3493506 ,
0.47360763, 0.63514084], dtype=float32))])
More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434 0.95693845 -0.6448113 -0.9968011 0.70489717 0.48645383
0.81601405 0.68891144 0.8622593 -0.4577487 0.16517176 0.5627296
-0.1547006 -0.7543828 -0.21297579 0.2706621 -0.50107604 -0.80227774
0.58026075 -0.35255522 0.76724803 0.5039584 -0.66347104 -0.4566935
-0.82037747 -0.05977842 0.529509 -0.28611887 0.62858206 0.73975134
0.1269484 0.52392966 -0.88686603 -0.0761585 0.3493506 0.47360763
0.63514084]" is not
During handling of the above exception, another exception occurred:
ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
return method(__ray_actor, *args, **kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
return method(self, *_args, **_kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
return next(self.local_it)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
yield self.sample()
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
return method(self, *_args, **_kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
batches = [self.input_reader.next()]
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
batches = [self.get_data()]
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
item = next(self._env_runner)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
_process_observations(
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
prep_obs = preprocessor.transform(raw_obs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
self.check_shape(observation)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
observation = convert_element_to_space_type(
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
return tree.map_structure(
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
assert_same_structure(structures[0], other, check_types=check_types)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
raise type(e)("%s\n"
ValueError: The two structures don't have the same nested structure.
First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434, 0.95693845, -0.6448113 , -0.9968011 , 0.70489717,
0.48645383, 0.81601405, 0.68891144, 0.8622593 , -0.4577487 ,
0.16517176, 0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
0.2706621 , -0.50107604, -0.80227774, 0.58026075, -0.35255522,
0.76724803, 0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
-0.05977842, 0.529509 , -0.28611887, 0.62858206, 0.73975134,
0.1269484 , 0.52392966, -0.88686603, -0.0761585 , 0.3493506 ,
0.47360763, 0.63514084], dtype=float32))])
More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434 0.95693845 -0.6448113 -0.9968011 0.70489717 0.48645383
0.81601405 0.68891144 0.8622593 -0.4577487 0.16517176 0.5627296
-0.1547006 -0.7543828 -0.21297579 0.2706621 -0.50107604 -0.80227774
0.58026075 -0.35255522 0.76724803 0.5039584 -0.66347104 -0.4566935
-0.82037747 -0.05977842 0.529509 -0.28611887 0.62858206 0.73975134
0.1269484 0.52392966 -0.88686603 -0.0761585 0.3493506 0.47360763
0.63514084]" is not
Entire first structure:
{'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
Entire second structure:
OrderedDict([('obs', .)])
During handling of the above exception, another exception occurred:
ray::PPOTrainer.train() (pid=3084, ip=127.0.0.1, repr=PPOTrainer)
File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
return method(__ray_actor, *args, **kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
return method(self, *_args, **_kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\trainable.py", line 319, in train
result = self.step()
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
return method(self, *_args, **_kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 972, in step
self.try_recover_from_step_attempt()
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
return method(self, *_args, **_kwargs)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2434, in try_recover_from_step_attempt
raise RuntimeError(
RuntimeError: Not enough healthy workers remain to continue.
Result for PPO_godot_60f66_00000:
pid=3084) date: 2023-02-06_15-32-19
experiment_id: b63d176c62e9481ca65e160a090f475d
hostname: DESKTOP-J8BR5AT
node_ip: 127.0.0.1
pid: 3084
timestamp: 1675726339
trial_id: 60f66_00000
2023-02-06 15:32:31,183 ERROR trainer.py:970 -- Error in train call, attempting to recover
pid=3084) Traceback (most recent call last):
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
pid=3084) step_attempt_results = self.step_attempt()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
== Status ==
Current time: 2023-02-06 15:32:31 (running for 00:01:03.04)
Memory usage on this node: 13.9/32.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/12 CPUs, 0/0 GPUs, 0.0/11.75 GiB heap, 0.0/5.87 GiB objects
Result logdir: C:\Users\Jetpackjules\ray_results\PPO/editor
Number of trials: 1/1 (1 ERROR)
+-----------------------+----------+----------------+
| Trial name | status | loc |
|-----------------------+----------+----------------|
| PPO_godot_60f66_00000 | ERROR | 127.0.0.1:3084 |
+-----------------------+----------+----------------+
Number of errored trials: 1
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+
| Trial name | # failures | error file |
|-----------------------+--------------+----------------------------------------------------------------------------------------------------|
| PPO_godot_60f66_00000 | 1 | C:\Users\Jetpackjules\ray_results\PPO/editor\PPO_godot_60f66_00000_0_2023-02-06_15-31-28\error.txt |
+-----------------------+--------------+----------------------------------------------------------------------------------------------------+
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
pid=3084) step_results = self._exec_plan_or_training_iteration_fn()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
pid=3084) results = next(self.train_exec_impl)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
pid=3084) return next(self.built_iterator)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) [Previous line repeated 1 more time]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) [Previous line repeated 1 more time]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
pid=3084) yield ray.get(futures, timeout=timeout)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
pid=3084) return func(*args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
pid=3084) raise value.as_instanceof_cause()
pid=3084) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
pid=3084) ValueError: The two structures don't have the same nested structure.
pid=3084)
pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
pid=3084)
pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434, 0.95693845, -0.6448113 , -0.9968011 , 0.70489717,
pid=3084) 0.48645383, 0.81601405, 0.68891144, 0.8622593 , -0.4577487 ,
pid=3084) 0.16517176, 0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
pid=3084) 0.2706621 , -0.50107604, -0.80227774, 0.58026075, -0.35255522,
pid=3084) 0.76724803, 0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
pid=3084) -0.05977842, 0.529509 , -0.28611887, 0.62858206, 0.73975134,
pid=3084) 0.1269484 , 0.52392966, -0.88686603, -0.0761585 , 0.3493506 ,
pid=3084) 0.47360763, 0.63514084], dtype=float32))])
pid=3084)
pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434 0.95693845 -0.6448113 -0.9968011 0.70489717 0.48645383
pid=3084) 0.81601405 0.68891144 0.8622593 -0.4577487 0.16517176 0.5627296
pid=3084) -0.1547006 -0.7543828 -0.21297579 0.2706621 -0.50107604 -0.80227774
pid=3084) 0.58026075 -0.35255522 0.76724803 0.5039584 -0.66347104 -0.4566935
pid=3084) -0.82037747 -0.05977842 0.529509 -0.28611887 0.62858206 0.73975134
pid=3084) 0.1269484 0.52392966 -0.88686603 -0.0761585 0.3493506 0.47360763
pid=3084) 0.63514084]" is not
pid=3084)
pid=3084) During handling of the above exception, another exception occurred:
pid=3084)
pid=3084) ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
pid=3084) File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
pid=3084) return method(__ray_actor, *args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
pid=3084) return next(self.local_it)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
pid=3084) yield self.sample()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
pid=3084) batches = [self.input_reader.next()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
pid=3084) batches = [self.get_data()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
pid=3084) item = next(self._env_runner)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
pid=3084) _process_observations(
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
pid=3084) prep_obs = preprocessor.transform(raw_obs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
pid=3084) self.check_shape(observation)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
pid=3084) observation = convert_element_to_space_type(
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
pid=3084) return tree.map_structure(
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
pid=3084) assert_same_structure(structures[0], other, check_types=check_types)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
pid=3084) raise type(e)("%s\n"
pid=3084) ValueError: The two structures don't have the same nested structure.
pid=3084)
pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
pid=3084)
pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434, 0.95693845, -0.6448113 , -0.9968011 , 0.70489717,
pid=3084) 0.48645383, 0.81601405, 0.68891144, 0.8622593 , -0.4577487 ,
pid=3084) 0.16517176, 0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
pid=3084) 0.2706621 , -0.50107604, -0.80227774, 0.58026075, -0.35255522,
pid=3084) 0.76724803, 0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
pid=3084) -0.05977842, 0.529509 , -0.28611887, 0.62858206, 0.73975134,
pid=3084) 0.1269484 , 0.52392966, -0.88686603, -0.0761585 , 0.3493506 ,
pid=3084) 0.47360763, 0.63514084], dtype=float32))])
pid=3084)
pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434 0.95693845 -0.6448113 -0.9968011 0.70489717 0.48645383
pid=3084) 0.81601405 0.68891144 0.8622593 -0.4577487 0.16517176 0.5627296
pid=3084) -0.1547006 -0.7543828 -0.21297579 0.2706621 -0.50107604 -0.80227774
pid=3084) 0.58026075 -0.35255522 0.76724803 0.5039584 -0.66347104 -0.4566935
pid=3084) -0.82037747 -0.05977842 0.529509 -0.28611887 0.62858206 0.73975134
pid=3084) 0.1269484 0.52392966 -0.88686603 -0.0761585 0.3493506 0.47360763
pid=3084) 0.63514084]" is not
pid=3084) Entire first structure:
pid=3084) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
pid=3084) Entire second structure:
pid=3084) OrderedDict([('obs', .)])
pid=3084) 2023-02-06 15:32:31,187 ERROR trainer.py:2427 -- Removing unhealthy worker 1
pid=3084) Traceback (most recent call last):
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 965, in step
pid=3084) step_attempt_results = self.step_attempt()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 1044, in step_attempt
pid=3084) step_results = self._exec_plan_or_training_iteration_fn()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2032, in _exec_plan_or_training_iteration_fn
pid=3084) results = next(self.train_exec_impl)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 756, in __next__
pid=3084) return next(self.built_iterator)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 843, in apply_filter
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) [Previous line repeated 1 more time]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 876, in apply_flatten
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 783, in apply_foreach
pid=3084) for item in it:
pid=3084) [Previous line repeated 1 more time]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 471, in base_iterator
pid=3084) yield ray.get(futures, timeout=timeout)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
pid=3084) return func(*args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
pid=3084) raise value.as_instanceof_cause()
pid=3084) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
pid=3084) ValueError: The two structures don't have the same nested structure.
pid=3084)
pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
pid=3084)
pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434, 0.95693845, -0.6448113 , -0.9968011 , 0.70489717,
pid=3084) 0.48645383, 0.81601405, 0.68891144, 0.8622593 , -0.4577487 ,
pid=3084) 0.16517176, 0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
pid=3084) 0.2706621 , -0.50107604, -0.80227774, 0.58026075, -0.35255522,
pid=3084) 0.76724803, 0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
pid=3084) -0.05977842, 0.529509 , -0.28611887, 0.62858206, 0.73975134,
pid=3084) 0.1269484 , 0.52392966, -0.88686603, -0.0761585 , 0.3493506 ,
pid=3084) 0.47360763, 0.63514084], dtype=float32))])
pid=3084)
pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434 0.95693845 -0.6448113 -0.9968011 0.70489717 0.48645383
pid=3084) 0.81601405 0.68891144 0.8622593 -0.4577487 0.16517176 0.5627296
pid=3084) -0.1547006 -0.7543828 -0.21297579 0.2706621 -0.50107604 -0.80227774
pid=3084) 0.58026075 -0.35255522 0.76724803 0.5039584 -0.66347104 -0.4566935
pid=3084) -0.82037747 -0.05977842 0.529509 -0.28611887 0.62858206 0.73975134
pid=3084) 0.1269484 0.52392966 -0.88686603 -0.0761585 0.3493506 0.47360763
pid=3084) 0.63514084]" is not
pid=3084)
pid=3084) During handling of the above exception, another exception occurred:
pid=3084)
pid=3084) ray::RolloutWorker.par_iter_next() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
pid=3084) File "python\ray\_raylet.pyx", line 633, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 674, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
pid=3084) return method(__ray_actor, *args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\iter.py", line 1151, in par_iter_next
pid=3084) return next(self.local_it)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 380, in gen_rollouts
pid=3084) yield self.sample()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
pid=3084) batches = [self.input_reader.next()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
pid=3084) batches = [self.get_data()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
pid=3084) item = next(self._env_runner)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 634, in _env_runner
pid=3084) _process_observations(
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 852, in _process_observations
pid=3084) prep_obs = preprocessor.transform(raw_obs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 275, in transform
pid=3084) self.check_shape(observation)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\models\preprocessors.py", line 68, in check_shape
pid=3084) observation = convert_element_to_space_type(
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\utils\spaces\space_utils.py", line 321, in convert_element_to_space_type
pid=3084) return tree.map_structure(
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 507, in map_structure
pid=3084) assert_same_structure(structures[0], other, check_types=check_types)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\tree\__init__.py", line 363, in assert_same_structure
pid=3084) raise type(e)("%s\n"
pid=3084) ValueError: The two structures don't have the same nested structure.
pid=3084)
pid=3084) First structure: type=dict str={'obs': [0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]}
pid=3084)
pid=3084) Second structure: type=OrderedDict str=OrderedDict([('obs', array([ 0.83141434, 0.95693845, -0.6448113 , -0.9968011 , 0.70489717,
pid=3084) 0.48645383, 0.81601405, 0.68891144, 0.8622593 , -0.4577487 ,
pid=3084) 0.16517176, 0.5627296 , -0.1547006 , -0.7543828 , -0.21297579,
pid=3084) 0.2706621 , -0.50107604, -0.80227774, 0.58026075, -0.35255522,
pid=3084) 0.76724803, 0.5039584 , -0.66347104, -0.4566935 , -0.82037747,
pid=3084) -0.05977842, 0.529509 , -0.28611887, 0.62858206, 0.73975134,
pid=3084) 0.1269484 , 0.52392966, -0.88686603, -0.0761585 , 0.3493506 ,
pid=3084) 0.47360763, 0.63514084], dtype=float32))])
pid=3084)
pid=3084) More specifically: Substructure "type=list str=[0.088083224826389, 1.24646044921875, 0.101170279085636, -0.99486911296844, 0.151397776285807, 0.134054991576049, 0.102607933250633, 0.0464364129143792, 0.124826517191019, 0.161533527546101, 0.16446912610853, 0.134228749318166, 0, 0, 0.485937032613668, 0.551738292247325, 0.588589161365956, 0.405113014014992, 0, 0, 0, 0.161773131774352, 0.760299149934236, 0.856947383365116, 0.895181690250431, 0.914702149124833, 0.925685616226884, 0.931842894167513, 0.934741930918651, 0.935008766414883, 0.932697364875862, 0.927319990622031, 0.917565268439216, 0.900363346477887, 0.867662962492522, 0.791059459651913, 0.454066336691917]" is a sequence, while substructure "type=ndarray str=[ 0.83141434 0.95693845 -0.6448113 -0.9968011 0.70489717 0.48645383
pid=3084) 0.81601405 0.68891144 0.8622593 -0.4577487 0.16517176 0.5627296
pid=3084) -0.1547006 -0.7543828 -0.21297579 0.2706621 -0.50107604 -0.80227774
pid=3084) 0.58026075 -0.35255522 0.76724803 0.5039584 -0.66347104 -0.4566935
pid=3084) -0.82037747 -0.05977842 0.529509 -0.28611887 0.62858206 0.73975134
pid=3084) 0.1269484 0.52392966 -0.88686603 -0.0761585 0.3493506 0.47360763
pid=3084) 0.63514084]" is not
pid=3084) Entire first structure:
pid=3084) {'obs': [., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., ., .]}
pid=3084) Entire second structure:
pid=3084) OrderedDict([('obs', .)])
pid=3084)
pid=3084) During handling of the above exception, another exception occurred:
pid=3084)
pid=3084) Traceback (most recent call last):
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\agents\trainer.py", line 2423, in try_recover_from_step_attempt
pid=3084) ray.get(obj_ref)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
pid=3084) return func(*args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1763, in get
pid=3084) raise value.as_instanceof_cause()
pid=3084) ray.exceptions.RayTaskError(StopIteration): ray::RolloutWorker.sample_with_count() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
pid=3084) File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
pid=3084) return method(__ray_actor, *args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
pid=3084) batch = self.sample()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
pid=3084) batches = [self.input_reader.next()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
pid=3084) batches = [self.get_data()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
pid=3084) item = next(self._env_runner)
pid=3084) StopIteration
pid=3084) 2023-02-06 15:32:31,190 ERROR worker.py:83 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::RolloutWorker.sample_with_count() (pid=13392, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000001968F6EF6D0>)
pid=3084) File "python\ray\_raylet.pyx", line 640, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 644, in ray._raylet.execute_task
pid=3084) File "python\ray\_raylet.pyx", line 593, in ray._raylet.execute_task.function_executor
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\function_manager.py", line 648, in actor_method_executor
pid=3084) return method(__ray_actor, *args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 817, in sample_with_count
pid=3084) batch = self.sample()
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\util\tracing\tracing_helper.py", line 451, in _resume_span
pid=3084) return method(self, *_args, **_kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 759, in sample
pid=3084) batches = [self.input_reader.next()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 104, in next
pid=3084) batches = [self.get_data()]
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\sampler.py", line 266, in get_data
pid=3084) item = next(self._env_runner)
pid=3084) StopIteration
pid=3084) 2023-02-06 15:32:31,647 ERROR worker_set.py:219 -- Failed to stop workers!
pid=3084) Traceback (most recent call last):
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\rllib\evaluation\worker_set.py", line 217, in stop
pid=3084) ray.get(tids)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\_private\client_mode_hook.py", line 105, in wrapper
pid=3084) return func(*args, **kwargs)
pid=3084) File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\worker.py", line 1765, in get
pid=3084) raise value
pid=3084) ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.
pid=3084) class_name: RolloutWorker
pid=3084) actor_id: def0d4155abb49949005ca3e01000000
pid=3084) pid: 13392
pid=3084) namespace: c2206530-7b97-4281-879d-362dbf58ca34
pid=3084) ip: 127.0.0.1
pid=3084) The actor is dead because its worker process has died. Worker exit type: INTENDED_EXIT
Traceback (most recent call last):
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\Scripts\gdrl.exe\__main__.py", line 7, in <module>
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\core\main.py", line 80, in main
training_function(args)
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\godot_rl_agents\wrappers\ray_wrapper.py", line 103, in rllib_training
results = tune.run(
File "C:\Users\Jetpackjules\anaconda3\envs\gdrl_conda\lib\site-packages\ray\tune\tune.py", line 633, in run
raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_godot_60f66_00000])
pid=13392) exit was not clean, using atexit to close env
pid=13392) close message sent
(gdrl_conda) PS C:\Users\Jetpackjules\Documents\Godot Gamez\ANTv0.61>
Any ideas as to what could be causing this array imbalance? My theory is that it has something to do with
func _send_dict_as_json_message(dict):
stream.put_string(JSON.stringify(dict))
in the sync.gd node, but I donāt understand what excatly it would doā¦ (Because the error messages is comparing a list to a dict as incompatibleā¦
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 20 (1 by maintainers)
Thanks! I will close this for now, as all my problems have been adressed. I appreciate the help.