habitat-lab: Math::Quaternion::fromMatrix(): the matrix is not orthogonal

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: master Habitat-Sim: master

❓ Questions and Help

I am running the rearrange task with a custom interface in order to use SAC from another repo. I am using the same yaml configuration, just with smaller RGB inputs (64x64). Everything works fine for a while and then I get the following error

W0729 05:21:54.534488 3717050 PhysicsManager.cpp:251] ::addObject : newObjectHandle : 003_cracker_box_:0003
Math::Quaternion::fromMatrix(): the matrix is not orthogonal:
Matrix(nan, nan, nan,
       nan, nan, nan,
       nan, nan, nan)

This happens always after 3188 steps, and always with newObjectHandle : 003_cracker_box_:0003. I am not setting any seed in my script (the policy is randomly initialized, and the actions are different across runs), but I am not sure how Habitat seeding is set. Also, the robot can pick 003_cracker_box_:0003 many times in past iterations, and I am sure I passing correct actions (7-dim actions in [0,1] and last action in [-1,1]).

I am not sure if there is some code error, of if the robot is trying to do some impossible things and thus it crashes. If this is the case, is there a way to prevent the whole code from crashing? Like simply ending the episode, resetting the robot and giving a negative reward to the agent.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 20 (6 by maintainers)

Most upvoted comments

Hello, I am also experiencing the same issue. Below I included a more detailed trace of the error by running with PYTHONFAULTHANDLER=1. I think this is actually a bug in Habitat Sim, not Habitat Lab. This error happens randomly for me, it will happen on some machines but not others and some methods but not others. For example, I am able to train a SAC policy for 1M steps on the Reacher environment (will be release in this PR https://github.com/facebookresearch/habitat-lab/pull/685) but when I try custom imitation learning algorithms, it fails.

I actually also occasionally encountered this error in the working / non-release version with the motion planning approaches specifically. I had not seen this issue in a while so I assumed it was gone, but now I am seeing it often.

Math::Matrix4::rotation(): the normalized rotation part is not orthogonal:
Matrix(nan, -nan, -nan,
       -nan, nan, nan,
       -nan, nan, -nan)
Fatal Python error: Aborted

Thread 0x00007f25a2e1f700 (most recent call first):
  File "/private/home/andrewszot/miniconda3/envs/l2l/lib/python3.7/threading.py", line 300 in wait
  File "/private/home/andrewszot/miniconda3/envs/l2l/lib/python3.7/threading.py", line 552 in wait
  File "/private/home/andrewszot/miniconda3/envs/l2l/lib/python3.7/site-packages/tqdm/_monitor.py", line 60 in run
  File "/private/home/andrewszot/miniconda3/envs/l2l/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/private/home/andrewszot/miniconda3/envs/l2l/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f25a261e700 (most recent call first):

Thread 0x00007f25a1e1d700 (most recent call first):

Current thread 0x00007f270cdfa740 (most recent call first):
  File "/private/home/andrewszot/hablab_fixes/habitat/tasks/rearrange/rearrange_sim.py", line 442 in internal_step
  File "/private/home/andrewszot/hablab_fixes/habitat/tasks/rearrange/rearrange_sim.py", line 373 in step
  File "/private/home/andrewszot/hablab_fixes/habitat/tasks/rearrange/actions.py", line 82 in step
  File "/private/home/andrewszot/hablab_fixes/habitat/core/embodied_task.py", line 303 in step
  File "/private/home/andrewszot/hablab_fixes/habitat/tasks/rearrange/rearrange_task.py", line 65 in step
  File "/private/home/andrewszot/hablab_fixes/habitat/tasks/rearrange/rearrange_reach_task.py", line 16 in step
  File "/private/home/andrewszot/hablab_fixes/habitat/core/env.py", line 267 in step
  File "/private/home/andrewszot/hablab_fixes/habitat/core/env.py", line 410 in step
  File "/private/home/andrewszot/miniconda3/envs/l2l/lib/python3.7/contextlib.py", line 74 in inner
  File "/private/home/andrewszot/hablab_fixes/habitat_baselines/common/environments.py", line 52 in step
  File "/private/home/andrewszot/hablab_fixes/habitat_baselines/utils/gym_adapter.py", line 125 in direct_hab_step
  File "/private/home/andrewszot/hablab_fixes/habitat_baselines/utils/gym_adapter.py", line 122 in step
  File "/private/home/andrewszot/miniconda3/envs/l2l/lib/python3.7/site-packages/gym/core.py", line 234 in step
  File "/private/home/andrewszot/hablab_fixes/habitat_baselines/utils/render_wrapper.py", line 63 in step
  File "./envs.py", line 97 in step
  File "/private/home/andrewszot/p-mbirlo/rl-toolkit/rlf/baselines/monitor.py", line 56 in step
  File "/private/home/andrewszot/p-mbirlo/rl-toolkit/rlf/baselines/vec_env/dummy_vec_env.py", line 52 in step_wait
  File "/private/home/andrewszot/p-mbirlo/rl-toolkit/rlf/rl/envs.py", line 328 in step_wait
  File "/private/home/andrewszot/p-mbirlo/rl-toolkit/rlf/baselines/vec_env/vec_env.py", line 108 in step
  File "/private/home/andrewszot/p-mbirlo/rl-toolkit/rlf/rl/runner.py", line 59 in training_iter
  File "/private/home/andrewszot/p-mbirlo/rl-toolkit/rlf/main.py", line 70 in run_policy
  File "main.py", line 130 in <module>
[1]    620528 abort (core dumped)

ASzot on Jul 30, 2021

This PR fixes the issue in my repository (https://github.com/ethanabrooks/habitat-sim-issue).

ethanabrooks on Apr 7, 2022

Thanks for your help with this! I will let you know if the issue is fixed on my end once the PR goes through.

ethanabrooks on Apr 6, 2022

I had some issues with the main branch but I found that these two commits work: Habitat-lab: fbdd9fd42e7716c04bbde1da0d867e6d79dd3490 Habitat-sim: 066e4343c27a03ccd969ac6a83cbd262d5c7f2f9

ethanabrooks on Apr 4, 2022

I am able to eliminate this error by eliminating the “LOOK_DOWN” and “LOOK_UP” actions.

ethanabrooks on Mar 29, 2022