NeRF-SLAM: AttributeError: 'gtsam.gtsam.GaussianFactorGraph' object has no attribute 'optimizeDensely'

I have solved the problem this way: Just clone the default branch of https://github.com/ToniRV/gtsam-1 and add the following code at line 408 and line 239 of gtsam/gtsam/linear/linear.i.

gtsam::VectorValues optimizeDensely() const; Vector vector(const gtsam::KeyVector& keys) const;

and make gtsam as in README.

It seems that the header of two functions for wrapping is missing.

I installed the gtsam 4.2a5 specified in the requirements.txt. However, There is a running error related to the gtsam function:

  File "./examples/slam_demo.py", line 200, in <module>
    run(args)
  File "./examples/slam_demo.py", line 179, in run
    and (not slam or slam_module.spin()) \
  File "/home/bxu/code/NeRF-SLAM/./examples/../pipeline/pipeline_module.py", line 101, in spin
    output = self.spin_once(input);
  File "/home/bxu/code/NeRF-SLAM/./examples/../slam/slam_module.py", line 11, in spin_once
    output = self.slam(input)
  File "/home/bxu/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/bxu/code/NeRF-SLAM/./examples/../slam/meta_slam.py", line 30, in forward
    output = self._frontend(batch["data"], self.state, self.delta)
  File "/home/bxu/code/NeRF-SLAM/./examples/../slam/vio_slam.py", line 114, in _frontend
    x0_visual, visual_factors, viz_out = self.visual_frontend(batch)  # TODO: currently also calls BA, and global BA
  File "/home/bxu/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/bxu/code/NeRF-SLAM/./examples/../slam/visual_frontends/visual_frontend.py", line 324, in forward
    self.__initialize()
  File "/home/bxu/code/NeRF-SLAM/./examples/../slam/visual_frontends/visual_frontend.py", line 648, in __initialize
    x0, rcm_factor = self.update(kf0=None, kf1=None, use_inactive=True)
  File "/home/bxu/.local/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 12, in decorate_autocast
    return func(*args, **kwargs)
  File "/home/bxu/code/NeRF-SLAM/./examples/../slam/visual_frontends/visual_frontend.py", line 437, in update
    x0, rcm_factor = self.ba(gru_estimated_flow, gru_estimated_flow_weight, damping,
  File "/home/bxu/code/NeRF-SLAM/./examples/../slam/visual_frontends/visual_frontend.py", line 1142, in ba
    gtsam_delta = linear_factor_graph.optimizeDensely() # Calls Eigen Cholesky, without a particularly smart ordering (vs Eigen::SimplicialLLt...)
AttributeError: 'gtsam.gtsam.GaussianFactorGraph' object has no attribute 'optimizeDensely'

It was tested using the following command: python ./examples/slam_demo.py --dataset_dir=./Datasets/Replica/office0 --dataset_name=nerf --buffer=100 --slam --img_stride=2 --fusion='nerf' --gui

When I checked the public functions in the gtsam.gtsam.GaussianFactorGraph class, it shows the following ones. It seems there is no python binder for optimizeDensely function.

gtsam.gtsam.GaussianFactorGraph.add(                           gtsam.gtsam.GaussianFactorGraph.error(                         gtsam.gtsam.GaussianFactorGraph.optimize(
gtsam.gtsam.GaussianFactorGraph.at(                            gtsam.gtsam.GaussianFactorGraph.exists(                        gtsam.gtsam.GaussianFactorGraph.optimizeGradientSearch(
gtsam.gtsam.GaussianFactorGraph.augmentedHessian(              gtsam.gtsam.GaussianFactorGraph.gradient(                      gtsam.gtsam.GaussianFactorGraph.print(
gtsam.gtsam.GaussianFactorGraph.augmentedJacobian(             gtsam.gtsam.GaussianFactorGraph.gradientAtZero(                gtsam.gtsam.GaussianFactorGraph.printErrors(
gtsam.gtsam.GaussianFactorGraph.clone(                         gtsam.gtsam.GaussianFactorGraph.hessian(                       gtsam.gtsam.GaussianFactorGraph.probPrime(
gtsam.gtsam.GaussianFactorGraph.deserialize(                   gtsam.gtsam.GaussianFactorGraph.jacobian(                      gtsam.gtsam.GaussianFactorGraph.push_back(
gtsam.gtsam.GaussianFactorGraph.dot(                           gtsam.gtsam.GaussianFactorGraph.keyVector(                     gtsam.gtsam.GaussianFactorGraph.saveGraph(
gtsam.gtsam.GaussianFactorGraph.eliminateMultifrontal(         gtsam.gtsam.GaussianFactorGraph.keys(                          gtsam.gtsam.GaussianFactorGraph.serialize(
gtsam.gtsam.GaussianFactorGraph.eliminatePartialMultifrontal(  gtsam.gtsam.GaussianFactorGraph.marginal(                      gtsam.gtsam.GaussianFactorGraph.size(
gtsam.gtsam.GaussianFactorGraph.eliminatePartialSequential(    gtsam.gtsam.GaussianFactorGraph.marginalMultifrontalBayesNet(  gtsam.gtsam.GaussianFactorGraph.sparseJacobian_(
gtsam.gtsam.GaussianFactorGraph.eliminateSequential(           gtsam.gtsam.GaussianFactorGraph.mro(                           
gtsam.gtsam.GaussianFactorGraph.equals(                        gtsam.gtsam.GaussianFactorGraph.negate(

i have solved this problem by installing newest gtsam with the command " pip3 install gtsam==4.2a9 " , and you don’t need to compile the gtsam source code.

@szgy66 I’ve installed roughly 5 times over the last 3 weeks. Currently close, but the demo fails with a segfault… Ideas?

First, a note - before “python setup.py install” I needed to do “export PATH=/usr/local/cuda/bin:$PATH” - one of the pytorch things override your native nvcc. Also, I’m installing to CUDA 11.6, python 3.10.9, , pytorch 1.12.1. I have done “export CUDA_VISIBLE_DEVICES=0” (after trying without.)

For some reason, when I launch I get a segfault. Note that the os.environ[‘CUDA_VISIBLE_DEVICES’]: ‘1’ seems to have overridden my setting to ‘0’. Not sure if that is useful.

(nerf) pg@pg-MS-7A59:~/repos/NeRF-SLAM$ python ./examples/slam_demo.py --dataset_dir=./Datasets/Replica/office0 --dataset_name=nerf --buffer=100 --slam --parallel_run --img_stride=2 --fusion='nerf' --multi_gpu --gui
Running with GPUs: 0,1
Running pipeline in parallel mode.
I0118 09:44:08.855345 29211 pipeline_module.py:85] Module: VioSLAM - Spinning.
Aborted (core dumped)
(nerf) pg@pg-MS-7A59:~/repos/NeRF-SLAM$ I0118 09:44:09.368386 29213 pipeline_module.py:85] Module: nerf - Spinning.
I0118 09:44:09.384157 29214 pipeline_module.py:85] Module: nerf - Spinning.
ic| os.environ['CUDA_VISIBLE_DEVICES']: '1'
Process Process-2:
Traceback (most recent call last):
  File "/home/pg/miniconda3/envs/nerf/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/pg/miniconda3/envs/nerf/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/pg/repos/NeRF-SLAM/./examples/../pipeline/pipeline_module.py", line 88, in spin
    self.initialize_module()
  File "/home/pg/repos/NeRF-SLAM/./examples/../fusion/fusion_module.py", line 25, in initialize_module
    self.fusion = NerfFusion(self.name, self.args, self.device)
  File "/home/pg/repos/NeRF-SLAM/./examples/../fusion/nerf_fusion.py", line 65, in __init__
    self.ngp = ngp.Testbed(mode, 0) # NGP can only use device = 0
RuntimeError: Could not allocate memory: /home/pg/repos/NeRF-SLAM/thirdparty/instant-ngp/dependencies/tiny-cuda-nn/include/tiny-cuda-nn/gpu_memory.h:123 cudaMalloc(&rawptr, n_bytes+DEBUG_GUARD_SIZE*2) failed with error no CUDA-capable device is detected
Loading 1000 images.

Running the demo without --multi_gpu I quickly get a segfault, but at least I get the ngp window. Here are the first 10 lines or so…

(nerf) pg@pg-MS-7A59:~/repos/NeRF-SLAM$ python ./examples/slam_demo.py --dataset_dir=./Datasets/Replica/office0 --dataset_name=nerf --buffer=100 --slam --parallel_run --img_stride=2 --fusion='nerf' --gui
Running with GPUs: 0
Running pipeline in parallel mode.
I0118 09:33:40.423174 27145 pipeline_module.py:85] Module: VioSLAM - Spinning.
Aborted (core dumped)
(nerf) pg@pg-MS-7A59:~/repos/NeRF-SLAM$ I0118 09:33:40.969152 27147 pipeline_module.py:85] Module: nerf - Spinning.
I0118 09:33:40.970565 27148 pipeline_module.py:85] Module: nerf - Spinning.
ic| os.environ['CUDA_VISIBLE_DEVICES']: '0'
Loading 1000 images.
09:33:42 SUCCESS  NGP Device set to: 0
  0%|                                                  | 0/1000 [00:00<?, ?it/s]09:33:42 WARNING  Vulkan error: loaderValidateLayers: Layer 0 does not exist in the list of available layers
09:33:42 WARNING  Could not initialize Vulkan and NGX. DLSS not supported. (/home/pg/repos/NeRF-SLAM/thirdparty/instant-ngp/src/dlss.cu:220 vkCreateInstance(&instance_create_info, nullptr, &vk_instance) failed)
  1%|▎                                         | 6/1000 [00:00<00:20, 47.91it/s]09:33:42 INFO     Loading network config from: /home/pg/repos/NeRF-SLAM/thirdparty/instant-ngp/configs/nerf/base.json

(and keeps running, but no viewer shows.)

btw, Removing the ‘–multi-gpu’ flag I still get the segfault - but at least the viewer starts…