crash during run_custm.py

[bblumberg]

I am using the docker version of bundlesdf running on an x86 machine with a 4070 ti.

• To get things running I had to uninstall and reinstall scipy, and pip install yacs
• Once those little things were fixed, I ran run_custom.py on the milk example. It runs for 15 minutes or so, generating output which appears normal before spitting out the error below. Any ideas?

Thanks,

bruce

-[bundlesdf.py] frame 1668813071116728056 pose update after
[[-0.709 -0.681 -0.182 0.105]
[-0.695 0.632 0.342 -0.081]
[-0.118 0.369 -0.922 0.437]
[ 0. 0. 0. 1. ]]
[2025-01-14 06:53:16.050] [warning] [Bundler.cpp:435] total keyframes=186, want to select 10
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:516] ids#=186, max_BA_frames-frames.size()=9
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:525] frames#=10
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813056725965400 visible=0.813003
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813056725965400) into pairs
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813056725965400 visible=0.789516
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813056725965400) into pairs
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813057005518533 visible=0.811209
[2025-01-14 06:53:16.090] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813057005518533) into pairs
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813057005518533 visible=0.789139
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813057005518533) into pairs
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813057204255187 visible=0.778267
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813057204255187) into pairs
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813057204255187 visible=0.753044
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813057204255187) into pairs
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813070042892544 visible=0.856001
[2025-01-14 06:53:16.091] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813070042892544) into pairs
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813070042892544 visible=0.834662
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813070042892544) into pairs
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813070174291989 visible=0.892966
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813070174291989) into pairs
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813070174291989 visible=0.866124
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813070174291989) into pairs
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813070341765204 visible=0.907371
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813070341765204) into pairs
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813070341765204 visible=0.883041
[2025-01-14 06:53:16.092] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813070341765204) into pairs
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813070574800195 visible=0.913079
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813070574800195) into pairs
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813070574800195 visible=0.894839
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813070574800195) into pairs
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:793] frame 1668813071086837259 and 1668813070957531326 visible=0.937269
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:802] add frame (1668813071086837259, 1668813070957531326) into pairs
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:793] frame 1668813071116728056 and 1668813070957531326 visible=0.919298
[2025-01-14 06:53:16.093] [warning] [Bundler.cpp:802] add frame (1668813071116728056, 1668813070957531326) into pairs
[bundlesdf.py] frame_pairs: 16
[2025-01-14 06:53:16.093] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813056725965400)
[2025-01-14 06:53:16.094] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813057005518533)
[2025-01-14 06:53:16.095] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813057204255187)
[2025-01-14 06:53:16.096] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813070042892544)
[2025-01-14 06:53:16.097] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813070174291989)
[2025-01-14 06:53:16.098] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813070341765204)
[2025-01-14 06:53:16.099] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813070574800195)
[2025-01-14 06:53:16.100] [warning] [FeatureManager.cpp:2690] _raw_matches found exsting pair (1668813071086837259, 1668813070957531326)
[loftr_wrapper.py] image0: torch.Size([8, 1, 400, 400])
Traceback (most recent call last):
File "run_custom.py", line 222, in
run_one_video(video_dir=args.video_dir, out_folder=args.out_folder, use_segmenter=args.use_segmenter, use_gui=args.use_gui)
File "run_custom.py", line 102, in run_one_video
tracker.run(color, depth, K, id_str, mask=mask, occ_mask=None, pose_in_model=pose_in_model)
File "/home/easybot/workspaces/BundleSDF/bundlesdf.py", line 543, in run
self.process_new_frame(frame)
File "/home/easybot/workspaces/BundleSDF/bundlesdf.py", line 494, in process_new_frame
self.find_corres(pairs)
File "/home/easybot/workspaces/BundleSDF/bundlesdf.py", line 362, in find_corres
corres = self.loftr.predict(rgbAs=imgs[::2], rgbBs=imgs[1::2])
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/easybot/workspaces/BundleSDF/loftr_wrapper.py", line 50, in predict
self.matcher(tmp)
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/easybot/workspaces/BundleSDF/BundleTrack/LoFTR/src/loftr/loftr.py", line 49, in forward
feats_c, feats_f = self.backbone(torch.cat([data['image0'], data['image1']], dim=0))
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/easybot/workspaces/BundleSDF/BundleTrack/LoFTR/src/loftr/backbone/resnet_fpn.py", line 116, in forward
x1_out = self.layer1_outconv2(x1_out+x2_out_2x)
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/nn/modules/container.py", line 141, in forward
input = module(input)
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 447, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/opt/conda/envs/py38/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 443, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

[pithreeone]

I'm also facing this issue. Has anyone found a solution?

[umurotti]

Same here