Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding OpenCV's GPU GFTT/ OpticalFlow and CudaSift support #1330

Merged
merged 15 commits into from
Sep 13, 2024

Conversation

matlabbe
Copy link
Member

@matlabbe matlabbe commented Aug 24, 2024

These new features are available if built with OpenCV CUDA.

New features:

  • Faster GFTT: with GFTT/Gpu=true
  • Faster optical flow based F2F Odometry: with Vis/CorFlowGpu=true
  • Faster stereo correspondences: with Stereo/Gpu=true
  • Faster SIFT features extraction (with new CudaSift dependency): with SIFT/Gpu=true
  • Faster F2F Odometry approach using parameters above like this: --Odom/Strategy 1 --Vis/CorFlowGpu true --Vis/CoreType 1 --Stereo/Gpu true --GFTT/Gpu true --Vis/BundleAdjustment 0

Use this fork to build CudaSift: https://github.com/matlabbe/CudaSift

@matlabbe matlabbe self-assigned this Aug 24, 2024
@matlabbe matlabbe marked this pull request as ready for review August 27, 2024 21:50
@matlabbe matlabbe changed the title Adding GFTT and SIFT Cuda support Adding GFTT and CudaSift support Aug 29, 2024
@matlabbe matlabbe changed the title Adding GFTT and CudaSift support Adding GFTT cuda and CudaSift support Aug 29, 2024
@matlabbe matlabbe changed the title Adding GFTT cuda and CudaSift support Adding OpenCV's GFTT-GPU and CudaSift support Aug 29, 2024
@matlabbe
Copy link
Member Author

matlabbe commented Aug 29, 2024

When building rtabmap_ros against rtabmap with this PR (on Ubuntu 20.04 ROS Noetic with local version of OpenCV 4.2+Cuda shadowing OpenCV system binaries), we can get this error:

/usr/bin/ld: /usr/local/lib/libopencv_cudafeatures2d.so.4.2.0: undefined reference to `cv::cuda::device::ThrustAllocator::getAllocator()'
/usr/bin/ld: /usr/local/lib/libopencv_cudafeatures2d.so.4.2.0: undefined reference to `cv::cuda::StreamAccessor::getStream(cv::cuda::Stream const&)'

It seems to be because during linking (see with make VERBOSE=1), the binary /usr/lib/x86_64-linux-gnu/libopencv_core.so.4.2.0 is in the list (because of ros dependency cv_bridge) before /usr/local/lib/libopencv_core.so.4.2.0, so cuda core functions are not found. Well, a workaround if we don't want to rebuild cv_bridge is to make a symlink from /usr/lib/x86_64-linux-gnu/libopencv_core.so.4.2.0 to /usr/local/lib/libopencv_core.so.4.2.0 (backup original binary file to revert afterwards) like this:

/usr/lib/x86_64-linux-gnu$ ls -l libopencv_core*
-rw-r--r-- 1 root root 9124066 Feb 17  2020 libopencv_core.a
lrwxrwxrwx 1 root root      21 Feb 17  2020 libopencv_core.so -> libopencv_core.so.4.2
lrwxrwxrwx 1 root root      38 Aug 29 16:50 libopencv_core.so.4.2 -> /usr/local/lib/libopencv_core.so.4.2.0
lrwxrwxrwx 1 root root      38 Aug 29 16:51 libopencv_core.so.4.2.0 -> /usr/local/lib/libopencv_core.so.4.2.0
-rw-r--r-- 1 root root 3840856 Feb 17  2020 libopencv_core.so.4.2.0.back

Note that this would only work if you rebuild exactly the same OpenCV version than the system one to avoid any conflicts.

@matlabbe
Copy link
Member Author

matlabbe commented Aug 31, 2024

Loop closure performance comparison with CudaSift (pretty similar to OpenCV's SURF and SIFT):
image

@matlabbe matlabbe changed the title Adding OpenCV's GFTT-GPU and CudaSift support Adding OpenCV's GPU GFTT/ OpticalFlow and CudaSift support Sep 1, 2024
@matlabbe matlabbe merged commit 69ac21f into master Sep 13, 2024
7 checks passed
@matlabbe matlabbe deleted the gftt_sift_cuda branch September 13, 2024 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant