Reload registration from file when image have different resolution

[jensjoachim]

Hi

I have followed the discussion on how to do video stitching with great success. Thanks!

Now I'm trying to store registration data in "cameras" tuple. This can be done but, gives me some other issues which I need a little guidance for.

Naive as I am I initially thought I could use Pickle to store the entire "stitcher" object, but it gave me an error.
Then I ended up only storing data in "self.cameras". This only works for the follow case:

stitcher = Stitcher(**settings_stitcher)
panorama = stitcher.stitch(imgs)  # Only if stitcking is a success
stitcher.load_registration()
while true: 
  panorama = stitcher.stitch(imgs)  

But NOT for:

stitcher = Stitcher(**settings_stitcher)
stitcher.load_registration()
while true: 
  panorama = stitcher.stitch(imgs)  # Fails

The problem is when I restart my program and let's say images are too dark to make the first successful stitch, then camera registration reload won't make stitcher work. Of course I could get around this by also storing the "good" images via my "store_registration()" function.

Further, I have found if I do a "load_registration()", when images/cameras are another resolution (same w/h ration). I can reuse the data from "self.cameras". This is extremely good!

Here comes the question:
Is there some way where I can make stitching work for any camera resolution with same width/height ration? Even if a set of images which can generate a successful stitch is NOT present?

Best Regards,
Joachim

[lukasalexanderweber]

Can you share your implementations of load_registration and store_registration?

During our research we also pickled the cameras and it worked.

[lukasalexanderweber]

Yeah I remember that pickling opencv objects is not that easy. I think you also need to store and reload maybe some image data, but I'm not sure. Can you share the error message?

[jensjoachim]

Yes. Exactly, Pickle don't like the opencv objects.

On running following code:

stitcher = Stitcher(**settings_stitcher)
stitcher.load_registration()
panorama = stitcher.stitch(imgs)

I get this error message:

Camera registration loaded
Traceback (most recent call last):
  File "/home/pi/projects/animal_detection_2/test/image_stitch/test_stitcher/test_stitcher.py", line 314, in <module>
    panorama = stitcher.stitch(imgs,settings_input["finder_lock"])
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/projects/animal_detection_2/test/image_stitch/test_stitcher/test_stitcher.py", line 98, in stitch
    imgs, masks, corners, sizes = self.warp_low_resolution(imgs, self.cameras)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.local/lib/python3.11/site-packages/stitching/stitcher.py", line 173, in warp_low_resolution
    imgs, masks, corners, sizes = self.warp(imgs, cameras, sizes, camera_aspect)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.local/lib/python3.11/site-packages/stitching/stitcher.py", line 186, in warp
    corners, sizes = self.warper.warp_rois(sizes, cameras, aspect)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.local/lib/python3.11/site-packages/stitching/warper.py", line 74, in warp_rois
    roi = self.warp_roi(size, camera, aspect)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.local/lib/python3.11/site-packages/stitching/warper.py", line 80, in warp_roi
    warper = cv.PyRotationWarper(self.warper_type, self.scale * aspect)
                                                   ~~~~~~~~~~~^~~~~~~~
TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'

[lukasalexanderweber]

Yeah you can see that the warper scales are not set up correctly. You need to dig deeper in warper.py. Feel free to improve the code by adding assertions like it's done in

stitching/stitching/images.py

Line 96 in cf74fdf

assert self._scales_set

[lukasalexanderweber]

Probably you must call

stitching/stitching/stitcher.py

Line 162 in cf74fdf

def estimate_scale(self, cameras):

with your restored cameras

[jensjoachim]

This worked! Thank you so much!
I'm not an SW or python expert, so this issue kinda broke my neck here.

[jensjoachim]

For whoever like to store/reload registration data (Homographic), these function can be used:

Store registration data:

    def store_registration(self,print_en=False):
        if not self.cameras_registered:
            print("Camera registration not stored!")
        else:    
            # Gather data
            attr_data_dict_index = ["aspect","focal","ppx","ppy","R","t"]
            attr_data_dict_list = []
            for i in range(len(stitcher.cameras)):
                attr_data_dict = {}
                for attr in attr_data_dict_index:
                    attr_data_dict[attr] = getattr(stitcher.cameras[i], attr)
                attr_data_dict_list.append(attr_data_dict)
            # Print camera data
            if print_en:
                print("Camera data to be stored:")
                print(attr_data_dict_list)
            # Store camera data
            with open('registration', 'wb') as file:
                pickle.dump(attr_data_dict_list, file)
            # Done
            print("Camera registration stored!")

Reload registration data:

    def load_registration(self,print_en=True):
        # Load camera data
        with open('registration', 'rb') as file:
            attr_data_dict_list_in = pickle.load(file)
        if print_en:
            print("Camera data loaded:")
            print(attr_data_dict_list_in)
        # Make tuple of CameraParams
        new_cam_param_tuple = ()
        for i in range(len(attr_data_dict_list_in)):
            new_cam_param = cv2.detail.CameraParams()
            for keys, value in attr_data_dict_list_in[i].items():
                setattr(new_cam_param,keys,value)
            new_cam_param_tuple = new_cam_param_tuple + (new_cam_param,)
        # Overwrite tupe in stitcher
        self.cameras = new_cam_param_tuple
        # Extra
        self.estimate_scale(self.cameras)
        # Set registration flag
        self.cameras_registered = True
        # Done
        print("Camera registration loaded")