RLE of SAM can't handle masks with no change

[MSt-10]

System Info

• transformers version: 4.49.0.dev0
• Platform: Windows-10-10.0.26100-SP0
• Python version: 3.11.11
• Huggingface_hub version: 0.27.1
• Safetensors version: 0.5.2
• Accelerate version: not installed
• Accelerate config: not found
• PyTorch version (GPU?): 2.5.1 (False)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?: No

Who can help?

@ArthurZucker

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

I'm fine-tuning the SamModel and using the fine-tuned model in a mask-generation pipeline afterward.
After some time in the training, I suddenly get the following error when using the fine-tuned model in the pipeline:

Traceback (most recent call last):
  File "***.py", line 17, in <module>
    outputs = generator(image)
              ^^^^^^^^^^^^^^^^
  File "transformers\pipelines\mask_generation.py", line 166, in __call__
    return super().__call__(image, *args, num_workers=num_workers, batch_size=batch_size, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\base.py", line 1354, in __call__
    return next(
           ^^^^^
  File "transformers\pipelines\pt_utils.py", line 124, in __next__
    item = next(self.iterator)
           ^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\pt_utils.py", line 269, in __next__
    processed = self.infer(next(self.iterator), **self.params)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\base.py", line 1269, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\mask_generation.py", line 237, in _forward
    masks, iou_scores, boxes = self.image_processor.filter_masks(
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\models\sam\image_processing_sam.py", line 847, in filter_masks
    return self._filter_masks_pt(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\models\sam\image_processing_sam.py", line 945, in _filter_masks_pt
    masks = _mask_to_rle_pytorch(masks)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\models\sam\image_processing_sam.py", line 1386, in _mask_to_rle_pytorch
    counts += [cur_idxs[0].item()] + btw_idxs.tolist() + [height * width - cur_idxs[-1]]
               ~~~~~~~~^^^
IndexError: index 0 is out of bounds for dimension 0 with size 0

Note: this error doesn't occur on every image, but just on some.

Code used to produce error:

    image = Image.open("PATH_TO_MY_IMAGE")

    model = SamModel.from_pretrained("PATH_TO_MY_CHECKPOINT")
    processor = SamImageProcessor.from_pretrained("facebook/sam-vit-huge")
    generator = pipeline(
        "mask-generation",
        model=model,
        device="cpu",
        points_per_batch=64,
        image_processor=processor
    )  # MaskGenerationPipeline

    outputs = generator(image)

Expected behavior

No error should be thrown and the RLE should be computed correctly.