CocoDetection() doesn't work using some train and validation images with some annotations

[hyperkai]

🚀 The feature

CocoDetection() doesn't work using stuff_train2017_pixelmaps with stuff_train2017.json and using stuff_val2017_pixelmaps with stuff_val2017.json as shown below:

from torchvision.datasets import CocoDetection

pms_stf_train2017_data = CocoDetection(
    root="data/coco/anns/stuff_trainval2017/stuff_train2017_pixelmaps",
    annFile="data/coco/anns/stuff_trainval2017/stuff_train2017.json"
)

pms_stf_val2017_data = CocoDetection(
    root="data/coco/anns/stuff_trainval2017/stuff_val2017_pixelmaps",
    annFile="data/coco/anns/stuff_trainval2017/stuff_val2017.json"
)

pms_stf_train2017_data[0] # Error

pms_stf_val2017_data[0] # Error

FileNotFoundError: [Errno 2] No such file or directory: '/.../data/coco/anns/stuff_trainval2017/stuff_train2017_pixelmaps/000000000009.jpg'

FileNotFoundError: [Errno 2] No such file or directory: '/../data/coco/anns/stuff_trainval2017/stuff_val2017_pixelmaps/000000000139.jpg'

And, CocoDetection() doesn't work using panoptic_train2017 with panoptic_train2017.json and using panoptic_val2017 and panoptic_val2017.json as shown below:

from torchvision.datasets import CocoDetection

pan_train2017_data = CocoDetection(
    root="data/coco/anns/panoptic_trainval2017/panoptic_train2017",
    annFile="data/coco/anns/panoptic_trainval2017/panoptic_train2017.json"
) # Error

pan_val2017_data = CocoDetection(
    root="data/coco/anns/panoptic_trainval2017/panoptic_val2017",
    annFile="data/coco/anns/panoptic_trainval2017/panoptic_val2017.json"
) # Error

KeyError: 'id'

Motivation, pitch

So, CocoDetection() should support for them.

Alternatives

No response

Additional context

No response