Train on images which contain no objects

[echonax07]

Hi first of all thank you for this awesome repo.

I have a highly unbalanced dataset and only 1300/64,000 images contain objects.

I saw that while training only the 1300 images are being used.

Is there a way where I can train on images that only contain background?

I think this will help me eliminate the false positives

Thank you

[Maenansyn]

During training, images with no annotations are filtered and excluded. In validation and testing they are included. Same applies for too small images.

The filtering takes place here if your dataset is for example based on COCO whose dataset class is inherited from the custom dataset class. If your dataset or its super classes does not use the custom dataset you might find something similar in those classes, too.

So if you want to use it, you could create a customized dataset class which inherits the COCO dataset class and overwrite the init with just excluding this part of code, something like this:

Code example

# Copyright (c) OpenMMLab. All rights reserved.
import os.path as osp
import warnings
import mmcv
from mmdet.datasets.builder import DATASETS
from mmdet.datasets.coco import CocoDataset
from mmdet.datasets.pipelines import Compose

@DATASETS.register_module()
class CocoUnfilteredDataset(CocoDataset):

    def __init__(self,
                 ann_file,
                 pipeline,
                 classes=None,
                 data_root=None,
                 img_prefix='',
                 seg_prefix=None,
                 proposal_file=None,
                 test_mode=False,
                 filter_empty_gt=True,
                 file_client_args=dict(backend='disk')):
        self.ann_file = ann_file
        self.data_root = data_root
        self.img_prefix = img_prefix
        self.seg_prefix = seg_prefix
        self.proposal_file = proposal_file
        self.test_mode = test_mode
        self.filter_empty_gt = filter_empty_gt
        self.file_client = mmcv.FileClient(**file_client_args)
        self.CLASSES = self.get_classes(classes)

        # join paths if data_root is specified
        if self.data_root is not None:
            if not osp.isabs(self.ann_file):
                self.ann_file = osp.join(self.data_root, self.ann_file)
            if not (self.img_prefix is None or osp.isabs(self.img_prefix)):
                self.img_prefix = osp.join(self.data_root, self.img_prefix)
            if not (self.seg_prefix is None or osp.isabs(self.seg_prefix)):
                self.seg_prefix = osp.join(self.data_root, self.seg_prefix)
            if not (self.proposal_file is None
                    or osp.isabs(self.proposal_file)):
                self.proposal_file = osp.join(self.data_root,
                                              self.proposal_file)
        # load annotations (and proposals)
        if hasattr(self.file_client, 'get_local_path'):
            with self.file_client.get_local_path(self.ann_file) as local_path:
                self.data_infos = self.load_annotations(local_path)
        else:
            warnings.warn(
                'The used MMCV version does not have get_local_path. '
                f'We treat the {self.ann_file} as local paths and it '
                'might cause errors if the path is not a local path. '
                'Please use MMCV>= 1.3.16 if you meet errors.')
            self.data_infos = self.load_annotations(self.ann_file)

        if self.proposal_file is not None:
            if hasattr(self.file_client, 'get_local_path'):
                with self.file_client.get_local_path(
                        self.proposal_file) as local_path:
                    self.proposals = self.load_proposals(local_path)
            else:
                warnings.warn(
                    'The used MMCV version does not have get_local_path. '
                    f'We treat the {self.ann_file} as local paths and it '
                    'might cause errors if the path is not a local path. '
                    'Please use MMCV>= 1.3.16 if you meet errors.')
                self.proposals = self.load_proposals(self.proposal_file)
        else:
            self.proposals = None

        # processing pipeline
        self.pipeline = Compose(pipeline)

That should roughly do the trick.

[Maenansyn]

But usually your algorithm should find regions of interests, before the classifcation of those predicted RoIs is run. These can then either be classified as one of your defined classes or as background. So it should use the background of your 1,300 images nonetheless. So, you can try this approach, but I don't think that it'll fix your problem and that this is situated somewhere else.

[echonax07]

Hi thanks for taking out the time to answer my question.

I found another way of including the negative samples by setting filter_empty_gt=False in the train dictionary in the config file.
Something like this

train=dict(
        img_prefix=path_train_data,
        classes=classes,
        ann_file=path_train_anno,
        pipeline=train_pipeline,
        filter_empty_gt=False),

For my dataset, the appearance of background texture highly resembles to the appearance of the objects. So I think it will help me eliminate the false positives that the previous model is producing.

[Maenansyn]

Ah, okay, then they might help. And I didn't know that config option. Thanks for the information. :)