How to feed negative samples during Faster R-CNN training

See original GitHub issue

Hi all, I have lots of non-annotated images in my training set, where there is no object of interest but there are couple other objects that should be interpreted as part of background. Is there any way I can provide background (negative) samples explicitly in my dataloder? I tried to set a single fake bounding box with label zero for those non-annotated images, and set my num_classes as 3, i.e., I have 2 objects and background, and then performed transfer learning,

model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True, pretrained_backbone=False) in_features = model.roi_heads.box_predictor.cls_score.in_features model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)

But I received a crash at /torchvision/models/detection/roi_heads.py", line 34, in fastrcnn_loss sampled_pos_inds_subset = torch.nonzero(labels > 0).squeeze(1)

I think this is happening because I have fed some images with only label zero, i.e., with no positive bbox. Is there any workaround for that purpose?

Issue Analytics

State:
Created 4 years ago
Reactions:7
Comments:23 (5 by maintainers)

Top GitHub Comments

6reactions

rronancommented, Jan 17, 2020

Hi, Do you plan to support these negative samples during training? By filtering the images beforehand, I get a lot of false positive detections during inference, when the input image in empty. Without this feature I need a second model to classify empty images, sharing the same backbone (or do you think of another workaround?). Thanks

2reactions

amirhesamyazdicommented, Mar 7, 2022

Thank you all for support. Can anyone unequivocally confirm the following?

1- Can we explicitly include empty images (no object) in Faster R-CNN family and DETR by simply adding the empty images in our coco format json file (so we have many extra entries in images) but then NO entry in annotations. Meaning there is nothing added under annotations[] with those image_id s.

2- I was thinking of adding an empty annotation covering area of 0 or a full annotation covering everything and label it as empty class explicitly but if the former can work, it would be easier and much better.

Can anyone confirm for sure that if I do it the first way, the model will in fact incorporate those images into loss calculation? I know it adds more background imbalance, but sometimes you gotta do what you gotta do to reduce false alarms.

Appreciate your help.