Python-Mmdetection v2.20.0: Open MMLab Detection Toolbox

icon
Latest Release: v2.20.0

New Features

  • Support TOOD: Task-aligned One-stage Object Detection (ICCV 2021 Oral) (#6746)
  • Support resuming from the latest checkpoint automatically (#6727)

Bug Fixes

  • Fix wrong bbox loss_weight of the PAA head (#6744)
  • Fix the padding value of gt_semantic_seg in batch collating (#6837)
  • Fix test error of lvis when using classwise (#6845)
  • Avoid BC-breaking of get_local_path (#6719)
  • Fix bug in sync_norm_hook when the BN layer does not exist (#6852)
  • Use pycocotools directly no matter what platform it is (#6838)

Improvements

  • Add unit test for SimOTA with no valid bbox (#6770)
  • Use precommit to check readme (#6802)
  • Support selecting GPU-ids in non-distributed testing time (#6781)

Contributors

A total of 16 developers contributed to this release. Thanks @ZwwWayne, @Czm369, @jshilong, @RangiLyu, @BIGWangYuDong, @hhaAndroid, @jamiechoi1995, @AronLin, @Keiku, @gkagkos, @fcakyon, @www516717402, @vansin, @zactodd, @kimnamu, @jenhaoyang

New Contributors

  • @jamiechoi1995 made their first contribution in https://github.com/open-mmlab/mmdetection/pull/6795
  • @Keiku made their first contribution in https://github.com/open-mmlab/mmdetection/pull/6865
  • @gkagkos made their first contribution in https://github.com/open-mmlab/mmdetection/pull/6744
  • @vansin made their first contribution in https://github.com/open-mmlab/mmdetection/pull/6858
  • @zactodd made their first contribution in https://github.com/open-mmlab/mmdetection/pull/6764
  • @kimnamu made their first contribution in https://github.com/open-mmlab/mmdetection/pull/6906
  • @jenhaoyang made their first contribution in https://github.com/open-mmlab/mmdetection/pull/6881

Full Changelog: https://github.com/open-mmlab/mmdetection/compare/v2.19.1...v2.20.0

Source code(tar.gz)
Source code(zip)

MMDetection

News: We released the technical report on ArXiv.

Documentation: https://mmdetection.readthedocs.io/

Introduction

The master branch works with PyTorch 1.3 to 1.5. The old v1.x branch works with PyTorch 1.1 to 1.4, but v2.0 is strongly recommended for faster speed, higher performance, better design and more friendly usage.

MMDetection is an open source object detection toolbox based on PyTorch. It is a part of the OpenMMLab project developed by Multimedia Laboratory, CUHK.

demo image

Major features

  • Modular Design

    We decompose the detection framework into different components and one can easily construct a customized object detection framework by combining different modules.

  • Support of multiple frameworks out of box

    The toolbox directly supports popular and contemporary detection frameworks, e.g. Faster RCNN, Mask RCNN, RetinaNet, etc.

  • High efficiency

    All basic bbox and mask operations run on GPUs. The training speed is faster than or comparable to other codebases, including Detectron2, maskrcnn-benchmark and SimpleDet.

  • State of the art

    The toolbox stems from the codebase developed by the MMDet team, who won COCO Detection Challenge in 2018, and we keep pushing it forward.

Apart from MMDetection, we also released a library mmcv for computer vision research, which is heavily depended on by this toolbox.

License

This project is released under the Apache 2.0 license.

Changelog

v2.0.0 was released in 6/5/2020. Please refer to changelog.md for details and release history.

Benchmark and model zoo

Supported methods and backbones are shown in the below table. Results and models are available in the model zoo.

ResNet ResNeXt SENet VGG HRNet
RPN
Fast R-CNN
Faster R-CNN
Mask R-CNN
Cascade R-CNN
Cascade Mask R-CNN
SSD
RetinaNet
GHM
Mask Scoring R-CNN
Double-Head R-CNN
Grid R-CNN (Plus)
Hybrid Task Cascade
Libra R-CNN
Guided Anchoring
FCOS
RepPoints
Foveabox
FreeAnchor
NAS-FPN
ATSS
FSAF
PAFPN

Other features

Installation

Please refer to install.md for installation and dataset preparation.

Get Started

Please see getting_started.md for the basic usage of MMDetection.

Contributing

We appreciate all contributions to improve MMDetection. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMDetection is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new detectors.

Citation

If you use this toolbox or benchmark in your research, please cite this project.

@article{mmdetection,
  title   = {{MMDetection}: Open MMLab Detection Toolbox and Benchmark},
  author  = {Chen, Kai and Wang, Jiaqi and Pang, Jiangmiao and Cao, Yuhang and
             Xiong, Yu and Li, Xiaoxiao and Sun, Shuyang and Feng, Wansen and
             Liu, Ziwei and Xu, Jiarui and Zhang, Zheng and Cheng, Dazhi and
             Zhu, Chenchen and Cheng, Tianheng and Zhao, Qijie and Li, Buyu and
             Lu, Xin and Zhu, Rui and Wu, Yue and Dai, Jifeng and Wang, Jingdong
             and Shi, Jianping and Ouyang, Wanli and Loy, Chen Change and Lin, Dahua},
  journal= {arXiv preprint arXiv:1906.07155},
  year={2019}
}

Contact

This repo is currently maintained by Kai Chen (@hellock), Yuhang Cao (@yhcao6), Wenwei Zhang (@ZwwWayne), Jiarui Xu (@xvjiarui). Other core developers include Jiangmiao Pang (@OceanPang) and Jiaqi Wang (@myownskyW7).

Comments

  • Error: 'eval_res' is empty
    Error: 'eval_res' is empty

    Jan 19, 2022

    The training proceed normally but fail to evaluate. I think the eval path is correct but I don't what the problem is. Here is the screenshot. Thank you !

    Screenshot from 2022-01-19 12-04-38

    Reply
  • Fail to create pixmap with Tk_GetPixmap in TkImgPhotoInstanceSetSize ( Fail to allocate bitmap)
    Fail to create pixmap with Tk_GetPixmap in TkImgPhotoInstanceSetSize ( Fail to allocate bitmap)

    Jan 19, 2022

    Describe the bug I run the cascade_mask_rcnn pretrained model with 20s long video, but it show up an error message when it run to 356th frame. this is the error message : Fail to create pixmap with Tk_GetPixmap in TkImgPhotoInstanceSetSize . I already search the error message, some solution said the plot is not close properly. And I can not track the code to close the plot.

    Is any package or library I miss?

    Reproduction

    1. I run this command
    python demo/video_demo.py demo/test1.mp4 configs/cascade_rcnn/cascade_mask_rcnn_x101_64x4d_fpn_mstrain_3x_coco.py checkpoints/cascade_mask_rcnn_x101_64x4d_fpn_mstrain_3x_coco_20210719_210311-d3e64ba0.pth --out result.mp4
    

    Environment

    sys.platform: win32 Python: 3.7.11 (default, Jul 27 2021, 09:42:29) [MSC v.1916 64 bit (AMD64)] CUDA available: True GPU 0: NVIDIA GeForce RTX 3060 CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4 NVCC: Not Available GCC: n/a PyTorch: 1.8.0 PyTorch compiling details: PyTorch built with:

    • C++ Version: 199711
    • MSVC 192829337
    • Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191125 for Intel(R) 64 architecture applications
    • Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
    • OpenMP 2019
    • CPU capability usage: AVX2
    • CUDA Runtime 11.1
    • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
    • CuDNN 8.0.5
    • Magma 2.5.4
    • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -DNDEBUG -DUSE_FBGEMM -DUSE_XNNPACK, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON,

    TorchVision: 0.9.0 OpenCV: 4.5.5 MMCV: 1.4.0 MMCV Compiler: MSVC 191627045 MMCV CUDA Compiler: 11.4 MMDetection: 2.20.0+ff9bc39

    Error message

    [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>                     ] 356/601, 2.0 task/s, elapsed: 180s, ETA:   124sFail to create pixmap with Tk_GetPixmap in TkImgPhotoInstanceSetSize
    
    Reply
  • [Enhance] Add setup multi-processing both in train and test.
    [Enhance] Add setup multi-processing both in train and test.

    Jan 19, 2022

    Motivation

    Add setup multi-processing both in train and test. Add unit tests.

    Reply
  • Inference error: AttributeError: module 'pycocotools' has no attribute '__version__'`
    Inference error: AttributeError: module 'pycocotools' has no attribute '__version__'`

    Jan 19, 2022

    `AttributeError Traceback (most recent call last) in 5 import torch 6 from mmcv import Config ----> 7 from mmdet.apis import inference_detector, init_detector, show_result_pyplot 8 import os 9 import glob

    ~/mmdetection/mmdet/apis/init.py in 2 init_detector, show_result_pyplot) 3 from .test import multi_gpu_test, single_gpu_test ----> 4 from .train import get_root_logger, set_random_seed, train_detector 5 6 all = [

    ~/mmdetection/mmdet/apis/train.py in 9 10 from mmdet.core import DistEvalHook, EvalHook ---> 11 from mmdet.datasets import (build_dataloader, build_dataset, 12 replace_ImageToTensor) 13 from mmdet.utils import get_root_logger

    ~/mmdetection/mmdet/datasets/init.py in 1 from .builder import DATASETS, PIPELINES, build_dataloader, build_dataset ----> 2 from .cityscapes import CityscapesDataset 3 from .coco import CocoDataset 4 from .custom import CustomDataset 5 from .dataset_wrappers import (ClassBalancedDataset, ConcatDataset,

    ~/mmdetection/mmdet/datasets/cityscapes.py in 14 15 from .builder import DATASETS ---> 16 from .coco import CocoDataset 17 18

    ~/mmdetection/mmdet/datasets/coco.py in 19 import pycocotools 20 if not hasattr(pycocotools, 'sphinx_mock'): # for doc generation ---> 21 assert pycocotools.version >= '12.0.2' 22 except AssertionError: 23 raise AssertionError('Incompatible version of pycocotools is installed. '

    AttributeError: module 'pycocotools' has no attribute 'version'`

    Reply
  • Support the README for a simple object detection without mask detection
    Support the README for a simple object detection without mask detection

    Jan 19, 2022

    Describe the feature I want the README for simple object detection without mask detection Motivation I am going to train a simple object detection model with SwinTransformer based RetinaNet. I can NOT find an exact README for training a model detecting ONLY bounding box without a mask.

    Reply
  • [Feature] Support visualization for Panoptic Segmentation
    [Feature] Support visualization for Panoptic Segmentation

    Jan 19, 2022

    support visualization for panoptic segmentation

    WIP 
    Reply
  • Out of memory when training on custom dataset
    Out of memory when training on custom dataset

    Dec 16, 2018

    I was trying to train a retinanet model on some custom dataset (e.g. WIDER face) and I've encountered consistent out of memory issue after several hundred (or thousand) iteration. The error messages are like the followings:

      File "/root/mmdetection/mmdet/core/bbox/assigners/max_iou_assigner.py", line 72, in assign
        overlaps = bbox_overlaps(bboxes, gt_bboxes)
      File "/root/mmdetection/mmdet/core/bbox/geometry.py", line 59, in bbox_overlaps
        ious = overlap / (area1[:, None] + area2 - overlap)
    RuntimeError: CUDA error: out of memory
    

    or

      File "/root/mmdetection/mmdet/core/bbox/assigners/max_iou_assigner.py", line 72, in assign
        overlaps = bbox_overlaps(bboxes, gt_bboxes)
      File "/root/mmdetection/mmdet/core/bbox/geometry.py", line 51, in bbox_overlaps
        wh = (rb - lt + 1).clamp(min=0)  # [rows, cols, 2]
    RuntimeError: CUDA error: out of memory
    

    The training machine has 2 x 1080 Ti and the OOM happens no matter imgs_per_gpu is 2 or 1. In fact, even if the initial memory consumption reduces to around 3~4G when imgs_per_gpu=1, it quickly grows to around 8G and fluctuate for a while until finally OOM. I've tried both distributed and non-distributed training and both suffer OOM.

    BTW, the machine has the following software configurations:

    • Pytorch = 0.4.1
    • CUDA = 9
    • Cudnn = 7

    Is this normal?

    Reply
  • Errors with vfnet and htc pretrained models
    Errors with vfnet and htc pretrained models

    Dec 15, 2020

    I call mmdetection from the mmpose in docker, that was made by mmpose docker/Dockerfile . Requirements were installed with pip and apt tools. I run with the following command:

    python demo/top_down_video_demo_with_mmdet.py --device=cpu output/mmdetection/configs/vfnet/vfnet_x101_32x4d_fpn_mdconv_c3-c5_mstrain_2x_coco.py https://openmmlab.oss-cn-hangzhou.aliyuncs.com/mmdetection/v2.0/vfnet/vfnet_x101_32x4d_fpn_mdconv_c3-c5_mstrain_2x_coco/vfnet_x101_32x4d_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-d300a6fc.pth configs/top_down/hrnet/coco-wholebody/hrnet_w48_coco_wholebody_384x288.py https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288-6e061c6a_20200922.pth --video-path output/input/input.mkv --out-video-root output/hrnet_w48_coco_wholebody_384x288/

    The most interesting here is vfnet_x101_32x4d_fpn_mdconv_c3-c5_mstrain_2x_coco.py and vfnet_x101_32x4d_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-d300a6fc.pth .

    That command returns error:

        main()
      File "demo/top_down_video_demo_with_mmdet.py", line 63, in main
        args.det_config, args.det_checkpoint, device=args.device)
      File "/opt/conda/lib/python3.7/site-packages/mmdet/apis/inference.py", line 41, in init_detector
        checkpoint = load_checkpoint(model, checkpoint, map_location=map_loc)
      File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 247, in load_checkpoint
        checkpoint = _load_checkpoint(filename, map_location)
      File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 219, in _load_checkpoint
        checkpoint = load_url_dist(filename)
      File "/opt/conda/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 111, in load_url_dist
        checkpoint = model_zoo.load_url(url, model_dir=model_dir)
      File "/opt/conda/lib/python3.7/site-packages/torch/hub.py", line 490, in load_state_dict_from_url
        raise RuntimeError('Only one file(not dir) is allowed in the zipfile')
    RuntimeError: Only one file(not dir) is allowed in the zipfile
    

    The same problem with mmdetection/configs/htc/htc_x101_64x4d_fpn_16x1_20e_coco.py and mmdetection/configs/htc/htc_x101_32x4d_fpn_16x1_20e_coco.py .

    At that, some detectors work fine, for example mmdetection/configs/cascade_rcnn/cascade_rcnn_r101_fpn_1x_coco.py with http://download.openmmlab.com/mmdetection/v2.0/cascade_rcnn/cascade_rcnn_r101_fpn_1x_coco/cascade_rcnn_r101_fpn_1x_coco_20200317-0b6a2fbf.pth model. I didn't remember exact names, but a lot of others detectors work also.

    Reply
  • ImportError: libtorch_cpu.so: cannot open shared object file: No such file or directory
    ImportError: libtorch_cpu.so: cannot open shared object file: No such file or directory

    May 6, 2020

    when I run "python tools/test.py configs/faster_rcnn_r50_fpn_1x.py checkpoints/faster_rcnn_r50_fpn_1x_20181010-3d1b3351.pth" there are some errors Traceback (most recent call last): File "tools/test.py", line 8, in from tools.fuse_conv_bn import fuse_module File "/mnt/WXRG0349/zhuoli/mmdetection/tools/fuse_conv_bn.py", line 7, in from mmdet.apis import init_detector File "/mnt/WXRG0349/zhuoli/mmdetection/mmdet/apis/init.py", line 1, in from .inference import (async_inference_detector, inference_detector, File "/mnt/WXRG0349/zhuoli/mmdetection/mmdet/apis/inference.py", line 11, in from mmdet.core import get_classes File "/mnt/WXRG0349/zhuoli/mmdetection/mmdet/core/init.py", line 7, in from .post_processing import * # noqa: F401, F403 File "/mnt/WXRG0349/zhuoli/mmdetection/mmdet/core/post_processing/init.py", line 1, in from .bbox_nms import multiclass_nms File "/mnt/WXRG0349/zhuoli/mmdetection/mmdet/core/post_processing/bbox_nms.py", line 3, in from mmdet.ops.nms import nms_wrapper File "/mnt/WXRG0349/zhuoli/mmdetection/mmdet/ops/init.py", line 19, in from .utils import get_compiler_version, get_compiling_cuda_version File "/mnt/WXRG0349/zhuoli/mmdetection/mmdet/ops/utils/init.py", line 2, in from .compiling_info import get_compiler_version, get_compiling_cuda_version ImportError: libtorch_cpu.so: cannot open shared object file: No such file or directory

    Environment pytorch1.4.0+python3.7.0 use "pip" install the pytorch

    Reply
  • YOLOv3 - Continue on #1695
    YOLOv3 - Continue on #1695

    Jun 20, 2020

    Hi all,

    @WenqiangX and I are glad to help implement YOLOv3, and perhaps v4 as well, if everything goes on well. We are from (MVIG, SJTU)[http://mvig.sjtu.edu.cn] and have spent quite a little time surveying all kinds of YOLOv3 implementation in PyTorch. Also, we have spent a lot of time working on the gluon-cv version of YOLOv3, which relays on mxnet. So, we are familiar with it.

    Months ago, @wuhy08 has contributed a lot (#1695), but there are still some problems to be solved. I have tested his implementation and found it can be trained using one vid card and achieve ~38 mAP in (618, 618) resolution, which is pretty good. However, it fails to train on multiple cards and achieves very low mAP even with find_unused_parameters=True. Still, great thanks to him and Western Digital.

    Basically, we aim to complete the following list:

    • [x] Merge @wuhy08 's code with mm-detection 2.0
    • [x] And test it!
    • [x] Move some hyper-params to the config file
    • [x] Refactor the backbone with your new ConvModule (as mentioned in #1695 (comment))
    • [x] Refactor the backbone and neck to support other modules
    • [x] Introduce some great ideas from gluon-cv
    • [x] Convert the darknet pretrained file (May have difficulty and help is needed)
    • [x] Fix the labels +1 / -1 problem

    If you have any suggestions, feel free to comment.

    Also, I hope that the authors of this project can manage your time to review the code to see if there is any major problems so that we can save some time.

    This pr is still in an early stage, but is expected to finish by the end of July.

    If everything is good, could you please close the original pr so that we can continue to work here?

    Reply
  • Pytorch2onnx
    Pytorch2onnx

    Jun 19, 2020

    Support convert RetinaNet from Pytorch to ONNX. We can verify the computation results between Pytorch and ONNX. We do several things in this PR: [1] Replace some Pytorch op that are not supported by ONNX [2] Replace some dynamic shape by static shape, as ONNX only support constant shape in some case [3] Fix some bugs in Pytorch1.3 while converting to ONNX, which may cause numerical error while running by onnxruntime [4] Update tool/pytorch2onnx.py file with our new API

    Reply
  • ModuleNotFoundError: No module named 'mmcv._ext'
    ModuleNotFoundError: No module named 'mmcv._ext'

    Jul 10, 2020

    Someone has already post this issue earlier, and I just wanna say I got the same issue. I'm using mmedetection on colab. I installed the program as usual, but when I tried to import inference_detector from mmdet.apis, I got the error as described in the title which I've never met before. I thought it may because some updates on your side were on progress, but I wasn't sure. Any suggestion? Thank you.

    installation/env 
    Reply