Running video demo

[rmhiri-gpsw]

Hello,

I tried to run the video demo using mmdet:

python demo/top_down_pose_tracking_demo_with_mmdet.py ./demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py ./faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/ViTPose_base_coco_256x192.py ./vitpose-b.pt --video-path ./test.MOV --out-video-root ./output_video/
but I have errors due to the version compatibility between mmcv, mmdet et the current ViTPose (or mmpose) version.

So here what I do, I install mmcv from sources (1.3.9 version as recommended in the read me of this repo) and the mmdet from sources as well (I tried last mmdet, mmdet==2.14.0 as it is recommended in the mminstall.txt for mmpose 0.24.0: ['mmcv-full>=1.3.8', 'mmdet>=2.14.0', 'mmtrack>=0.6.0']., and mmdet==2.23.0)

here is what I have with the following versions for example (pip list) :
mmcv 1.3.9
mmdet 2.14.0
mmpose 0.24.0

Note that I use this :
torch 1.11.0+cu113
torchvision 0.12.0+cu113

I got this error :
/home/ubuntu/venv/lib/python3.8/site-packages/mmcv/cnn/bricks/transformer.py:27: UserWarning: Fail to import ``MultiScaleDeformableAttention`` from ``mmcv.ops.multi_scale_deform_attn``, You should install ``mmcv-full`` if you need this module. warnings.warn('Fail to import ``MultiScaleDeformableAttention`` from ' Traceback (most recent call last): File "demo/top_down_pose_tracking_demo_with_mmdet.py", line 190, in <module> main() File "demo/top_down_pose_tracking_demo_with_mmdet.py", line 74, in main assert has_mmdet, 'Please install mmdet to run the demo.' AssertionError: Please install mmdet to run the demo.

when I put mmdet to 2.23.0 I got this error :

AssertionError: MMCV==1.3.9 is used but incompatible. Please install mmcv>=1.3.17, <=1.6.0.

Tried to set mmcv>=1.3.17, did not resolve the problem !

can you please tell us which versions (mmcv and mmdet) are recommended to run ViTPose on videos ?

[Annbless]

Hi,

you can try the web demo here link.

As demonstrated in the error msg, it seems that the error is caused that the installed package is mmcv not mmcv-full. Please use MMCV_WITH_OPS=1 pip install xxx to install mmcv-full.

[kbpk]

Hi,

it worked for me like in the web demo with YOLOX (proper installation order):

• CUDA 11.7
• Python 3.9 (also works with 3.10)
• torch (CUDA 11.6, latest)
• (optional, I think) you can install NVIDIA/apex (latest - probably you'll have to comment some code (exactly add return in 32 line of setup.py) due to CUDA version), it's some demo's (optional?) dependency
• mmcv v1.5.0
• ViTPose (latest)
• that line pip install timm==0.4.9 einops
• mmdetection (latest) - there you have YOLOX config files (download links for models)

[YuBeomGon]

thank kbpk,
I get great help

for nvidia rtx 2080ti , 3090ti
its working

install cuda 11.7 (compatible to torch 1.12.1)
https://developer.nvidia.com/cuda-11-7-1-download-archive?target_os=Linux&target_arch=x[…]ribution=Ubuntu&target_version=20.04&target_type=deb_local
rm -rf ~/anaconda3/envs/mmdemo
conda create --name mmdemo python=3.9 -y
conda activate mmdemo
2. install torch 1.12.1
https://pytorch.org/get-started/previous-versions/
conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge
3. mmcv (it takes long time)
(need to mmcv>=1.3.17, <=1.7.0.)
pip install mmcv-full==1.5.0 -f https://download.openmmlab.com/mmcv/dist/cu117/torch1.12.1/index.html
https://github.com/open-mmlab/mmcv
ViTPose
https://github.com/ViTAE-Transformer/ViTPose

git clone https://github.com/ViTAE-Transformer/ViTPose.git
cd ViTPose
pip install -v -e .
pip install timm==0.4.9 einops
mmdet 3rd party package install
pip install mmdet
https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md/#Installation
pose inference with det
(in ViTPose )
python demo/top_down_video_demo_with_mmdet.py demo/mmdetection_cfg/retinanet_swin-t-p4-w7_fpn_2x_coco.py demo/mmdet_checkpoints/swin_t_p4_w7_naju_epoch_24.pth configs/inter/ViTPose_base_coco_256x192.py work_dirs/ViTPose_base_coco_256x192/best_AP_epoch_70.pth --video-path video/out_7.mp4 --out-video-root outputs (편집됨)

[ZhengdiYu]

thank kbpk, I get great help

for nvidia rtx 2080ti , 3090ti its working

install cuda 11.7 (compatible to torch 1.12.1) https://developer.nvidia.com/cuda-11-7-1-download-archive?target_os=Linux&target_arch=x[…]ribution=Ubuntu&target_version=20.04&target_type=deb_local rm -rf ~/anaconda3/envs/mmdemo conda create --name mmdemo python=3.9 -y conda activate mmdemo 2. install torch 1.12.1 https://pytorch.org/get-started/previous-versions/ conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge 3. mmcv (it takes long time) (need to mmcv>=1.3.17, <=1.7.0.) pip install mmcv-full==1.5.0 -f https://download.openmmlab.com/mmcv/dist/cu117/torch1.12.1/index.html https://github.com/open-mmlab/mmcv ViTPose https://github.com/ViTAE-Transformer/ViTPose

git clone https://github.com/ViTAE-Transformer/ViTPose.git cd ViTPose pip install -v -e . pip install timm==0.4.9 einops mmdet 3rd party package install pip install mmdet https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md/#Installation pose inference with det (in ViTPose ) python demo/top_down_video_demo_with_mmdet.py demo/mmdetection_cfg/retinanet_swin-t-p4-w7_fpn_2x_coco.py demo/mmdet_checkpoints/swin_t_p4_w7_naju_epoch_24.pth configs/inter/ViTPose_base_coco_256x192.py work_dirs/ViTPose_base_coco_256x192/best_AP_epoch_70.pth --video-path video/out_7.mp4 --out-video-root outputs (편집됨)

It's still not working for me. It seems the mmcv version needed for mmdet is always not compatible the version that mmpose used in this repo needs.

[NielsRogge]

Video demo available at https://huggingface.co/spaces/hysts/ViTPose-transformers. Code is open-source at https://huggingface.co/spaces/hysts/ViTPose-transformers/tree/main.

See also #157