EVAD复现 Efficient Video Action Detection with Token Dropout and Context Refinement-优快云博客

本文链接：https://blog.youkuaiyun.com/WhiffeYF/article/details/134479996

安装EVAD

官方安装流程：https://github.com/MCG-NJU/EVAD/blob/main/INSTALL.md

平台：Autodl：https://www.autodl.com/
环境：
PyTorch 1.8.1
Python 3.8(ubuntu18.04)
Cuda 11.1

在这里插入图片描述

pip install cython
pip install simplejson
conda install av -c conda-forge -y
pip install -U iopath
pip install psutil
pip install opencv-python
pip install pandas
pip install tensorboard
pip install moviepy
pip install timm==0.4.12

Detectron2

git clone https://gitee.com/YFwinston/detectron2 detectron2_repo
python -m pip install -e detectron2_repo

git clone https://gitee.com/YFwinston/EVAD.git
cd EVAD
python setup.py build develop

demo 测试

ava.json
在/root/EVAD/demo/下面的ava.json写入

{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, "paint": 38, "play board game": 39, "play musical instrument": 40, "play with pets": 41, "point to (an object)": 42, "press": 43, "pull (an object)": 44, "push (an object)": 45, "put down": 46, "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60, "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, "give/serve (an object) to (a person)": 64, "grab (a person)": 65, "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}

配置文件：
在/root/EVAD/demo/下面的
ViT_B_16x4_demo.yaml写入

TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 16
  EVAL_PERIOD: 5
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_INFLATE: False
  CHECKPOINT_FILE_PATH: "/root/EVAD/demo/vitb-k400.pth"
  CHECKPOINT_TYPE: pytorch
  CHECKPOINT_CLEAR_NAME_PATTERN: ("backbone.","encoder.")
DATA:
  PATH_TO_DATA_DIR: "path/to/ava"
  NUM_FRAMES: 16
  SAMPLING_RATE: 4
  INPUT_CHANNEL_NUM: [3]
DETECTION:
  ENABLE: True
  ALIGNED: True
AVA:
  FULL_TEST_ON_VAL: True
  MIN_SIZE_TRAIN: [288,]
  MIN_SIZE_TEST: 288
  CROP_ENABLED: False
  IMG_PROC_BACKEND: detectron2
  DETECTION_SCORE_THRESH: 0.7
  TRAIN_PREDICT_BOX_LISTS: []
  TEST_PREDICT_BOX_LISTS: ["ava_val_v2.2.csv"] # Only used for generating coco format json file for person detector head
ViT:
  EMBED_DIM: 768
  DEPTH: 12
  NUM_HEADS: 12
  USE_CHECKPOINT: True
  DROP_PATH_RATE: 0.1
  FPN_INDICES: [ 2, 5, 8, 11 ]
SOLVER:
  BASE_LR_SCALE_NUM_SHARDS: True
  # default 0.000025 for BATCH=16 0.00001 for BATCH=8
  BASE_LR: 0.000025
  LR_POLICY: steps_with_relative_lrs
  STEPS: [0, 5, 8]
  LRS: [1, 0.1, 0.01]
  LR_SCALE: 8.0
  MAX_EPOCH: 12
  MOMENTUM: 0.9
  WEIGHT_DECAY: 0.0001
  WARMUP_EPOCHS: 2.0
  WARMUP_START_LR: 0.0000025
  OPTIMIZING_METHOD: adamw
MODEL:
  NUM_CLASSES: 80
  ARCH: vit
  MODEL_NAME: EVAD
  LOSS_FUNC: bce
  ROI_BOX_HEAD:
    POOLER_TYPE: "ROIAlignV2"
    POOLER_RESOLUTION: 7
    POOLER_SAMPLING_RATIO: 2
  CRD:
    EMBED_DIM: 384
    DEPTH: 6
    NUM_HEADS: 6
    DROP_PATH_RATE: 0.1
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 4
DATA_LOADER:
  NUM_WORKERS: 8
  PIN_MEMORY: True
NUM_GPUS: 8
NUM_SHARDS: 1
RNG_SEED: 0
LOG_MODEL_INFO: False
OUTPUT_DIR: '/root/autodl-tmp/'
LOG_PERIOD: 50
#TENSORBOARD:
  #ENABLE: True
DEMO:
  ENABLE: True
  LABEL_FILE_PATH: "/root/EVAD/demo/ava.json" # Add local label file path here.
  INPUT_VIDEO: "/root/EVAD/demo/1.mp4"
  OUTPUT_FILE: "/root/EVAD/demo/1_1.mp4"

cd /root/EVAD/
python projects/evad/run_net.py --cfg demo/ViT_B_16x4_demo.yaml

报错

Traceback (most recent call last):
  File "projects/evad/run_net.py", line 5, in <module>
    from slowfast.utils.misc import launch_job, launch_job_slurm, launch_job_tencent
  File "/root/EVAD/slowfast/utils/misc.py", line 21, in <module>
    from slowfast.datasets.utils import pack_pathway_output
  File "/root/EVAD/slowfast/datasets/__init__.py", line 7, in <module>
    from .kinetics import Kinetics  # noqa
  File "/root/EVAD/slowfast/datasets/kinetics.py", line 14, in <module>
    from . import video_container as container
  File "/root/EVAD/slowfast/datasets/video_container.py", line 4, in <module>
    import av
  File "/root/miniconda3/lib/python3.8/site-packages/av/__init__.py", line 20, in <module>
    from av._core import time_base, library_versions
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /root/miniconda3/lib/python3.8/site-packages/av/../../../././libicuuc.so.73)

在这里插入图片描述