【Paddle笔记】使用PaddleGan工具包实现表情迁移(First Order Motion)

PaddleGan工具包提供了Motion Driving DEMO，实现用一个视频驱动图片成动画，让图片做出与驱动视频中相似的动作。

• 使用S3FD人脸检测模型将照片中的每张人脸检测出来并抠出
• 使用First Order Motion模型对抠出的每张人脸进行脸部表情迁移
• 将完成表情迁移的人脸进行适当剪裁后贴回原照片位置
• 人脸增强特效使得驱动后的大大提升视频中人脸清晰度

PaddleGAN针对人脸的相关处理提供faceutil工具，包括人脸检测、五官分割、关键点检测等能力。

1、环境安装

1.1 运行环境

1.1.1 Conda虚拟环境

创建ppai
conda create -n ppai python=3.7
激活环境
conda activate ppai

1.1.2 PyTorch

Python3.7推荐版本:
pytorch1.13.1 + torchvision0.14.1 + torchaudio0.13.1 + cuda11.7
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia

安装遇到疑难参考 这里

1.1.3 Tensorflow

Python3.7推荐版本:
tensorflow-gpu 1.15
pip install tensorflow-gpu==1.15 -i https://pypi.tuna.tsinghua.edu.cn/simple

安装遇到疑难参考 这里

1.2 Paddle核心框架

1.2.1 安装Paddle框架

• CPU 版本
  pip install paddlepaddle
• GPU 版本
  pip install paddlepaddle-gpu

(ppai) xf@VP01:~/ai/face$ pip install paddlepaddle-gpu
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple, https://pypi.ngc.nvidia.com
Collecting paddlepaddle-gpu
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/48/6d/66e8d97902ce232d07efd0741325ad3b7e45d7b9ccf7140e7a637db8893d/paddlepaddle_gpu-2.4.2-cp37-cp37m-manylinux1_x86_64.whl (584.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 584.8/584.8 MB 6.8 MB/s eta 0:00:00
Requirement already satisfied: opt-einsum==3.3.0 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (3.3.0)
Requirement already satisfied: requests>=2.20.0 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (2.28.1)
Requirement already satisfied: six in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (1.16.0)
Requirement already satisfied: Pillow in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (9.4.0)
Requirement already satisfied: decorator in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (5.1.1)
Requirement already satisfied: protobuf<=3.20.0,>=3.1.0 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (3.20.0)
Requirement already satisfied: astor in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (0.8.1)
Requirement already satisfied: numpy>=1.13 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (1.21.5)
Requirement already satisfied: paddle-bfloat==0.1.7 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from paddlepaddle-gpu) (0.1.7)
Requirement already satisfied: charset-normalizer<3,>=2 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu) (2.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu) (1.26.14)
Requirement already satisfied: certifi>=2017.4.17 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu) (2022.12.7)
Requirement already satisfied: idna<4,>=2.5 in /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages (from requests>=2.20.0->paddlepaddle-gpu) (3.4)
Installing collected packages: paddlepaddle-gpu
Successfully installed paddlepaddle-gpu-2.4.2

1.2.2 验证框架是否安装成功

python -c "import paddle; print(paddle.__version__)"

(ppai) xf@VP01:~/ai/face$ python -c "import paddle; print(paddle.__version__)"
2.4.2

python -c "import paddle;paddle.utils.run_check()"

(ppai) xf@VP01:~/ai/face$ python -c "import paddle;paddle.utils.run_check()"
Running verify PaddlePaddle program ...
W0428 03:41:27.101418  1187 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 12.1, Runtime API Version: 10.2
W0428 03:41:27.775382  1187 gpu_resources.cc:91] device: 0, cuDNN Version: 7.4.
PaddlePaddle works well on 1 GPU.
PaddlePaddle works well on 1 GPUs.
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.

1.3 PaddleGAN 生成对抗网络

飞桨生成对抗网络开发套件–PaddleGAN，为开发者提供经典及前沿的生成对抗网络高性能实现，并支撑开发者快速构建、训练及部署生成对抗网络，以供学术、娱乐及产业应用。

1.3.1 安装ppgan

pip install --upgrade ppgan

1.3.2 安装其他依赖

pip install -r requirements.txt

1.3.3 下载示例代码、配置文件、中英文档

git clone https://github.com/PaddlePaddle/PaddleGAN
cd PaddleGAN

(ppai) xf@VP01:~/ai/face$ git clone https://github.com/PaddlePaddle/PaddleGAN
Cloning into 'PaddleGAN'...
remote: Enumerating objects: 5606, done.
remote: Counting objects: 100% (408/408), done.
remote: Compressing objects: 100% (222/222), done.
remote: Total 5606 (delta 238), reused 319 (delta 182), pack-reused 5198
Receiving objects: 100% (5606/5606), 163.63 MiB | 17.37 MiB/s, done.
Resolving deltas: 100% (3612/3612), done.
Updating files: 100% (565/565), done.
(ppai) xf@VP01:~/ai/face$ cd PaddleGAN
(ppai) xf@VP01:~/ai/face/PaddleGAN$ ll
total 56
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 ./
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 ../
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 .git/
-rwxrwxrwx 1 xf xf  1242 Apr 28 05:02 .gitignore*
-rwxrwxrwx 1 xf xf  1040 Apr 28 05:02 .pre-commit-config.yaml*
-rwxrwxrwx 1 xf xf    47 Apr 28 05:02 .style.yapf*
-rwxrwxrwx 1 xf xf 11438 Apr 28 05:02 LICENSE*
-rwxrwxrwx 1 xf xf 13335 Apr 28 05:02 README.md*
-rwxrwxrwx 1 xf xf 14430 Apr 28 05:02 README_cn.md*
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 applications/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 benchmark/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 configs/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 data/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 deploy/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 docs/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 education/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 paddlegan-wechaty-demo/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 ppgan/
-rwxrwxrwx 1 xf xf     0 Apr 28 05:02 python*
-rwxrwxrwx 1 xf xf   160 Apr 28 05:02 requirements.txt*
-rwxrwxrwx 1 xf xf  1900 Apr 28 05:02 setup.py*
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 test/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 test_tipc/
drwxrwxrwx 1 xf xf   512 Apr 28 05:02 tools/

1.4 Paddle VisualDL 可视化训练工具

python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple

1.5 错误

1.5.1 ImportError: libcudart.so.10.2: cannot open shared object file: No such file or directory

创建so软链接：

cd /usr/local/cuda/lib64
sudo ln -s libcudart.so.11.0 libcudart.so.10.2
sudo ldconfig

libcudart.so.11.0 这个版本根据本地实际安装为准

(ppai) xf@VP01:/usr/local/cuda/lib64$ ll
...
lrwxrwxrwx 1 root root        17 Sep 22  2022 libcudart.so -> libcudart.so.11.0
lrwxrwxrwx 1 root root        17 Apr 17 13:12 libcudart.so.10.0 -> libcudart.so.11.0
lrwxrwxrwx 1 root root        20 Sep 22  2022 libcudart.so.11.0 -> libcudart.so.11.8.89
-rw-r--r-- 1 root root    679264 Sep 22  2022 libcudart.so.11.8.89
...

1.5.2 ImportError: /usr/local/cuda/lib64/libcudart.so.10.2: version `libcudart.so.10.2’ not found

本机安装的CUDA是11.8版本，上小节1.5.1中创建的软链接是11.8，正常是CUDA向下兼容，但这里程序明确请求libcudart.so库版本要10.2，但我不想对CUDA11.8降级，其他软件需要用。我们只需要找个libcudart.so.10.2文件放到lib64，让Paddle能调用就行了。

  File "/home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages/paddle/fluid/core.py", line 274, in <module>
    from . import libpaddle
ImportError: /usr/local/cuda/lib64/libcudart.so.10.2: version `libcudart.so.10.2' not found (required by /home/xf/anaconda3/envs/ppai/lib/python3.7/site-packages/paddle/fluid/libpaddle.so)