AGX Xavier 搭建360环视教程【四、主线流程和原理-硬解码】

Jetson Xavier AGX 下基于 GStreamer + CUDA + OpenCV 的 四路 RTSP → 硬解码 → CUDA remap → 拼接 → GStreamer 推 RTSP 的 C++代码。

📌 一、思路

1️⃣ 用 GStreamer pipeline 拉取 RTSP 流
2️⃣ 用 nvv4l2decoder 实现 硬件 H.264/H.265 解码
3️⃣ 将解码后的帧放入 appsink，交给 OpenCV (或 CUDA)
4️⃣ 用 cv::cuda::remap 做畸变矫正
5️⃣ 用 OpenCV CUDA 拼接
6️⃣ 拼接好的帧放回 appsrc，通过 nvv4l2h264enc 编码
7️⃣ rtspclientsink 或 udpsink 推给外部 RTSP 服务器（比如 mediamtx）

---

全流程用 appsink/appsrc 把 GStreamer 和 OpenCV/CUDA 串起来

核心思路

• 每路 RTSP 用独立的 GStreamer pipeline + appsink 拉帧。

• 用 OpenCV CUDA (cv::cuda) 做 remap。

• 拼接后从 GPU 拷贝回 CPU (GpuMat.download)，推给 appsrc。

• appsrc 是把 CPU 内存帧送回 GStreamer 编码并通过 RTSP 回推。

• nvv4l2decoder → Jetson 硬件 H264 解码单元

• cv::cuda::remap → GPU 上跑畸变校正

• appsrc / appsink → GStreamer 和 OpenCV 交互

• 全流程 GPU 加速，CPU 只做拼接上传和结果推送。

✅ 二、环境检查

1、确认 nvv4l2decoder、nvvidconv 可用

在终端执行：

gst-inspect-1.0 nvv4l2decoder gst-inspect-1.0 nvvidconv

如果输出了插件描述，比如：

gst-inspect-1.0 nvv4l2decoder
Factory Details:
  Rank                     primary + 11 (267)
  Long-name                NVIDIA v4l2 video decoder
  Klass                    Codec/Decoder/Video
  Description              Decode video streams via V4L2 API
  Author                   Nicolas Dufresne <nicolas.dufresne@collabora.com>, Viranjan Pagar <vpagar@nvidia.com>

Plugin Details:
  Name                     nvvideo4linux2
  Description              Nvidia elements for Video 4 Linux
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideo4linux2.so
  Version                  1.14.0
  License                  LGPL
  Source module            nvvideo4linux2
  Binary package           nvvideo4linux2
  Origin URL               http://nvidia.com/

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstNvV4l2VideoDec
                               +----nvv4l2decoder

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
  
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      image/jpeg
      video/x-h264
          stream-format: { (string)byte-stream }
              alignment: { (string)au }
      video/x-h265
          stream-format: { (string)byte-stream }
              alignment: { (string)au }
      video/mpeg
            mpegversion: 4
           systemstream: false
                 parsed: true
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/mpeg
            mpegversion: [ 1, 2 ]
           systemstream: false
                 parsed: true
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/x-divx
            divxversion: [ 4, 5 ]
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
      video/x-vp8
      video/x-vp9
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "nvv4l2decoder0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  device