Android Codec2 CCodec (三三)组件处理与数据输出 - Ⅰ

本节主要对数据的输出过程做简单梳理,组件的处理过程可以参考SimpleC2Component和C2SoftHevcDec这两章内容,本节不做展开。

1、onWorkDone_nb

组件有数据输出会调用IComponent注册的onWorkDone_nb方法:

virtual void onWorkDone_nb(
        std::weak_ptr<C2Component> /* c2component */,
        std::list<std::unique_ptr<C2Work>> c2workItems) override {
    // 1. 取消input buffer的注册
    for (const std::unique_ptr<C2Work>& work : c2workItems) {
        if (work) {
            if (work->worklets.empty()
                    || !work->worklets.back()
                    || (work->worklets.back()->output.flags &
                        C2FrameData::FLAG_INCOMPLETE) == 0) {
                InputBufferManager::
                        unregisterFrameData(mListener, work->input);
            }
        }
    }

    // 2. 找到IComponentListener
    sp<IComponentListener> listener = mListener.promote();
    if (listener) {
        WorkBundle workBundle;

        sp<Component> strongComponent = mComponent.promote();
        // 3. 开始传输output buffer
        beginTransferBufferQueueBlocks(c2workItems, true);
        // 4. 将C2Work转换为WorkBundle
        if (!objcpy(&workBundle, c2workItems, strongComponent ?
                &strongComponent->mBufferPoolSender : nullptr)) {
            LOG(ERROR) << "Component::Listener::onWorkDone_nb -- "
                        << "received corrupted work items.";
            endTransferBufferQueueBlocks(c2workItems, false, true);
            return;
        }
        // 5. 调用HIDL Callback
        Return<void> transStatus = listener->onWorkDone(workBundle);
        if (!transStatus.isOk()) {
            LOG(ERROR) << "Component::Listener::onWorkDone_nb -- "
                        << "transaction failed.";
            endTransferBufferQueueBlocks(c2workItems, false, true);
            return;
        }
        // 6. 结束传输
        endTransferBufferQueueBlocks(c2workItems, true, true);
    }
}

onWorkDone_nb有以下几点内容需要说明:

  1. 取消InputBufferManager中注册的C2FrameData。input buffer被销毁会自动取消注册,在我们自己实现的组件中,input buffer使用完成应该主动执行clear进行销毁。如果input buffer事先已经被clear了,那么onWorkDone_nb这里调用的unregisterFrameData没有做实质性的动作;
  2. 如果携带了FLAG_INCOMPLETE,说明是一包多帧的情况,要等到frameIndex对应的最后一帧到来再取消注册。
  3. 调用beginTransferBufferQueueBlocks,这个步骤其实就是设置一个flag,详细可以参考 Android Codec2(二四)C2GraphicBlock,该方法只在video且有surface的情况下有用。
virtual Return<void> onWorkDone(const WorkBundle& workBundle) override {
    std::list<std::unique_ptr<C2Work>> workItems;
    if (!objcpy(&workItems, workBundle)) {
        LOG(DEBUG) << "onWorkDone -- received corrupted WorkBundle.";
        return Void();
    }
    // release input buffers potentially held by the component from queue
    std::shared_ptr<Codec2Client::Component> strongComponent =
            component.lock();
    if (strongComponent) {
        strongComponent->handleOnWorkDone(workItems);
    }
    if (std::shared_ptr<Codec2Client::Listener> listener = base.lock()) {
        listener->onWorkDone(component, workItems);
    } else {
        LOG(DEBUG) << "onWorkDone -- listener died.";
    }
    return Void();
}

Codec2Client首先会将WorkBundle转换成C2Work,然后调用Codec2Client::Component的handleOnWorkDone方法,最后调用Callback将C2Work送回到CCodec。

void Codec2Client::Component::handleOnWorkDone(
        const std::list<std::unique_ptr<C2Work>> &workItems) {
    // Output bufferqueue-based blocks' lifetime management
    mOutputBufferQueue->holdBufferQueueBlocks(workItems);
}

handleOnWorkDone只是调用了OutputBufferQueue的holdBufferQueueBlocks方法。

void OutputBufferQueue::holdBufferQueueBlocks(
        const std::list<std::unique_ptr<C2Work>>& workList) {
    forEachBlock(workList,
                 std::bind(&OutputBufferQueue::registerBuffer,
                           this, std::placeholders::_1));

template <typename BlockProcessor>
void forEachBlock(C2FrameData& frameData,
                  BlockProcessor process) {
    for (const std::shared_ptr<C2Buffer>& buffer : frameData.buffers) {
        if (buffer) {
            // 获取C2GraphicBlock
            for (const C2ConstGraphicBlock& block :
                    buffer->data().graphicBlocks()) {
                process(block);
            }
        }
    }
}

template <typename BlockProcessor>
void forEachBlock(const std::list<std::unique_ptr<C2Work>>& workList,
                  BlockProcessor process) {
    for (const std::unique_ptr<C2Work>& work : workList) {
        if (!work) {
            continue;
        }
        for (const std::unique_ptr<C2Worklet>& worklet : work->worklets) {
            if (worklet) {
                forEachBlock(worklet->output, process);
            }
        }
    }
}

holdBufferQueueBlocks会调用forEachBlock,可以看到它会从C2Buffer中获取C2GraphicBlock,因此,只有video会使用 OutputBufferQueue。

bool OutputBufferQueue::registerBuffer(const C2ConstGraphicBlock& block) {
    // 1. 从C2ConstGraphicBlock获取_C2BlockPoolData
    std::shared_ptr<_C2BlockPoolData> data =
            _C2BlockFactory::GetGraphicBlockPoolData(block);
    if (!data) {
        return false;
    }
    std::scoped_lock<std::mutex> l(mMutex);
    // 2. 如果没有producer退出
    if (!mIgbp || mStopped) {
        return false;
    }

    uint32_t oldGeneration;
    uint64_t oldId;
    int32_t oldSlot;

    // 3. 从_C2BlockPoolData获取generation,producer id,和slot id
    if (!_C2BlockFactory::GetBufferQueueData(
            data, &oldGeneration, &oldId, &oldSlot) || (oldId == 0)) {
        return false;
    }
    // 4. 如果surface没有变化则进入if
    if ((oldId == mBqId) && (oldGeneration == mGeneration)) {
        // 5.
        _C2BlockFactory::HoldBlockFromBufferQueue(data, mOwner, getHgbp(mIgbp), mSyncMem);
        mPoolDatas[oldSlot] = data;
        mBuffers[oldSlot] = createGraphicBuffer(block);
        mBuffers[oldSlot]->setGenerationNumber(mGeneration);
        return true;
    }
    int32_t d = (int32_t) mGeneration - (int32_t) oldGeneration;
    return false;
}

registerBuffer会把C2ConstGraphicBlock注册到OutputBufferQueue中:

  1. 从C2ConstGraphicBlock获取_C2BlockPoolData,这部分参考 Android Codec2(二四)C2GraphicBlock;
  2. 判断是否有producer,如果没有直接退出,因此只有在video decoder且有surface的情况下才会使用OutputBufferQueue;
  3. 从_C2BlockPoolData获取generation,producer id,和slot id;
  4. 如果surface没有变化则进入真正的注册过程,首先调用HoldBlockFromBufferQueue,将mHeld置为true,表示client已经持有此buffer;然后将_C2BlockPoolData记录到mPoolDatas;最后使用C2ConstGraphicBlock创建GraphicBuffer并记录到mBuffers。

client端使用_C2BlockPoolData中的mHeld字段来判断buffer是由Codec2Client持有还是由NativeWindow持有。

接下往下看onWorkDone调用:

virtual void onWorkDone(
        const std::weak_ptr<Codec2Client::Component>& component,
        std::list<std::unique_ptr<C2Work>>& workItems) override {
    (void)component;
    sp<CCodec> codec(mCodec.promote());
    if (!codec) {
        return;
    }
    codec->onWorkDone(workItems);
}

实际调用的是CCodec的onWorkDone方法:

void CCodec::onWorkDone(std::list<std::unique_ptr<C2Work>> &workItems) {
    if (!workItems.empty()) {
        Mutexed<std::list<std::unique_ptr<C2Work>>>::Locked queue(mWorkDoneQueue);
        bool shouldPost = queue->empty();
        queue->splice(queue->end(), workItems);
        if (shouldPost) {
            (new AMessage(kWhatWorkDone, this))->post();
        }
    }
}

CCodec::onWorkDone会将C2Work加入到mWorkDoneQueue,然后由CCodec的消息处理线程来处理。这段处理比较长,我们分段阅读:

case kWhatWorkDone: {
    std::unique_ptr<C2Work> work;
    bool shouldPost = false;
    {
        Mutexed<std::list<std::unique_ptr<C2Work>>>::Locked queue(mWorkDoneQueue);
        if (queue->empty()) {
            break;
        }
        work.swap(queue->front());
        // 拿到一个C2Work
        queue->pop_front();
        shouldPost = !queue->empty();
    }
    if (shouldPost) {
        (new AMessage(kWhatWorkDone, this))->post();
    }

CCodec对输出的处理并没有使用while循环,因为除了kWhatWorkDone需要处理外,还有其他控制命令需要处理。从上面的代码可以看出每次处理完一个output,都会发一个kWhatWorkDone到队列的末尾,所以只要mWorkDoneQueue有元素,消息队列中就会有一条kWhatWorkDone消息。

接下来是对C2Work携带的信息进行解析,这段不做展开:

    // 1. 如果有输出且output没有被丢弃
    if (!work->worklets.empty()
            && (work->worklets.front()->output.flags
                    & C2FrameData::FLAG_DISCARD_FRAME) == 0) {

        // 2. 将output携带的信息拷贝到updates
        std::vector<std::unique_ptr<C2Param>> updates;
        for (const std::unique_ptr<C2Param> &param
                : work->worklets.front()->output.configUpdate) {
            updates.push_back(C2Param::Copy(*param));
        }
        // 3. 拿到output buffer
        unsigned stream = 0;
        std::vector<std::shared_ptr<C2Buffer>> &outputBuffers =
            work->worklets.front()->output.buffers;
        for (const std::shared_ptr<C2Buffer> &buf : outputBuffers) {
            for (const std::shared_ptr<const C2Info> &info : buf->info()) {
                // 4. 获取C2Buffer携带的信息
                updates.emplace_back(
                        C2Param::CopyAsStream(*info, true /* output */, stream));
            }

            const std::vector<C2ConstGraphicBlock> blocks = buf->data().graphicBlocks();
            if (!blocks.empty()) {
                const C2ConstGraphicBlock &block = blocks[0];
                updates.emplace_back(new C2StreamCropRectInfo::output(
                        stream, block.crop()));
            }
            ++stream;
        }
        // 5. 将信息更新到config中
        sp<AMessage> oldFormat = config->mOutputFormat;
        config->updateConfiguration(updates, config->mOutputDomain);
        RevertOutputFormatIfNeeded(oldFormat, config->mOutputFormat);

        stream = 0;
        const static C2Param::Index stdGfxInfos[] = {
            C2StreamRotationInfo::output::PARAM_TYPE,
            C2StreamColorAspectsInfo::output::PARAM_TYPE,
            C2StreamDataSpaceInfo::output::PARAM_TYPE,
            C2StreamHdrStaticInfo::output::PARAM_TYPE,
            C2StreamHdr10PlusInfo::output::PARAM_TYPE,  // will be deprecated
            C2StreamHdrDynamicMetadataInfo::output::PARAM_TYPE,
            C2StreamPixelAspectRatioInfo::output::PARAM_TYPE,
            C2StreamSurfaceScalingInfo::output::PARAM_TYPE
        };
        // 6. 检查video流信息
        for (const std::shared_ptr<C2Buffer> &buf : outputBuffers) {
            if (buf->data().graphicBlocks().size()) {
                for (C2Param::Index ix : stdGfxInfos) {
                    if (!buf->hasInfo(ix)) {
                        const C2Param *param =
                            config->getConfigParameterValue(ix.withStream(stream));
                        if (param) {
                            std::shared_ptr<C2Param> info(C2Param::Copy(*param));
                            buf->setInfo(std::static_pointer_cast<C2Info>(info));
                        }
                    }
                }
            }
            ++stream;
        }
    }
    if (config->mInputSurface) {
        if (work->worklets.empty()
                || !work->worklets.back()
                || (work->worklets.back()->output.flags
                        & C2FrameData::FLAG_INCOMPLETE) == 0) {
            config->mInputSurface->onInputBufferDone(work->input.ordinal.frameIndex);
        }
    }
    if (initDataWatcher.hasChanged()) {
        initData = initDataWatcher.update();
        AmendOutputFormatWithCodecSpecificData(
                initData->m.value, initData->flexCount(), config->mCodingMediaType,
                config->mOutputFormat);
    }
    outputFormat = config->mOutputFormat;

信息获取完成后调用CCodecBufferChannel::onWorkDone:

mChannel->onWorkDone(
        std::move(work), outputFormat, initData ? initData.get() : nullptr);

2、CCodecBufferChannel::onWorkDone

void CCodecBufferChannel::onWorkDone(
        std::unique_ptr<C2Work> work, const sp<AMessage> &outputFormat,
        const C2StreamInitDataInfo::output *initData) {
    if (handleWork(std::move(work), outputFormat, initData)) {
        feedInputBufferIfAvailable();
    }
}

onWorkDone需要传入三个参数:

  • work:输出数据;
  • outputFormat:更新后的输出格式;
  • initData:这个应该是给encoder用的,编码后的csd数据;

onWorkDone会调用handleWork对output进行处理,如果返回true还会调用feedInputBufferIfAvailable。接下来先看handleWork是如何处理的,这个方法非常长,我们分段阅读。

if (work->result == C2_OK){
    notifyClient = true;
} else if (work->result == C2_NOT_FOUND) {
    ALOGD("[%s] flushed work; ignored.", mName);
} else {
    ALOGD("[%s] work failed to complete: %d", mName, work->result);
    mCCodecCallback->onError(work->result, ACTION_CODE_FATAL);
    return false;
}

首先检查C2Work的result字段,如果不是C2_OK或者C2_NOT_FOUND,直接上抛error。

if ((work->input.ordinal.frameIndex -
        mFirstValidFrameIndex.load()).peek() < 0) {
    // Discard frames from previous generation.
    ALOGD("[%s] Discard frames from previous generation.", mName);
    notifyClient = false;
}

3、extraBuffers


原文阅读:
Android Codec2(三三)组件处理与数据输出 - Ⅰ

扫描下方二维码,关注公众号《青山渺渺》阅读音视频开发内容。

Android中,编解码6K视频可以通过使用CCodec来实现。CCodecAndroid Q以后的版本采用的一种加载解码插件的方式。在CCodec中,可以通过添加相应的信息到kComponents列表中来支持自定义解码库。这些信息包括组件名称、软件编解码库后缀名和组件角色。在NuPlayer中,解码器由NuPlayer::Decoder进行抽象。在NuPlayer开始后,首先完成了IMediaSource的开始,然后通过异步消息kWhatScanSources开始执行NuPlayer::instantiateDecoder()实例化解码器。对于音频解码器,会创建DecoderPassThroughDecoder,而对于视频解码器,只会创建Decoder。在Decoder的configure()方法中,会通过MediaCodec::CreateByType()创建MediaCodec,并通过MediaCodecList::findMatchingCodecs()查找支持当前解码格式的解码器的名字。创建MediaCodec时,会初始化mGetCodecBase为一个std::function对象,后续的MediaCodec::init()会调用此lambda函数来创建更底层的CodecBase,其中包括CCodec的创建。因此,通过使用CCodec和相关的组件,可以实现Android上对6K视频的编解码。\[1\]\[2\]\[3\] #### 引用[.reference_title] - *1* *3* [Android 12 原生播放器的编解码 Codec 2](https://blog.youkuaiyun.com/liyangzmx/article/details/124582754)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^control_2,239^v3^insert_chatgpt"}} ] [.reference_item] - *2* [Android 系统软解码方案实现](https://blog.youkuaiyun.com/shijiheike/article/details/130107438)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^control_2,239^v3^insert_chatgpt"}} ] [.reference_item] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

青山渺渺

感谢支持

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值