Camera-MTK 不同apk调用Camera Device_mtk 算法app调用-优快云博客

本文链接：https://blog.youkuaiyun.com/Cmatrix204/article/details/123667830

手机中除了单独的Camera apk会使用到Camera Device之外，有很多其他的场景会使用到Camera Device，比如欧美市场的Google Len、Whatsapp、以及短信发送里面的拍照、拍视频功能，这些都可以看做不同的apk使用系统底层的Camera Device，这样的情况下如何去管控不同apk调用Camera Device以及不同三方不开源apk Feature切换之前造成的差异问题，也会造成不小的困扰，分享一下利用Camera架构管控不同不开源apk调用Camera的问题。

1. FrameWork层 - check调用camera Device的不通场景

在OpenCamera阶段，FrameWork层相关接口获取调用open的Activity是比较容易的，但是CameraService、HAL层想拿到这个Activity的信息然后进行针对不开源apk的操作就比较难了，就涉及到能够在FrameWork层、CameraService、HAL之间能传递信息的传递量。我选择了使用MTK 传递captureRequest的metadata完成这个功能。

CameraManager.java里面openCameraDeviceUserAsync接口有获取Camera User的相关操作，mContext.getOpPackageName()可以获取，具体代码如下：

/**
      * Helper for opening a connection to a camera with the given ID.
      *
      * @param cameraId The unique identifier of the camera device to open
      * @param callback The callback for the camera. Must not be null.
      * @param executor The executor to invoke the callback with. Must not be null.
      * @param uid      The UID of the application actually opening the camera.
      *                 Must be USE_CALLING_UID unless the caller is a service
      *                 that is trusted to open the device on behalf of an
      *                 application and to forward the real UID.
      *
      * @throws CameraAccessException if the camera is disabled by device policy,
      * too many camera devices are already open, or the cameraId does not match
      * any currently available camera device.
      *
      * @throws SecurityException if the application does not have permission to
      * access the camera
      * @throws IllegalArgumentException if callback or handler is null.
      * @return A handle to the newly-created camera device.
      *
      * @see #getCameraIdList
      * @see android.app.admin.DevicePolicyManager#setCameraDisabled
      */
     private CameraDevice openCameraDeviceUserAsync(String cameraId,
             CameraDevice.StateCallback callback, Executor executor, final int uid)
             throws CameraAccessException {
         CameraCharacteristics characteristics = getCameraCharacteristics(cameraId);
         CameraDevice device = null;
 
         synchronized (mLock) {
 
             ICameraDeviceUser cameraUser = null;
 
             //创建CameraDeviceImpl
             android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                     new android.hardware.camera2.impl.CameraDeviceImpl(
                         cameraId,
                         callback,
                         executor,
                         characteristics,
                         mContext.getApplicationInfo().targetSdkVersion);
 
             ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();
 
             try {
                 if (supportsCamera2ApiLocked(cameraId)) {
                     // Use cameraservice's cameradeviceclient implementation for HAL3.2+ devices
                     ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
                     if (cameraService == null) {
                         throw new ServiceSpecificException(
                             ICameraService.ERROR_DISCONNECTED,
                             "Camera service is currently unavailable");
                     }
                     
                     //mContext.getOpPackageName() check调用Camera User的package Name
                     cameraUser = cameraService.connectDevice(callbacks, cameraId,
                             mContext.getOpPackageName(), uid);
                 } else {
                     // Use legacy camera implementation for HAL1 devices
                     int id;
                     try {
                         id = Integer.parseInt(cameraId);
                     } catch (NumberFormatException e) {
                         throw new IllegalArgumentException("Expected cameraId to be numeric, but it was: "
                                 + cameraId);
                     }
 

                     Log.i(TAG, "Using legacy camera HAL.");
                     cameraUser = CameraDeviceUserShim.connectBinderShim(callbacks, id,
                             getDisplaySize());
                 }
             } catch (ServiceSpecificException e) {

                ...
             }
    ...
}

Open阶段获取到其他不开源的三方apk打开了Camera Device，但是仅靠open阶段是没法把这个三方apk的信息传到CameraService、HAL层的，所以借助CameraDeviceImpl完成三方信息的下发动作。

//在open阶段获取打开Camera Device不开源apk的信息
private CameraDevice openCameraDeviceUserAsync(String cameraId,
             CameraDevice.StateCallback callback, Executor executor, final int uid)
             throws CameraAccessException {

    ...

                        if(mContext.getOpPackageName().contains("com.whatsapp")) {
                                 deviceImpl.isWhatsapp = true;
                         }
                         else {
                                 deviceImpl.isWhatsapp = false;
                         }

    ...
}

CameraDeviceImpl createCaptureRequest接口可以在任意下发预览、拍照请求阶段将三方apk信息传给HAL。

public boolean isWhatsapp = false;

@Override
     public CaptureRequest.Builder createCaptureRequest(int templateType)
             throws CameraAccessException {
         synchronized(mInterfaceLock) {
             checkIfCameraClosedOrInError();
 
             CameraMetadataNative templatedRequest = null;
 
             templatedRequest = mRemoteDevice.createDefaultRequest(templateType);
 
             // If app target SDK is older than O, or it's not a still capture template, enableZsl
             // must be false in the default request.
             if (mAppTargetSdkVersion < Build.VERSION_CODES.O ||
                     templateType != TEMPLATE_STILL_CAPTURE) {
                 overrideEnableZsl(templatedRequest, false);
             }
 
              //发送CaptureRequest - CONTROL_PACKAGENAME
                         if(isWhatsapp)
                         {
                             templatedRequest.set(CaptureRequest.CONTROL_PACKAGENAME, CaptureRequest.CONTROL_PACKAGENAME_WHATSAPP);
                         }
                         else
                         {
                             templatedRequest.set(CaptureRequest.CONTROL_PACKAGENAME, CaptureRequest.CONTROL_PACKAGENAME_DEFAULT);
                         }
 
             CaptureRequest.Builder builder = new CaptureRequest.Builder(
                     templatedRequest, /*reprocess*/false, CameraCaptureSession.SESSION_ID_NONE,
                     getId(), /*physicalCameraIdSet*/ null);
 
             return builder;
         }
}

2.HAL层 - 解析传入的CaptureRequest判断三方apk

HAL层可以在parseMeta接口里面获取由FrameWork层传下来的capture Request值，进而判断是哪个三方apk进入处理相关的针对性方案。

在处理WhatsApp下发的MTK_CONTROL_AF_TRIGGER请求时，进行特殊的处理以解决过曝问题。

vendor/mediatek/proprietary/hardware/mtkcam/aaa/source/common/hal3a/v1.0/Hal3AAdapter3.cpp

MUINT8
Hal3AAdapter3::parseMeta(const vector<MetaSet_T*>& requestQ) {

    ...

    MINT32 i4IsoSpeedMode = 0;
     if(QUERY_ENTRY_SINGLE(_halmeta, MTK_HAL_REQUEST_ISO_SPEED, i4IsoSpeedMode)) // Hal1
     {
         MY_LOGD_IF(mParams.i4IsoSpeedMode != i4IsoSpeedMode, "[%s] MTK_HAL_REQUEST_ISO_SPEED(%d -> %d)", __FUNCTION__, mParams.i4IsoSpeedMode, i4IsoSpeedMode);
         mParams.i4IsoSpeedMode = i4IsoSpeedMode;
     }
     else if(QUERY_ENTRY_SINGLE(_appmeta, MTK_3A_FEATURE_AE_REQUEST_ISO_SPEED, i4IsoSpeedMode)) // Hal3
     {
         MY_LOGD_IF(mParams.i4IsoSpeedMode != i4IsoSpeedMode, "[%s] MTK_3A_FEATURE_AE_REQUEST_ISO_SPEED(%d -> %d)", __FUNCTION__, mParams.i4IsoSpeedMode, i4IsoSpeedMode);
         mParams.i4IsoSpeedMode = i4IsoSpeedMode;
     }

    
     //get FrameWork层下发的相关CaptureRequest     
     MBOOL IswhatsApp = MFALSE;
     MUINT8 packagename_value = 0;
     if(QUERY_ENTRY_SINGLE(_appmeta, MTK_CONTROL_PACKAGENAME, packagename_value)) // Hal1
     {
                 if(packagename_value == 1)
                 {
                         IswhatsApp = MTRUE;
                 }
                 MY_LOGD_IF(fgLogEn0, "[%s] MTK_CONTROL_PACKAGENAME_VALUE %d, IswhatsApp : %d", __FUNCTION__, packagename_value,IswhatsApp);
     }
     else
     {
                 MY_LOGD_IF(fgLogEn0, "[%s]  fail to get MTK_CONTROL_PACKAGENAME_VALUE %d", __FUNCTION__, packagename_value);
     }
         
    
    ...


     if ( (!u1RepeatTag) || ReparseMetaForDummy) // not repeating tag, parse app meta
     {
         AAA_TRACE_HAL(ParseAppMeta);
         mUpdateMetaResult.clear();
         for (MUINT32 j = 0; j < _appmeta.count(); j++)
         {
             IMetadata::IEntry entry = _appmeta.entryAt(j);
             mtk_camera_metadata_tag_t tag = (mtk_camera_metadata_tag_t)entry.tag();
 
             // convert metadata tag into 3A settings.
             switch (tag)
             {

                ...

                case MTK_CONTROL_AF_TRIGGER:
                 {
                     MUINT8 AFTrigger = entry.itemAt(0, Type2Type< MUINT8 >());
                     MY_LOGD_IF(mAfParams.u1AfTrig != AFTrigger, "[%s] MTK_CONTROL_AF_TRIGGER(%d -> %d)", __FUNCTION__, mAfParams.u1AfTrig, AFTrigger);
                     mAfParams.u1AfTrig = AFTrigger;
                          
                    
                     针对WhatsAopp使用Camera过程中打开闪光灯过曝问题进行处理               
                     //only WhatsApp ,need change AF cancel to idle,avoid do AF cancel per-frame
                     if (AFTrigger == MTK_CONTROL_AF_TRIGGER_CANCEL && mu4LastAfTrig == AFTrigger && IswhatsApp){
                      MY_LOGD_IF(fgLogEn0, "[%s]  only WhatsApp ,need change AF cancel to idle,avoid do AF cancel per-frame", __FUNCTION__);
                      mAfParams.u1AfTrig = MTK_CONTROL_AF_TRIGGER_IDLE;
                      mu4LastAfTrig = MTK_CONTROL_AF_TRIGGER_CANCEL;
                    }else{
                           mu4LastAfTrig =  AFTrigger;
                    }
                                                                                                                           
                }

                 break;
                 case MTK_CONTROL_AF_TRIGGER_ID:
                 {
                     MINT32 i4AfTrigId = entry.itemAt(0, Type2Type< MINT32 >());
                     MY_LOGD_IF(fgLogEn1, "[%s] MTK_CONTROL_AF_TRIGGER_ID(%d)", __FUNCTION__, i4AfTrigId);
                 }
                 break;
 
             // Lens
                 case MTK_LENS_FOCUS_DISTANCE:
                 {
                     MFLOAT fFocusDist = entry.itemAt(0, Type2Type< MFLOAT >());
                     MY_LOGD_IF(mAfParams.fFocusDistance != fFocusDist, "[%s] MTK_LENS_FOCUS_DISTANCE(%3.6f -> %3.6f)", __FUNCTION__, mAfParams.fFocusDistance, fFocusDist);
                     mAfParams.fFocusDistance = fFocusDist;
                 }
                 break;
 

                 case MTK3_DUALCAM_CALIB_FACTORY_MODE:
                 {
                     MINT32 factory_mode = entry.itemAt(0, Type2Type< MINT32 >());
                     MY_LOGD("[%s]MTK3_DUALCAM_CALIB_FACTORY_MODE --> %d", __FUNCTION__, factory_mode);
                     BSTBokehParam* pBokehParam = BSTBokehParam::createInstance();
                     pBokehParam->setBstFactorymode(factory_mode);
                 }
                 break;

                 ...
               }


        ...

}

总结：

一不小心就2022了，一个项目上新的GPU渲染美颜、滤镜算法，从年前搞到现在，才功能OK，最近再加上疫情才有时间对以前处理过的case进行整理，新的一年会就简单的以一个小问题的解决作为开端吧，新的一年里会主要利用优快云对工作中解决的问题进行整理，大的Camera框架梳理可能会做的比较少，希望在新的一年里用优快云记录一下小白的成长吧！

2022.3.21 东航MU5735，民航人坚持的4227天的安全记录被打破了，祝各位平平安安，人生遂意吧，希望中国民航 “虽低却不惶恐，旦高仍不自傲”，2022为了下一个坚守，冲鸭！！！